Costa Rica, who were reduced to 10-man in the second half, were put ahead by Bryan Ruiz in the 52nd minute but a late equaliser by Sokratis Papastathopoulos early in stoppage time rescued Greece as the match headed to extra time.
The two teams, however, could not be separated even in the extra time and finally Costa Rica sealed their maiden quarterfinal berth when Michael Umana converted his team's fifth penalty after Fanis Gekas missed Greece's fourth at
Arena Pernambuco. Celso Borges, Bryan Ruiz, Giancarlo Gonzalez, Joel Campbell and Umana converted all the five penalties for Costa Rica, while Konstantinos Mitroglou, Lazaros Christodoulopoulos and Jose Holebas scored for Greece before Costa Rica goalkeeper Navas made a brilliant save from Theofanis Gekas.
Costa Rica players broke into a dance amid cheering from their fans but the Greece players were left teary-eyed following their exit from the World Cup.
At Fortaleza, Klaas Jan Huntelaar scored an injury-time penalty to send the Netherlands into the quarterfinals with a hard-fought 2-1 win over Mexico.
Arjen Robben was adjudged to have been tripped in the area by Rafa Marquez in the injury time and Huntelaar coolly placed a brilliant penalty into the bottom-left corner to knock Mexico out of the tournament.
However, it was Wesley Sneijder who rescued the 2010 runners-up, in the 88th minute, bringing up the equaliser, cancelling out the 48th minute brilliant goal by Giovani Dos Santos which had put Mexico ahead.
Mexicans had dominated the match and looked destined to reach their first quarter-finals since 1986 but Sneijder's effort two minutes from time and Huntelaar's penalty in added time crushed their dreams.
In a battle of attrition, neither Greece nor Costa Rica could break the deadlock in the first half of the second match of the day.
Greece, playing their first ever game in the knockout stages of the World Cup, could have gone ahead at the 37th minute when Dimitrios Salpingidis almost sent Jose Holebas' cross home but goalkeeper Keylor Navas stuck his leg out in time to make a good save.
Costa Rica too had a chance to go one-up when Christian Bolanos penetrated the left side of the penalty area in the eighth minute but his shot went over the crossbar.
Seven minutes into the second half, Costa Rica finally broke Greece's defensive when Bryan Ruiz found the bottom right corner of the net with a left-footed shot from the edge of the box after being set up by Cristian Bolanos.
Costa Rica, who stunned Uruguay and Italy before drawing with England to finish top of Group D, were reduced to a 10-men team in the 66th minute when Oscar Duarte was sent off for a second yellow.
Desperate for the equaliser, Greece coach Fernando Santos made few substitution changes, one of them was replacing Salpingidis with Theofanis Gekas. The experienced striker got into action soon when he tried to steer a low cross from Holebas but Michael Umana made a important block.
Greece created an opportunity in the 88th minute when Lazaros Christodoulopoulos ran in from the right but goalkeeper Navas was up for it.
Sokratis Papastathopoulos finally fired in the equaliser for Greece in the second minute of stoppage time when the centre back found the back of the net on the rebound after goalkeeper Navas had initially palmed away Gekas' effort successfully from close range.
In the extra time too, goalkeeper Navas came up with some brillant saves to keep Costa Rica in the hunt.
Playing in the knockout stages for a sixth straight World Cup, Mexico dominated the proceedings right from the start under hot and humid conditions at Fortaleza and had four shots at the target in the first half itself but none of them enough to put them ahead.
Herrera came close early on after being fed by Peralta but his effort went wide of the bottom-left post, while a long shot by Salcido was easily parried away by Cillessen.
Holland, who struggled to keep possession early on, got a decent chance in the 28th minute when skipper Robin Van Persie, who is returning into the team after a suspension, brought a lofted ball down in the penalty area but he failed to connect properly as his right-footed effort went wide.
Mexico tested the Dutch backline once again in the next minute when Santos played a pass into Peralta, who received the ball and fired it at Cillessen from 25 yards.
In the 42nd minute, Mexico had another opportunity when Santos raced into the rival box and found Andres Guardado's pass but his acute angle shot was blocked by Cillessen.
In the stoppage time, Holland got a rare chance when Van Persie pounced on the ball after a lapse by the Marquez and raced towards the penalty area with Arjen Robben in support but the Bayern Munich winger played it late.