Higuain buried a shot into the corner of the net in the eighth minute at the national stadium to set up a semi-final clash against 2010 runners-up the Netherlands or surprise package Costa Rica, who play later on Saturday.
The 26-year-old Higuain reacted brilliantly when an Angel Di Maria pass was deflected towards him off Belgium defender Jan Vertonghen. He swivelled and struck the ball first time as Argentina ended a run of being eliminated in the last eight at the two most recent editions of soccer's global showpiece.
"I knew the goal was going to come and it came at an important time," Higuain, who had not scored for his nation in six matches, said in a television interview.
"We were able to reach a semi-final which we haven't achieved for many years. Now we need to play well and try and reach the final," he added.
"We gave our all, it's a World Cup, it doesn't happen every day. There are two matches left to achieve this beautiful dream."
Belgium, whose best World Cup run was in 1986 when they were knocked out by Diego Maradona's Argentina in the semi-finals, had a couple of chances but otherwise offered little in attack despite sustained late pressure that set nerves among the thousands of Argentines thronging the giant arena jangling.
Argentina looked the more ambitious in the early stages and Higuain's beautifully executed goal gave them a deserved lead.
They came close to a second in the 28th minute when Lionel Messi, who was a constant threat to the Belgium defence having set the tournament alight in the group stages, threaded a superb ball through for a galloping Di Maria.
However, the winger injured himself when his effort was blocked and was replaced by Enzo Perez a few minutes later.
Belgium were starting to enjoy more possession without really threatening before Messi flashed a free kick narrowly wide five minutes before the break.
With Argentina apparently happy to sit on their lead, Belgium's best effort of the half came in the 42nd minute when Kevin Mirallas sent a diving header wide from Vertonghen's centre.
Higuain was in fine form and raced through the centre 10 minutes into the second half but, with Belgium goalkeeper Thibaut Courtois beaten, his shot clipped the top of the crossbar and the ball flew into the crowd.
Marouane Fellaini sent a 61st-minute header just over the bar in another good chance for Belgium and there was some desperate defending from the Argentines in the closing minutes.
But they held on and captain Messi could even afford to squander a late one-on-one with Courtois as his side progressed along with hosts Brazil and Germany, who went through on the other side of the draw on Friday.
"It's a great joy for this squad, for the technical staff, for the people, for my family. We all crossed the Rubicon!," said Argentina coach Alejandro Sabella, adding that it was probably his team's finest performance so far in Brazil.