Clint Dempsey's early goal gave the U.S. the perfect start and, after Andre Ayew equalised for Ghana, John Brooks powered home an 86th-minute header to grab the win and maintain the coach's 100 percent record in World Cup openers.
The 49-year-old, a World Cup winner with West Germany in 1990, said his captain's strike, officially timed at 29 seconds after kickoff, had perhaps come too early.
"The whole bench was jumping around," he said. "If you score after just a minute you think there can't be anything better than that.
"After that it maybe wasn't so good because we sat back and let them come to us, that is why I was screaming on the sideline to take control of the game. We had problems controlling the ball and making passes."
Klinsmann said he always believed the U.S. would win even when Ghana equalised but knows they will have to improve some aspects of their game for the Group G games with Portugal and Germany.
"It was a very enjoyable moment when the goal came, when a late goal like this happens then you see a bench jumping up full of joy and excitement," he said.
"(But) the match showed us a lot of areas we have to do better for the second game. We gave too much of the game to Ghana. We will take all those pieces and analyse them and show the players."
The former Germany striker played in three World Cups, helping his team start the tournament by beating Yugoslavia 4-1 in 1990, Bolivia 1-0 in 1994 and the United States 2-0 in 1998.
Klinsmann also coached Germany to a 4-2 victory over Costa Rica in their first game of the 2006 tournament on home soil.
Goalscorer Dempsey claimed a little bit of tournament history for himself at the Dunas arena, becoming the first American to score at three World Cups.
The 31-year-old ended the contest with a suspected broken nose after taking a blow to the face from a Ghanaian boot but he was not going to get much sympathy from his coach.
"I broke mine three or four times," said Klinsmann. "It's alright. He'll be alright in a few days ..."