The dress-conscious Modi was attired in all-white kurta pyjama, sleeveless jacket and white sandals when he visited the temples, apparently to send out a message of solemnness of the occasion.
In Kinkakuji, the Prime Minister mingled with tourists and visitors, shook hands, pulled ears of a child and posed for photographs with groups of people.
He began his day with a visit to the ancient Toji Temple, which is inspired by the trinity of Brahma, Vishnu, Mahesh of Hindu philosophy.
He was accompanied by his Japanese counterpart Shinzo Abe when he went around the famous temple which is a UNESCO world heritage site.
Modi was at the temple complex for about half an hour during which he enquired about the history of the eighth-century Buddhist pagoda.
Chief Monk Mori took the Prime Minister around the complex having wooden temple including the five-storied main pagoda.
"I am Modi, you are Mori," he told the chief priest Yasu Nagamori jokingly after reading name on his Identity card.
The main pagoda stands 57 meters tall, making it the tallest pagoda in Japan, and has become a symbol of both the temple and Kyoto as it can be seen from many places across the city.
While leaving the complex, Modi thanked Abe for accompanying him to the temple and spending time with him.
Abe, on his part, told Modi that this was only the second time that he had visited Toji temple, the last being during his student days.
The Japanese Prime Minister had specially come here from Tokyo to meet Modi and be with him.
Monk Hasi, who also accompanied Modi at Toji temple, said: "We are happy that the Prime Minister visited us. It is a matter of pride for our temple. He has a big heart".
Earlier, a number of Indians carrying the Tricolour had gathered at the temple to see Modi, who went and shook hands with the excited crowd.
After Toji, Modi went to gold leaf-coated Kinkakuji temple. The current form of the temple was unveiled in 1955, five years after the 14th-century original was torched by one of the temple's monks.
He offered prayer at the Buddhist temple and then went around the lake and garden surrounding it. He also enquired about the history of the temple.
While taking a round of the complex, he mingled with visitors and posed for photographers with some of them.
He also surprised a child of around 10 years by suddenly pulling his ears and photographers obliged by clicking it.
Foreigners visiting the temple were excited at seeing the Indian Prime Minister there. Many were seen clicking him with their mobile phones.
"We have come at the right time," an American tourist was heard telling his companion after learning that Modi is also there.
A group of women Japanese visitors, with whom Modi posed for photographs, were very excited by his gesture.