"Sonia had made P V Narasimha Rao Prime Minister. But she wasn't very fond of him. I, too, had fallen out with him and joined the Tiwari Congress, but we later made up," Singh recalls in his just-released book "One Life Is Not Enough" published by Rupa.
Rao had sought the services of Singh, then a close Gandhi family friend, in December 1994 to repair his relation with Gandhi.
"In December, 1994, when my relations with him (Rao) had been repaired, he asked me to see him at his house, 5 Race Course Road. He seemed uncharacteristically agitated and restless.
"He said, 'I can take on Sonia Gandhi. But I do not want to do so. Some of her advisers have been filling her ears against me. I don't take them seriously. Sonia's case is different. Her attitude towards me is affecting my health. If she wants me to go, she only has to say so.
"I have done my best to meet all her desires and requirements promptly. You worked closely with her and must know and should know why Sonia is so hostile to me'," Singh writes in his book.
Natwar Singh records, "An impression was created by one or two senior members of the CWC that she was not happy with the reform process and that Rao was ignoring her. Almost all the senior members of the Congress Party were aware of Gandhi giving him the cold shoulder.
"In the months to come, a stage was reached when communication between 7 RCR (PM's residence) and 10 Janpath almost ceased," he writes.
Rao told Singh that he went out of his way to be polite to her and the government did whatever she wished but she never even telephoned him.
He recalled how she turned down his offer of installing a RAX phone (dedicated telephones available only to ministers and top officials) at her residence so that she could speak directly to him after initially agreeing to the idea. "It was like a slap on my face," Rao had told Natwar Singh.
Though Rao was keen on a patch-up with Gandhi, Singh says it did not happen.
Rao even roped in Mohammad Yunus, a trusted Gandhi aide, to smoothen his relation with Gandhi but it did not work.
Things had soured between them to the extent that when there was intelligence report of a likely attack on her by a Khalistani terror group in Kerala in September 1995, where she was to attend a conference, Rao did not speak to her directly and instead asked Singh to tell her to cancel her trip, Singh writes.
Gandhi still decided to go ahead with her trip and Rao flew commandos and the venue became a mini fortress.
What was really the reason behind Gandhi's coldness to Rao has not been spelt out in the book but Singh says one of the things was that she was upset over the slow pace of trial of Rajiv Gandhi's killers.
"P V's reply was that he had sent P Chidambaram (Minister) to her with the necessary papers. He had also sent Home Minister S B Chavan to brief her... He said he had himself gone with the necessary files and explained to her the legal difficulties in hastening the trial. According to him, she had listened and said nothing," Singh writes.
Singh, however, notes that the concerted efforts of Rao baiters within Congress did not succeed because "he was far cleverer than all of us".
The former minister, who had joined the breakaway Congress faction formed by likes of perceived Gandhi loyalists N D Tiwari and Arjun Singh before patching up with Rao, describes Rao as a man of learning, a scholar and a thinker of the first order.
Though he adds that Rao was no saint and his private life was inclined towards "passion and sensuality".
"He was astute, crafty and patient but also capable of radioactive sarcasm. He smiled without a smile," Singh writes.
Natwar Singh recalls that in the 1996 Lok Sabha elections every candidate of the Tiwari Congress other than Sis Ram Ola lost the elections.
"Some senior members had hoped that the party had or would have the support of Sonia Gandhi. But Ms Gandhi did not oblige and remained aloof from all political activities. I
remember Arjun Singh telling me, 'This is the reward for loyalty'," he says.
Rao, Singh recalls, faced the most serious crisis of his tenure on 6 December, 1992-the Babri Masjid in Ayodhya was demolished by unruly Hindu karsevaks. The UP and the Central governments looked helplessly on and the world saw the demolition live on television.
"The seeds for Hindutva had been sown. PV had lost his nerve and came out of the whole sad episode with his fingers burnt. His indecisiveness, at such a time, did him no credit."
Despite his personal fondness for Rajiv Gandhi, Singh says the late prime minister "mishandled" the fallout of the Shah Bano case, the Ram Janmabhoomi issue and the agitation in Darjeeling hills.
He also faulted him over his handling of the Bofors kickback controversy.
"I felt then, and still do, that the Prime Minister could have handled the matter in a more nuanced manner... He could have exercised nuanced restraint. This he did not do.
Instead, he plunged into the Bofors mud. Some of it stuck," he writes in his autobiography 'One life is not enough'.
Rajiv's frequent reshuffling of his council of ministers--there were over two dozen of them and the only Cabinet minister to have completed five-year tenure was Railway Minister Madhavrao Scindia--meant they were not able to settle down in their jobs or offer any long-term policy proposals.
Rajiv, says Singh, was initially in support of Supreme Court ruling in Shah Bano case despite the anger of Muslim clerics and put up Arif Mohammad Khan, a junior minister, to defend the ruling. He, however, soon changed his mind and Khan resigned.
"The countrywide reaction was that the Prime Minister had not handled the issue well," writes Singh, a member of Rajiv's ministry.
The Babri Masjid-Ram Janmabhoomi issue surfaced after locks placed on the site were taken away and worship begun, allegedly at the behest of Gandhi's close aide Arun Nehru, and the issue culminated in the BJP leader L K Advani's rath yatra and the mosque's demolition, he writes.
Singh, however, praises Rajiv for "changing the mindset of a large number of Indians" and preparing the country for 21st century.
"What he achieved during his five-year term was quite remarkable," he writes, showcasing his China visit as a high point of his prime ministership.
Though Rajiv was too trusting of people, he said, adding that he was also "the most natural, charming and least pompous man I had met".
Singh writes that during the first 18 months of his tenure Gandhi depended wholly on "a team of ignoramuses with inflated egos".
In an interview, he had identified two of them as Gopi Arora and Arun Nehru, both dead, but declined to name the third one as he was very old and the matter was not so important now.
Gandhi's friend and ministerial colleague Arun Singh had kept him in dark about 'Operation Brasstacks', which brought India and Pakistan close to war, which finally led to his sack, says Singh.