The remark made by Mustafa, a former state minister, at a news conference in Kochi yesterday also figured at a meeting of the Congress party's state executive in Thiruvananthapuram to review the party's performance in the state.
The Kerala state youth Congress is also reported to have demanded action against Mustafa who blamed Rahul for the party's debacle in Lok Sabha polls. Congress got 44 seats.
"He (Rahul) should resign from the party. He should not continue. If he doesn't resign voluntarily, he should be removed. He acted like a joker(during the poll campaign)....," Mustafa had said.
The Congress and its UDF allies have done comparatively well in Kerala well bagging 12 of the 20 seats at stake.
However, the party suffered some glaring losses with the LDF wresting its traditional strongholds like Idukki and Chalakkudi.
Senior Congress leader and former union minister Mani Shankar Aiyar disagreed with Mustafa's assessment.
"I do not agree with what a senior Kerala leader said about Rahul," he said in Nagpur.
On change of party leadership, Aiyar said, "Congress had (fared) badly previously in regimes of prime ministers Indira Gandhi, Rajiv Gandhi and also under P V Narasimha Rao. But at no time did the question of change of leadership crop up".
Backing Sonia and Rahul, Aiyar questioned whether parties like SP, BSP, who too got bruised in polls, have changed their top leadership
"There is no question of changing Congress leadership of Sonia Gandhi or Rahul Gandhi. Samajwadi Party chief Mulayam Singh, BSP supremo Mayawati and Akali Dal chief Parkash Singh Badal's party also fared poorly and they have quickly removed their aides.
"But have these leaders quit or is there any demand for their removal," Aiyar asked.
Congress spokesperson Rashid Alvi condemned the remarks and said they were made out of frustration.
Another Congress spokesperson Meem Afzal sidestepped Mustafa's comment and asked party leaders to raise issues internally.
Former Minority Affairs Minister K Rehman Khan also said the issues should be raised in the party forum.
"Who is responsible and what has happened is a matter to be investigated. Congress president and Vice President have said that they have taken the responsibility. It does not mean that you go on criticizing them. Going public with your views is against party discipline," he said.