In a brief statement by way of clarification, Shinde, who is also the Leader of the Lok Sabha, said his comment has created a "misunderstanding" and that he had no intention to link terrorism with any religion.
"Since controversy has been created on account of my statement, I am issuing this clarification and expressing regret to those who felt hurt by my statement," he said.
"My statement given in Jaipur last month has created a misunderstanding. It has been understood to mean that I was linking terrorism to a particular religion and was accusing certain political organisations of being involved in organising terror camps.
"I had no intention to link terror to any religion. There is no basis for suggesting that terror can be linked to organisations mentioned in my brief speech in Jaipur," he added.
Shortly after Shinde expressed regret, BJP welcomed it but said it ought to have come much earlier.
"The delay had created a condition which gladdened the hearts of terrorists and their patrons in Pakistan," BJP chief spokesman Ravi Shanker Prasad said.
Shinde had made the controversial remark at Jaipur exactly a month ago at the Congress' Chinthan Shivir raising the hackles of the principal Opposition party and the Sangh Parivar.
Earlier in the day, Shinde along with Parliamentary Affairs Minister Kamal Nath had a meeting with Leader of the Opposition in Lok Sabha and BJP leader Sushma Swaraj after which government managers worked overtime to formulate a statement which was acceptable to both sides.
The Home Minister has been under severe attack from BJP which had promised to rake up the issue in the Budget session beginning tomorrow.
"I will continue to perform my duties to the best of my ability to ensure that harmony is maintained in the social fabric of India," Shinde said.
Shinde's statement came on a day when BJP took to the streets in Delhi to protest against his remarks.
These statements from Shinde comes in the wake of PM Manmohan Singh imploring the Opposition to let Parliament function and is definitely meant to assuage the feelings of the BJP and the other saffron parties who had expressed outrage at being linked to terror.
Indian Prime Minister Manmohan Singh pleaded with opposition parties on Wednesday to allow parliament to function during a session that begins this week, when the government plans to present the annual budget and legislation on economic reform.
The last two sessions of India's federal parliament have been disrupted by opposition members protesting against corruption, reforms and affirmative action policies, although the winter session in December finally managed to conduct business for a few days.
The month-long monsoon session in August and September passed only four bills out of 30 due to be debated.
"It is our sincere hope that we will have a productive, constructive debate leading to agreed solutions to the many national problems that our country faces," Singh told reporters after addressing parliamentary leaders from all parties.
"Parliament is a forum for discussion, for dialogue and all parties have an obligation to ensure that parliament runs smoothly."
Singh is grappling with the worst economic slowdown in a decade and faces a general election next year.
The assembly session begins on Thursday with an address by the president. Debates begin on Friday.
The government plans to present bills for insurance and pension reform that will open those industries to more foreign investment and to change land acquisition laws to make it easier for companies to buy land for industrial and infrastructure projects.
It will present the annual budget for the fiscal year ending in March 2014 next week. Also planned are a food security bill to provide subsidised grain for the poor, the setting up of an anti-graft ombudsman and the approval of an ordinance providing for harsher punishment for perpetrators of sex crimes.
The ruling coalition, headed by the Congress party, is technically in a minority but is supported by regional allies.
The main opposition Bharatiya Janata Party (BJP), which has led protests in the legislature during previous sessions, has criticised a kickbacks scandal in the $750 million purchase of VIP helicopters from Anglo-Italian company AgustaWestland that erupted last week.
The Hindu nationalist party is, however, most incensed by comments attributed to Home Minister Sushil Shinde that the group was promoting "Hindu terrorism", and has demanded an apology.
"We would like the house to function," top BJP member and opposition leader Sushma Swaraj told reporters.
"There are many issues that the house needs to discuss but it will depend on how this matter is resolved by them," she said of the government.