Akhilesh Yadav, chief minister of Uttar Pradesh, where two cousins aged 12 and 14 were raped and hanged last week, has faced criticism for failing to visit the scene and for accusing the media of hyping the story.
Gaur, who is from Prime Minister Narendra Modi's Bharatiya Janata Party (BJP), said on Thursday that the crime of rape can only be considered to have been committed if it is reported to police.
"This is a social crime which depends on men and women. Sometimes it's right, sometimes it's wrong," said Gaur, the home minister responsible for law and order in BJP-run Madhya Pradesh.
"Until there's a complaint, nothing can happen," Gaur told reporters.
Gaur also expressed sympathy with Mulayam Singh Yadav, head of the regional Samajwadi Party. In the recent election, Mulayam criticised legal changes that foresee the death penalty for gang rape, saying: "Boys commit mistakes: Will they be hanged for rape"
The BJP dismissed Gaur's comments as an expression of his personal views, and not the party's.
Modi, who was sworn in as prime minister last week after a landslide election victory, has so far remained silent over the double killing in the village of Katra Shahadatganj, around half a day's drive east of New Delhi.
The father and uncle of one of the victims said they tried to report the crime to local police but had been turned away. Three men have been arrested over the killings. Two policemen were held on suspicion of trying to cover up the crime.
Although a rape is reported in India every 21 minutes on average, law enforcement failures mean that such crimes - a symptom of pervasive sexual and caste oppression - are often not reported or properly investigated, human rights groups say.
More sex crimes have come to light in recent days. A woman in a nearby district of Uttar Pradesh was gang-raped, forced to drink acid and strangled to death. Another was shot dead in northeast India while resisting attackers, media reports said.
U.N. Secretary General Ban Ki-moon has said he was "especially appalled" by the rape and murder of the two girls.
"We say no to the dismissive, destructive attitude of, 'Boys will be boys'," he said in a statement this week that made clear his contempt for the language used by Mulayam Singh Yadav.