Final results of the plebiscite showed 50.3 per cent of voters backed the "Stop Mass Immigration" plan pushed by Swiss right-wing populists.
The fall-out from the result could sink a raft of deals with the EU, including on the economic front.
Switzerland is ringed by EU member countries and does the bulk of its trade with the 28-nation bloc, but has remained steadfast about not becoming a member.
The European Commission said it would assess EU ties with Switzerland, raising the prospect of restricted trade or other retaliatory steps.
"The EU will examine the implications of this initiative on EU-Swiss relations as a whole," it said a statement.
EU foreign ministers were scheduled to meet today in Brussels but it was not clear whether the Swiss vote would be added to the agenda.
Wolfgang Schaeuble, finance minister of Germany, Switzerland's top trade partner, said the result "is going to create plenty of problems for Switzerland in a host of areas". But he said it was also a warning sign of European globalisation fears.
Swiss Foreign Minister Didier Burkhalter said he planned to tour European capitals to explain the vote and seek a solution, starting with Berlin.
"The people are sovereign, and a healthy system doesn't force the public to follow political authorities with outsized powers," Burkhalter said.
The Swiss government and a broad swathe of economic lobby groups fearing the EU fall-out had battled the immigration curb plan.
But under Switzerland's system of direct democracy, voters have the last word on a huge range of issues.
The French-language newspaper Le Temps noted how French-speaking areas and larger cities voted against the immigration curbs, while German-speaking and rural areas generally voted for them.
Hundreds of people came out to demonstrate against the referendum result in the capital Bern and in the city of Lucerne. "We are ashamed," shouted protesters in Bern.
The Swiss government said it would examine over coming weeks how to "recast relations" with the EU, but stressed that current immigration rules would remain in place until the new ones were drawn up.
The vote obliges the government to renegotiate within three years a 2007 deal struck with Brussels that gave most EU citizens free access to the Swiss labour market.
It was one of a series of accords reached in 1999 after five years of talks that were seen as a way for Switzerland and the EU to enjoy access to each other's markets without Switzerland having to opt for full EU membership.