What’s wrong with exit polls?

By Vasant V Bang

Exit polls have gone wrong once again. Issues affecting the validity of exit polls, especially in India, are beyond operational and ethical. There are deeper conceptual and methodological issues.

The first and foremost issue pertains to the research question. Trying to understand the winning possibility of political parties in a constituency is not the same as their winning possibility across several constituencies. The two situations present different research questions. While making a choice, voters are likely to consider both, the party and the candidate. Let us assume that there are 100 constituencies in a state designated as 1, 2, 3, and so on. Further assume that candidates in constituency 1 are A, B, and C belonging to parties as X, Y, and Z. The research question in constituency 1 is about knowing the winning probability of AX versus BY versus CZ. Even if one candidate is allowed to contest elections in more than one constituency, the research question in constituencies 2 to 100 cannot be exactly the same as in constituency 1.

Second, a direct corollary of difference between the research questions is that research within a constituency and across constituencies involve different universes about which generalisations are sought. A perfectly randomised selection of respondent voters, chosen in randomly selected voting booths across a constituency, can account for heterogeneity among voters. Hence, its findings can be generalised to the respective constituency. But if a random sample of voting booths is not taken from each and every constituency in a state, it does not account for heterogeneity in combinations of political parties and candidates. This method limits the generalisability of findings across state.

Contrast this with the presidential elections in the US. The whole of United States, across counties, votes for the same combination of party and candidate. Since the research question to be studied remains the same, entire US constitutes one research universe though voters in the same universe can be heterogenous in terms of demographics, psychographics, and behaviour. The validity of the research in such a scenario depends on the extent to which this heterogeneity is accounted for in sampling. But countries like India, where each constituency can have different combinations of parties and candidates, cannot be treated as a single universe unless voters vote for the parties without considering the candidates. Previous researches have shown that even otherwise frivolous characteristics like the looks of a candidate influence voting behaviour. In a research conducted in Canada, the candidates contesting in an election were classified on the basis of their looks using well accepted norms and votes received by them were compared. It was found that candidates with better looks received several times more votes than other candidates. In India there are many other characteristics like religion, caste, sub-caste, community, and gender which influence voting behaviour.

The third conceptual issue is related to difference of opinion, intention, and behaviour of human beings. Just because a respondent has a good opinion about a particular candidate it does not mean the respondent intends to vote for him or her. Unlike opinion polls which aim at studying opinions and intentions, an exit poll aims at studying the behaviour of people. Hence, the predictive validity of an exit poll can be easily ascertained from actual election results. But it is difficult to do the same in case of an opinion poll. In that sense opinion polls are more like market research studies for commercial products. Several studies about the launch of new products have revealed that positive intentions reported in market research do not necessarily get translated into actual purchase.

The fourth issue pertains to the method of knowing the truth through observation as against asking the respondents. The aforementioned secondary data-based research conducted in Canada was followed by a questionnaire-based survey. In this follow-up research, 73% of the respondent voters denied the role of looks of a candidate in their voting decision. But in exit polls data collection involves asking questions rather than an observation of actual behaviour. The limitations of direct questioning as a method of data collection have been well documented. There is a difference between studying behaviour by asking questions in case of a commercial product and a sensitive issue like voting in an election. Buying a product is not as sensitive an issue as voting in an election. Besides, responding to a question about commercial products serves as feedback which, if used rightly by the marketing companies, can translate into a benefit for the respondents. But after having voted, revealing the choice of candidate in an exit poll accrues no additional benefit to the voters. In fact, there is a real danger that disclosing voting behaviour may invite persecution by those who did not receive the respondent’s vote.

The fifth issue is related to non-response bias. Several researchers have pointed out that those who refuse to participate in an opinion or exit poll are unlikely to match the behaviour of those who participate.

Even if actual poll results match the predictions of an exit poll, this does not indicate its validity unless the aforementioned conceptual and methodological issues are resolved.

The writer is a management advisor who has taught and done market research for over 30 years.

Disclaimer: Views expressed are personal and do not reflect the official position or policy of FinancialExpress.com. Reproducing this content without permission is prohibited.