Let us start with the biggest paradox. In all probability, you have never been interviewed for any of the opinion polls. How can the polls then claim to reflect ?public opinion??
The story goes that a woman actually asked this question to Gallup. Gallup explained that the likelihood of being interviewed for a Gallup poll is very low. It is, in fact, just a little bit higher than the likelihood of being hit by lightning in the continental US. The woman exclaimed that she had indeed been hit by lightning, but she had never been met by a Gallup pollster till then. Presumably meeting Gallup himself didn?t count!
In any case, this is because almost all consumer research surveys are sample surveys. Censuses are rare in the consumer research context. They are infrequent even in the larger context of the nation as a whole. The Indian census, for instance, is carried out once every ten years. It happens to be one such instance right now. The reader will be aware that the process has begun for the census of 2011. As an aside, the census will put out the population count at the reference point??sunrise of March the 1st?.
Sample surveys work because it?s not necessary to meet everyone to estimate the prevalence of a particular phenomenon. Just as you sample a spoonful to test if a dish is done, all you really need is a ?spoonful?.
The usual clich? used to illustrate this point is of a cup of tea and the sugar test. One needs just a spoonful of tea to check if it is sugared. Of course, it?s not error-proof. A spoonful may be good enough for a ?homogeneous? situation like in a pot of tea. But it will not be enough to get the full picture if it is a heterogeneous case, like a samosa heated in a microwave.
Anyone with experience of heating a samosa in a microwave will testify that one cannot rely only on a sample from the outer jacket. You need at least two samples: one from the outer jacket (which may be merely hot) and one from the core inside (positively scalding). The fact that a microwave heats from the inside out is a paradox in itself. But that is another story.
All we really need is an indication of whether it is likely to be too hot to put inside one?s mouth. For this a test based on those two samples, one from the outer jacket and one from the innards, is good enough. It can yield a fair idea of how hot the samosa really is.
The statistical argument is that a precision of the result from a sample depends only on the sample size. It does not matter how big the sample is relative to the population it is drawn from. Continuing with the tea example, it does not matter if it is one cup of tea or one gigantic pot of tea?you still need only one spoonful to check.
This becomes a real paradox when we start imagining highly heterogeneous contexts in the real world of people. Every election season sees some political party muttering something about how a survey of a mere thousand people cannot really reflect the reality (especially if it is predicted to lose).
How can this be so? And, more important, how big a sample do we really need in order to ?get the full picture?? We shall examine these in the later columns. I must note here that while exploring these we also will stray from the realm of Paradox, and wander into the realm of Myth. Specifically, the ?Myth of the minimum sample of 30? and a variety of other faith-based statistical superstitions that lie underneath.
But sampling errors are only one part of the story. More than sampling errors and the math, it is important to appreciate that there are hundreds of other factors that render the survey results to be exactly wrong.
Consider the satisfaction measurement surveys at a retail outlet, like a grocery supermarket. In these, a sample of customers are intercepted and surveyed over a period of a week or ten days, every few months. It?s obvious that the survey will only catch those who actually visit the store. It will simply not include customers who have stopped shopping there because they are really unhappy with the store.
So, one needs to be cautious while interpreting the satisfaction scores. More important, let us assume that the situation continues with more customers vanishing every week. So, the base shrinks and is ?purified? to the more satisfied lot who alone continue to shop from there.
The satisfaction scores will then misleadingly go up from one track to the other, even as the Researcher Nero fiddles with the calculator on the sampling error.
The author is president, Ipsos Indica Research. These are his personal views