Sometimes, a proud nation has to recognise that its statistical institution is failing the most basic of credibility “smell” tests; and is in dire need of reform.
Some results of the NSO Consumer Expenditure Survey (CES) for 2017-18 have been published in leaky form. It is hoped that the official release (not endorsement for reasons enunciated below) of the unit-level data will follow soon, so researchers, analysts, and politicians, and even former prime ministers can evaluate for themselves how bad NSO data really are.
The previous NSO survey on employment (PLFS) estimated the population in 2017-18 to be 1,074 million, when even mathematically challenged individuals estimated it to be upwards of 1,300 million (actually 1,339 million). That error of 265 million under-estimation was a national record for the highest under-estimation of such a basic number; most NSO surveys have had an under-estimation of population of around 5-10 %—the 2017-18 underestimation at 20 % is a record.
This under-estimation has consequences, for a major policy variable of interest—jobs and job growth. The unemployment rate is not affected by estimate of aggregate population; but number of jobs are affected. Most scholars have estimated the employment generated by the PLFS data to be a decline of around 18 million in the number of jobs in 2017-18 relative to 2011-12. This is according to the usual status of employment, a measure which counts both half-time work (employed for 30-182 days) and full time work (employed for more than 182 days) as full-time employment. In a detailed (forthcoming) paper on employment, Tirtha Das and I find that the desired full-time jobs (defined as principal status) increased by 8 million between 2011-12 and 2017-18—a not-that-different increase over what was obtained in the go-go-growth years of 2004-05 to 2011-12 (14 million increase, but over seven years).
It is much easier to count people as employed or not employed, than to ask them about their monthly per capita expenditures. This is where the world record is on the way to being established. The CES survey for 2017-18 shows that real monthly consumer expenditures per person (MPCE in NSO parlance and NOT income as mistakenly assumed by some) declined from Rs 1,573 in 2011-12 to Rs 1,514 in 2017-18 (data converted from 2009-10 prices to 2011-12 prices to make consistent with other data.)
In my book “Imagine there’s no country”, published by the Peterson Institute in 2002, I had documented how there had been a declining trend in the amount of survey capture, i.e., over time, household surveys (S) were capturing less and less of consumption as revealed by an alternative calculation—national accounts (NA). While the two definitions (survey and national accounts) are not identical, they are broadly comparable.
The average S/NA ratio, around the world, was in the mid-80s in the 1980s, i.e., if the NA estimate of per-capita consumption was 100, then the household survey would estimate it to be 85. It is worth remembering that the S/NA ratio in India in the 1950s and 1960s was upwards of 95 %. Too high to be true? In a manner of speaking, yes. For then the household survey provided the estimate of consumption for national accounts.
But, with time, economies became complicated; national accounts data moved with the times, became more sophisticated and captured the trends in the economy much better than surveys. Survey organisations like NSO refused to move. In the 1983 NSO consumer expenditure survey (CES), the S/NA ratio in India had collapsed to 63% from the high 70s level of just a decade earlier. It was to be 30 years later (in 2012) when the world reached the low 60’s average.
That year (2011-12) India recorded a 55% ratio for S/NA. Just six years later (2017-18), the S/NA ratio in India has collapsed to just 33%—the second lowest ever recorded around the world for economies without hyper-inflation (when S/NA ratios really get distorted) and with populations above 10 million. The worst ever—Nigeria in 2009 with a S/NA ratio of 27.2 %.
There is yet another comparison one can make. The two most recent consumption surveys in India, just six years apart, yield a decline of 22 percentage points. This is the second worst sequential decline in the world. The worst—Pakistan in 2001 when the S/NA ratio was 46.9%, down 26.9 percentage points from the 73.8% estimate recorded in 1998.
The NSO secular decline has now persisted for some 50 years and marks a sad occasion for an institution that was a trend setting statistical institution in not only emerging economies, but in the world. In the early 1950s, world famous statistician PC Mahalanobis was the head.
I was privileged to be a member of the first National Statistical Commission of India headed by an internationally renowned economist Suresh Tendulkar. I was sent to Calcutta by Tendulkar to interact with the NSSO, and find out why the Indian S/NA ratio had sharply declined and what could be done to improve survey response. I met with little success and came back frustrated with the ancient techniques being followed by NSO.
The most recent statistical commission chairman, PC Mohanan, was a colleague. He has been quoted as not being surprised with the decision of the government to not accept the findings of the latest record low setting NSO survey. His view—government is suppressing reports not “favourable”.
If I thought that NSO consumption surveys were misleading and not acceptable in 2002 and 2006, I can be forgiven for thinking that the surveys are even less acceptable today. The NSO 2017-18 data reveals results that are truly bizarre—a decline in average real consumption of 0.6% per year between 2011-12 and 2017-18, when the NA consumption estimate is of a plus 5.8% annual growth. As discussed above, the NSO estimate for 2017-18 is so out of the box that it is actually out of any (reasonable) ball-park. If the government does not accept the findings of the survey (as suggested by a recent press-release) then a genuine reform of the NSO can actually begin. If not the previous glory, a reformed NSO can again become a respectable institution. That will not be easy, but it is a path worth embarking upon.
I have been surprised by how many respected analysts have pointed to the “findings” of NSO 2017-18 and related it to the slowdown in the economy in 2019-20. Some of these very same “analysts” were cheering the RBI/MPC a year ago when it raised repo rates to 6.5% in June 2018, the very last month of the 2017-18 survey. Their reason for cheering the MPC—growth was too high, so high that it was leading to high and accelerating inflation. Both views cannot be right, and it is worse than disingenuous to hold both views simultaneously. The so-called experts have to make up their mind—if growth was disturbingly high in June 2018, then it cannot under any stretch of the imagination be argued that the CES 2017-18 survey is even close to being right.
Not every government report should be accepted. Just like individuals fail exams, and editors reject papers (and columns!), sometimes institutions fail to produce a credible report. But, I do believe the unit-level data should be released. Let the world, and experts, find out for themselves how truly informative and credible the NSO CES data really are.
The writer is Executive Director, IMF representing India, Sri Lanka, Bangladesh and Bhutan.Twitter: @surjitbhalla. Views are personal