In June 2023, Pramit Bhattacharya authored an excellent working paper for Carnegie Endowment for International Peace. It is titled India’s Statistical System: Past, Present, Future and starts with a historical sketch that goes back to the colonial period. While this is a rich paper, let me give a few selective quotes.
(1) “Gadgil argued that a poor country such as India should not invest in a new statistical organisation. Instead, the resources earmarked for NSS should strengthen administrative data systems and upgrade the state statistical bureaus. He supported Sukhatme’s contention that statistics should be collected by individual ministries and departments that had domain knowledge, not by an outside agency. Mahalanobis argued that an independent survey agency was essential to collect unbiased data since departmental administrators have a strong incentive to misreport figures.” Today, no one will posit the administrative data versus census/survey as either/or. Both have their uses. Administrative data tends to be almost real-time, but incentive structures are such that one can’t blindly trust such data (think of dashboards). The evident “success” in administrative data must be validated through census/surveys.
(2) “After the NSS was moved out of ISI, the organic link between the world of statistical research and official statistics was ruptured. The NSSO governing council headed by a nonofficial expert could only partially fill that void. Some of the scientific rigour and flexibility of the NSS program was lost.” I think this is related to a broader issue, that of the Indian Statistical Service (ISS). India has produced, and continues to produce, some of the best statisticians in the world, and the ISI (Indian Statistical Institute) brand (particularly B. Stat) is recognised everywhere. But, unlike in the past, do these famous statisticians and excellent students join ISS today? If ISS has deteriorated, and it is the foundation for India’s official statistics, without revamping the service, how can one possibly revamp India’s statistical system?
(3) “A growing band of economic reformers began pressuring the NSSO to redo the old questionnaires. The ham-handed manner in which those changes were introduced contaminated the results of the 1999–2000 consumption expenditure survey. The official poverty estimates based on that round were discredited, and it took several years to clarify the status of poverty in India.” We will return to this point.
(4) “Four years after the Rangarajan Commission submitted its report, the union government decided to act on it. In 2005, a government resolution announced the establishment of the National Statistical Commission (NSC). It promised that the commission would receive statutory backing ‘within one year.’ … Eighteen years after that resolution, the union government has yet to enact a law providing statutory backing to the NSC. The NSC has failed to realise most of its objectives.”
(5) “Until the late 1990s, the NSSO regional offices had the power to employ field investigators locally. In 1997, the Fifth Central Pay Commission, which sets guidelines for HR policies across central ministries, recommended a new cadre called the Subordinate Statistical Service to improve the promotion prospects of field enumerators and junior officers. … The new HR policy disrupted the NSSO’s field operations.”
(6) “India’s statistical system stands at the crossroads today. On the one hand, there is a danger that the statistical system could be marginalised further to suppress inconvenient facts. On the other hand, the statistical crisis presents an opportunity for inexpensive reforms that can sharpen India’s competitive edge in the global economy. … A new reforms commission can outline a road map for reforms and reassure all stakeholders about the government’s seriousness in fixing a major governance deficit.”
Pramit Bhattacharya’s stock-taking paper is a critique of the official statistical system, and in some sections, strong words are used, such as that on recent controversies. But the broader point is that the statistical system has not only been criticised by those who oppose the government and are outside it but also by those who work for the government and, thus, support its policies. Witness several papers and articles by three members of the Economic Advisory Council to the Prime Minister (Bibek Debroy, Sanjeev Sanyal, Shamika Ravi).
Indeed, the Rangarajan Commission was a long time ago (2001), and it is a good idea to set up a new reforms commission (quote 6) to set up a road map, especially because India will be a member of the UN Statistical Commission from 2024. On the point in quote (4), though, I am sceptical. There were cogent reasons for why the Madhav Menon Committee’s (2011) recommendation of giving statutory backing to NSC (making it answerable to Parliament) wasn’t accepted. It wasn’t merely inertia and protecting one’s turf.
To quote from the paper again, “Sen’s successor (Pronab Sen’s), T. C. A. Anant, stalled the Menon committee proposals since they would have constrained the powers of the chief statistician, according to a former Indian Statistical Service (ISS) officer who was posted at the NSC secretariat during those years. The conflict between the chief statistician and the NSC had begun in the Tendulkar-Sen era (Suresh Tendulkar).” Yes, there is a need to reconcile the roles of the Chief Statistician as head of MOSPI (Ministry of Statistics and Programme Implementation) and the Secretary of NSC. However, statutory backing is likely to make NSC independent but more irrelevant.
While one can continue to argue on that, I don’t think anyone can argue on quote (2), (3) and (5), all highlighting a serious human resources issue. Within the present ISS, there is little expertise in survey design, and most of the time is spent authoring unimaginative reports. That’s why the processing of census (not just population census) and survey results is delayed. That’s why questions are ham-handed (quote 3), inconsistent and non-comparable over time. Every survey (through a working group) starts afresh, ignoring what was asked earlier for that particular survey and other surveys too. That’s the reason one has to fish around for enumerators in this day and age.
Every questionnaire (both census and survey) can be streamlined and simplified so that it is completed in 20 minutes. How many people have the time and inclination to respond to a questionnaire that takes two hours to complete? If supporters and critics of the government agree on any single matter, this is a rare phenomenon. Hence, there is no question that MOSPI needs to take this seriously.
Bibek Debroy
Chairman, Economic Advisory Council to the PM