What Makes A Poll-i-Tick
Why are most Indian pre-election opinion poll predictions incorrect?
It’s voting time in the country and the pre-election opinion polls have set the tone for what is set to be one of the most volatile elections in recent times. With pollsters predicting a drop in Congress’ fortunes, the Congress has started clamouring for a ban on them. The Bharatiya Janata Party has of course followed suit with what is an unsaid rule in Indian politics – take the exact opposite stand to that of your rivals irrespective of the issue – and has mocked the Congress for being whiney losers. The Election Commission also joined the party and wants opinion polls banned from the date of notification of an election till the completion of election process – much to Congress’ pleasure and the BJP’s indignation.
Support Independent Media
The media must be free and fair, uninfluenced by corporate or state interests. That's why you, the public, need to pay to keep news free.
ContributeWhether opinion polls do have any bearing on the actual results of the elections is debatable. Yet that hardly deters most media outlets from dutifully conducting polls during the countdown to elections. While poll pundits in the United States seem to almost always get the numbers right, Indian pollsters have at best been sketchy with their poll predictions. So, what does it take to predict the outcome of elections with slightly more accuracy than the gold-dreaming seer of Unnao?
Allan Litchman is a political historian at American University who is known to have got every single poll prediction right ever since the 1984 US Presidential Elections. When Newslaundry asked Litchman – whom an American magazine once described as “that rare thing: a cheerful, optimistic progressive” – the secret behind his almost unrealistically correct forecasts, he more than lived up to the generous description with the following reply:
My formula is no secret. The Keys are 13 diagnostic questions that are stated as propositions that favour re-election of the incumbent party. When five or fewer of these propositions are false or turned against the party holding the White House, that party wins another term in office. When six or more are false, the challenging party wins. The Keys were initially developed through the retrospective study of presidential elections from 1860 to 1980 and subsequently applied successfully to predicting the results of elections from 1984 to 2012. The Keys to the White House demonstrate that the American electorate chooses a president according to the performance of the party holding the White House as measured by the consequential events and episodes of a term — economic boom and bust, foreign policy successes and failures, social unrest, scandal, and policy innovation.
Litchman, though, categorically mentioned that his model is designed and meant for the American system, which is obviously distinctly different from the Indian. He also said that the multi-party system in the Indian scheme of things adds many more variables, which make calculations and projections much more complex. Perhaps that explains why we get it wrong more often than we get it right.
Click here for a graphical representation of opinion polls carried out in India in the last few years.
For an expert perspective on the science of psephology (the science of elections and everything related to them) in the Indian context, Newslaundry spoke to Rajeeva Karandikar, Director of the Chennai Mathematical Institute. Karandikar is somewhat of a pioneer in forecasting Indian elections and since 2005 has been making seat predictions on the basis of vote projections done by the Centre for the Study of Developing Societies (CSDS). Karandikar told Newslaundry that it isn’t as much the size of the sample as the “quality” that matters. Quality, according to him, comes from keeping things simple and picking voters at random (for instance, every tenth name) from the voters’ list. The idea is essentially simple – to have the most varied demography as possible, which is achieved by the statistical technique of random sampling, tweaked to suit Indian voting conditions. The principle of wider representation is for obvious reasons universal as is evident from Litchman’s reply to our question of what a good sample is: A very good sample that takes into account the various characteristics of the electorate, such as race, religion, gender, and socio-economic standing.
Karandikar is of the view that there is nothing called a perfect sample size, and it is a variable which is contingent purely on the objective of the prediction. He said that while predicting the results of every constituency calls for a very large sample size, a much smaller sample is good enough to predict the overall seat distribution of an election. According to Litchman though, a sample of some 800 to 1,500 registered or likely voters is sufficient to overcome the problem of sampling error in polls. The reduction in sampling error is fairly minimal as sample size grows. Of course, Litchman’s numbers pre-assume the fact that the sample is “good” as he defines it.
While Litchman’s methodology is public and is easily available through a simple Google search, Indian pollsters aren’t so transparent about the model or method they employ. Of the opinion polls this season, most haven’t bothered to reveal something even as basic as the number of respondents. Only the CNN-IBN-The Hindu–CSDS polls have been published with a detailed note on the methodology of compilation.
There are many theories about why Indian opinion poll predictions are so far off the mark. Karandikar admitted that it is indeed challenging to predict with precision in a country like India where there are a host of factors at play. “Forget the American system; even the British system (on which ours is modeled) cannot be directly replicated because there are so many factors”, conceded Karandikar. Yashwant Deshmukh, founder and managing director of C-Voter, the agency which was commissioned by Times Now to conduct its opinion polls this year was quoted by The Hindu as saying that it is the unscientific and inaccurate interpretation of poll agencies’ findings by politicians and the media that has lead to the dubiousness of such polls in India.
It is important to understand that opinion polls are at times not only about the math but often have deeper sociological bearings. Therefore, the entire premise on which they stand could be flawed and perhaps they are not meant to be accurate by their very nature. Anirban Sengupta who teaches Sociology at the Ambedkar University in Delhi said, “If a response is considered as a social action, following Max Weber one may argue that while communicating a specific response an individual is always attempting to communicate/generate a particular meaning. Therefore, to consider such responses as objective portrayal of ‘reality’ may be methodologically problematic when looked at from a constructivist standpoint”.
In all fairness, opinion polls in a country like ours, where there are so many factors to account for, can never be perfectly accurate. Psephology in India is a work in progress that will hopefully mature soon and get better with time. Much like the Indian democracy itself it seems. If they don’t get banned that is, like most other things which don’t work in the ruling government’s favour.
References for Infographic:
Power NL-TNM Election Fund
General elections are around the corner, and Newslaundry and The News Minute have ambitious plans together to focus on the issues that really matter to the voter. From political funding to battleground states, media coverage to 10 years of Modi, choose a project you would like to support and power our journalism.
Ground reportage is central to public interest journalism. Only readers like you can make it possible. Will you?