Pollsters struggled to predict the right vote shares. But when they did, they got the seats wrong. There are exceptions.

The season’s feasting on election coverage is finally over. The Aam Aadmi Party won a historic mandate in Punjab and the Bharatiya Janata Party kept Manipur, Uttar Pradesh, Goa and Uttarakhand in its kitty. Pollsters, who spent a fortune on surveys, sampling and mathematical modelling, partnered with news channels and presented their projections.

In Uttar Pradesh, Punjab and Manipur, the results matched the broad trends indicated by exit polls. But things were trickier in Goa and Uttarakhand, where cliffhanger contests forecasted by most pollsters did not materialise.

Matching the actual numbers with exit poll data, two problems emerge: pollsters still struggle at hitting the vote share sweet spot, often fumbling badly. But when they do, the numbers either don’t convert to correct seat predictions or do not meet the same success in projections of runner up parties.

It would be prudent to give preference to accuracy of vote share over seats. If a pollster scores good on votes, it has done a part of its job right, even if the seats are predicted wrongly. But a correct seat prediction based on a wrong vote share is likely baseless since it can’t establish sound sampling.

By the time the counting stopped on Thursday, most pollsters had laid claim to giving us the right data. (Axis My India’s Pradeep Gupta cried and danced again. Don’t know why.) But most of it was empty boasting. We break down their predictions and explain why.

Chanakya shines in Punjab

The Aam Aadmi Party bagged 92 seats in Punjab, becoming the first party to touch the 90 mark in the state since 1962. The stalwarts of the Congress and Akali Dal – the Badals, Charanjit Singh Channi and Navjot Singh Sidhu – could not hold their seats.

On India Today, Axis My India had predicted 83 seats for AAP, with its closing bracket stretching till 90. 41 percent was the predicted vote share. Jan Ki Baat on India News gave the party the same vote share but with 72 seats.

According to the Election Commission, AAP ended up with 42 percent of votes and 92 seats. So both the exit polls fell a little short of grasping the intensity of the pro-AAP sentiment.

The ones that did not get Punjab right were P-MARQ, Veto and CVoter. CVoter, which prides itself on getting Bihar correct when others fumbled, projected only 56 seats with 39 percent vote share on ABP News. That makes its exit poll the farthest from results in terms of seats, but still better than Republic TV’s P-MARQ, whose AAP vote share – 35.6 percent – was way off the mark.

Veto’s performance on Times Now was above average. It had forecasted 40.4 percent of all votes for the Kejriwal-led party with 70 seats.

The winner of Punjab projections was Chanakya. It gave the highest seats to AAP – 100 seats with a rise or drop in 10 seats on the back of 43 percent vote share. This was not spot on but close. Unlike other pollsters, Chanakya accurately captured the 15 percent lapse in Congress’s vote share in the state, a drop from 38.5 percent in 2017 to 23 percent this year.

Capture trend in UP, not details

The BJP led by Yogi Adityanath became the first party to get re-elected in UP after completing a five-year term. The BJP alliance – which includes Apna Dal (Sonewal) and Nishad Party – won 273 seats. Most exit polls captured the broad trend, but their seats and vote share predictions were scattered.

Axis My India, for instance, pegged the BJP alliance at 307 seats. It goofed up despite giving a large bracket of 288-326 seats to the alliance. Its vote share projection was 46 percent. The Samajwadi Party alliance was handed 86 seats with 36 percent votes.

While the EC website tells us that BJP took 41.3 percent of all votes, the individual figures of Apna Dal and Nishad Party are yet to come out. In the SP-led alliance that came second with 125 seats, the party significantly increased its vote share – up to 32 percent from 21.8 percent in 2017. Its alliance partner Rashtriya Lok Dal (RLD) cornered 2.85 percent votes. The vote share of ally Suheldev Bhartiya Samaj Party (SBSP) has not been released yet.

Chanakya gave 294 seats to the BJP alliance, just about grazing past the actual result with its opening bracket of 273. The predicted vote share was 43 percent. For SP and friends, the numbers stood at 105 seats with 35 percent vote share.

CVoter’s vote share projection was not the best one. It underpredicted the BJP alliance’s success with 41 percent and SP alliance at 34 percent. As a result, the translation into seats was flawed, with 236 to BJP+ and 140 to SP+. Jan Ki Baat gave the same vote share to BJP and its allies and bumped up SP+ to 35 percent – but again, the seat projection was a miss.

P-MARQ and Veto’s UP predictions were most mediocre. The former predicted 40.1 percent vote share for the BJP alliance with 240 seats, the latter put these numbers at 39 percent with 225 seats.

Exaggerations in Goa

None of the exit polls could capture the BJP’s win in Goa, the state with the lowest assembly seats this season. The saffron party won 20 seats and has the support of one independent candidate to form a government. It got 33 percent of all votes, 8 percent more than than the Congress’s and its ally Goa Forward Party (25.3 percent), which got them 12 seats.

Exit polls broadly predicted a hung assembly. And to understand why they got it wrong, look no further than the exaggerated vote share they attributed to the Congress – Axis My India gave it 32 percent, ETG Research 31 percent, CVoter and Jan Ki Baat 30 percent, CNX 29 percent and P-MARQ 28 percent. We couldn’t find what Veto’s numbers were.

Chanakya did not put out an exit poll on Goa.

Vote shares fumble in Uttarakhand

The BJP retained Uttarakhand, winning 47 seats, despite Chief Minister Pushkar Singh Dhami losing his constituency. The Congress could only manage 19. None of the exit polls could capture these numbers.

The reason seems to be that no pollster could accurately predict the vote shares. If they got BJP right, they projected Congress wrong, and vice versa.

Let’s take a look. The EC website tells us that the BJP won 44.3 percent of all votes in Uttarakhand. Axis My India rightly called this with its 44 percent forecast. Chanakya stood behind at 43 percent. ETG, Jan Ki Baat and CVoter pegged this figure at 41 percent. Once again, P-MARQ was far off at 40 percent.

The Congress bagged 38 percent of votes. But here, Axis My India went 2 percent overboard at a 40 percent prediction, and so did Jan Ki Baat. Chanakya, however, got it wrong at 34 percent, but in a surprise, P-MARQ gave the closest call – 38.4 percent.

ETG Research projected a conservative 37 percent and CVoter a liberal 39 percent for the Congress.

Some hits, most misses in Manipur

The BJP found its fourth victory on Thursday in Manipur, where it won 32 seats with nearly 38 percent vote share. The Congress – much hyped by pollsters – could gather only 5 seats with 17.3 percent votes.

Once again, there were no exit poll heroes in the northeastern state, but some decent cameos. CVoter’s BJP vote prediction was spot on, but it could not convert it to seats. Its Congress vote share projection – 29 percent – was awful. Conversely, Axis My India forecast got the Congress vote share right – 18 percent – and so did the seats. But it put the BJP share of votes at 41 percent with 38 seats.

P-MARQ had given 36 percent votes to BJP and 26 percent to the Congress. Jan Ki Baat gave ditto numbers for the former but increased the latter forecast to 28 percent. The brackets of these two pollsters, however, could not predict the constituencies won.