Today-Chanakya ’s aggressive exit poll forecast of 340 seats for the National Democratic Alliance (NDA) turned out to be on the money. The forecast is being compared to the victory of the BJP itself and congratulations are coming in from peers and media personalities. But how much of the forecast was due to their genius? And how much due to dumb luck and/or voodoo?

We took a closer look at the data in the Vozag exit polling database to better understand the issue.

First, some background on how the exit and opinion poll seat projection process works.

1. The first part is to get a good understanding of the vote share. How many people voted for which party and what is their share? This is the most important part of the process. You get this wrong and everything else could go wrong. This part has all the issues and science of surveying, sample sizes, respondent outreach etc.

2. The second part takes the vote share and converts it into seat projections. There are a number of mechanisms to do this including the Cube Rule, Uniform State and National Swing, or whatever in-house-voodoo each polling firm has in order to convert vote share to seat projections.

Given the background, let’s dig deeper into the data for one state – Uttar Pradesh (that accounted for the biggest variance). At the national level, NDTV–Hansa had projected 279 for NDA (variance of 58 to actual results) and CNN-IBN-Lokniti had projected 276 (variance of 61). Meanwhile, Chanakya projected 340 (a 3 seat variance).

Uttar Pradesh was the biggest factor in the variances for NDTV, CNN-IBN and other channels. In Uttar Pradesh, NDTV–Hansa’s NDA seat projections were 56 (a 17 seat variance from actual) and CNN-IBN’s projected NDA seats were 49 seats (24 seat variance from actual). These 17 seats explain 30% of the variance for NDTV and the 24 seat variance explains 40% of the variance for CNN-IBN at the national total number.

Digging further into the vote share variance and seat variance in Uttar Pradesh, things get interesting as the table below shows.

Just looking at the table, one can see Chanakya actually underestimated BJP/NDA vote by 9.3% and yet by the time they converted vote share to seats, they were close to the mark. Even though NDTV and CNN-IBN underestimated BJP vote share by 3.3%, they were much closer to the actual than Today’s Chanakya.

When compared to NDTV and CNN-IBN, Chanakya underestimated vote share by 6% and yet with the severely underestimated vote share had much higher seat projections than NDTV-Hansa or CNN-IBN-Lokniti.

Is that genius, dumb luck or voodoo?