Poll of Polls: the polls got little right
Updated on 10 May 2010
In his final assessment, Professor Leighton Vaughan Williams of Nottingham Business School concludes that apart from predicting a hung parliament, the opinion polls got little else right in the run-up to the 6 May election.
Apart from the exit polling, not one of the pollsters came out of the election with flying colours, the consistent bias being to overestimate the Liberal Democrat vote share and to under-estimate that of Labour. Broadly speaking, the telephone pollsters performed better than the online pollsters.
Final National Polls
- Angus Reid/PB: Con 36, Lab 24, Lib Dem 29
Surveyed 2,283 adults on May 4 and May 5 - ComRes/ITV News/ Independent: Con 37, Lab 28, Lib Dem 28
Telephoned 1,025 adults on May 4 and May 5 - Harris/Daily Mail: Con 35, Lab 29, Lib Dem 27
Surveyed 3,406 voters between May 4 and May 5 - ICM/Guardian: Con 36, Lab 28, Lib Dem 26
Interviewed 2,022 adults between May 3 and May 4 - Opinium/Daily Express: Con 35, Lab 27, Lib Dem 26
Surveyed 1,383 votes between May 4 and May 5 - Populus/Times: Con 37, Lab 28, Lib Dem 27
Interviewed 2,505 voters on May 4 and May 5 - TNS BMRB: Con 33, Lab 27, Lib Dem 29
Interviewed 1,864 adults between 29 April and May 4 - YouGov/Sun: Con 35, Lab 28, Lib Dem 28
6,483 adults questioned on May 4 and May 5
An alternative approach is to omit the best and worst score for each party and to average the remainder to obtain an adjusted average. When more than one pollster gives equal top score or equal bottom score, only one of these is omitted. This technique gives an average of 35.7 per cent for the Conservatives, 27.7 per cent for Labour and 27.5 per cent for the Liberal Democrats.
An opinion poll by Ipsos/Mori, issued on polling day itself, for the Evening Standard, showed the same broad picture, with the Conservatives on 36 per cent, Labour on 29 per cent and the Liberal Democrats on 27 per cent. Ipsos/Mori interviewed 1,216 voters on May 5th.
Assuming a uniform national swing, the Conservatives would on the basis of the raw average of the eight eve-of-poll pollsters (adjusted average in brackets) win 285 seats (285) in the House of Commons, Labour 250 seats (251) and the Liberal Democrats 84 seats (83). On this basis, the Conservatives would be the largest party but 41 short of an overall majority.
Applying a more sophisticated seats projection methodology, which we term ANS (Adjusted National Swing), which allows for the differential impact of swing on different seats, the projection of the number of seats on this basis of the raw average (adjusted in brackets) comes out as: Conservatives: 299 seats (300), Labour: 228 seats (228), Liberal Democrat: 88 seats (87). Including the poll released to the Evening Standard on polling day itself pushes the Conservative vote projection up to 305.
The spread betting operators were projecting at 10am on the morning of polling day -
- IG (mid-point of spreads): Conservative 316.5 seats, Labour 220.5 seats, Lib Dem 82 seats
- Spreadex (mid-point of spreads): Conservative 317.5 seats, Labour 218.5 seats, Lib Dem 80 seats
- Sporting Index: Conservative 317.5, Labour 218.5 seats, Lib Dem 80 seats
Spread-betting average: Conservative 317 seats, Labour: 219 seats, Lib Dem 81 seats
Other projections
- Betfair’s "Election Predict": Conservatives 321 seats, Labour 218 seats, Lib Dem 82 seats
(Betfair: Prob.of Con majority 41.3%, No overall majority 56.7%, Lab majority 1.7%, Lib Dem majority 0.3%). - Politics Home Projection: Con 307 seats, Lab 229 seats, Lib Dem 82 seats
- UK Polling Report Projection: Con 305 seats, Lab 225 seats, Lib Dem 85 seats
- Forecast UK Projection: Con 303 seats, Lab 215 seats, Lib Dem 99 seats
- LSE (Uniform National Swing): Con 275, Lab 264, Lib Dem 79
- LSE (With Tactical Voting): Con 251, Lab 286, Lib Dem 81
- LSE (including ‘marginals’ effect): Con 310, Lab 211, Lib Dem 97
- BBC ‘Poll of Polls’ (Assuming UNS): Con 281, Lab 259, Lib Dem 81
- Angus Reid: Con 330, Lab 175, Lib Dem 112.5
- TNS: Con 292, Lab 204, Lib Dem 114
- Peter Kellner (YouGov): Con 305, Lab 235, Lib Dem 80
- Election Prediction Project: Con 287, Lab 251, Lib Dem 80
(When projections have been given as a range, the mid-point is reported).
And my final prediction
40 minutes before the official close of polling, I was interviewed by Radio New Zealand, who asked me what the polls were showing. I ventured an estimate, based on my analysis of all the polling, that the Conservatives would end up with 305 seats in the House of Commons. I think I'll take that! And just be glad I wasn't pressed for an estimate for Labour and the Liberal Democrats as well.
The exit poll, for the BBC, ITV News and Sky, conducted by GfK NOP and Ipsos/Mori, ultimately predicted 305 seats for the Conservatives, 255 seats for Labour and 61 seats for the Liberal Democrats. The exit poll surveyed 17,607 voters at 130 polling stations across Great Britain.
Actual outcome
Seats
- Conservative: 306 seats (projected: 307)
- Labour: 258 seats
- Liberal Democrats: 57 seats
Vote share
- Conservative 36.9 per cent
- Labour: 29.7 per cent
- Liberal Democrat: 23.6 per cent
Where the polls got it wrong – and where they got it right!
Let's give credit to the pollsters where it’s due.
Of the nine final polls reported here, five got the final Conservative share within a percentage point of where it ended up. Five of the nine got the Labour vote share within two points.
The flaw common to all was an over-estimate of the Liberal Democrat vote, by an average of about 4 points.
How can we explain this? My best guess is that it’s a combination of four factors. One factor is a late tactical switch of Lib Dem voters, in large part to Labour, to keep the Conservatives out.
The second factor was that the Lib Dem vote was already showing signs of slippage and this continued.
Thirdly, there may have been a "shy" Tory and Labour effect, i.e. Conservative and Labour voters were more "shy" of expressing their voting intentions than Lib Dem voters.
Finally, there were an exceptionally high number of "Don’t Knows" and "Might Change" in the final polling evidence, which may have worked against the Lib Dems when voters were faced with the final decision as to where to place their pencils.
All of which basically adds up to the fact that the Lib Dem may have been "softer" than that of the other parties.
It's interesting to note, though, that the net loss of seats suffered by the Lib Dems can in large part be attributed to their performance in the close recounts. If they'd won just half of these, they would have been close to their 2005 result.
In contrast, the Conservatives did exceptionally well in the close recounts, against Labour as well as the Lib Dems.
Better news for the Lib Dems is the number of seats in which they came second, which rose from 188 to 242. In contrast, Labour dropped to third place in an additional 81 seats.
Regionally, the swing from Labour to the Conservatives was only 2.5 per cent in London, compared to 6.1 per cent in the rest of England.
What about the much talked about "marginals" effect, i.e. that the Tories would perform much better in the marginals than nationally?
Well, where was it? Across the country, the swing from Labour to the Conservatives was 5.03 per cent. In Labour-held marginals with a majority of less than 10 per cent, the swing was 6 per cent, and 5.13 per cent in the next tier of marginals (Labour ahead by 10 to 20 per cent). Not a huge "marginals" effect, it seems.
So how did they achieve 305 seats when they should have gained just 287 seats on a uniform national swing? Partly it can be explained by the fact that the average conceals the fact that they did better than that in the very seats where the extra votes were needed to put them over the top.
And then there's that "recount" effect. With a very few exceptions, whenever the result came down to a statistical toss-up, whether against Labour or the Liberal Democrats, the Tories won the toss!
