The 2015 UK General Election, the EU Referendum and, most recently, the US Presidential Election all produced results which most major pollsters and media outlets failed to predict. Amongst growing distrust of opinion polls across the Western World, Thomas Loughran considers the use of political opinion polls and where to go from here.

• The decline in trust for opinion polling has wider implications for the quality of democratic political debate
• Vote projection models and survey data often get lumped in with opinion polls but they represent very different methodological approaches and aims
• Political numbers have been weaponised and so opinion polls are more often used as ammunition in partisan debates that generate more heat than light
• To rebuild the reputation of opinion poll data, we should acknowledge the true public value of opinion polls which requires taking a back step from the prediction business
• We must review how opinion polls are reported and used in future political debate

First there was the 2015 General Election, then the EU Referendum, then the US Presidential Election.  It is not an exaggeration to say that this has been a disastrous 18 months for the reputation of political polls and, by association, for public trust in opinion polling and ‘political numbers’ more broadly.  After all, if the major headline predictions made by polls turn out to be incorrect then why should we trust them the rest of the time either? In fact defending the value of opinion polling in the current climate would appear to be an exercise in deep futility.

It is hardly surprising that if a process based on predicting future outcomes is consistently failing to do so then it faces something of an existential crisis.  Yet the decline in trust for opinion polling has wider implications for the quality of democratic political debate.  A brief look at the way in which ‘facts’ are often marshalled in political discussions on social media, and the dominance of partisan noise, should point to a convincing case of the need for high quality, trusted political data sources in which opinion polling should be a major component.  Ironically though, just as the recent US Presidential Election has raised the profile of false news and false facts as a potential threat to the norms of democratic debate, traditional sources of public opinion information have rarely been more comprehensively disdained.  It is likely to be a long climb back if confidence is to be restored and, as well as the technical improvements to the polls themselves, it also requires a more mature approach to the discussion of opinion poll data in politics.

Currently, the counter claims made by pollsters and academics in defence of opinion polling largely rest on two lines of argument.  The first is that, contrary to reports, election polls were not actually anywhere near as wrong as was reported.  Certainly in the case of the EU Referendum and the US Presidential Election there is some evidence to support this.  Final national opinion polls showed Hilary Clinton winning the popular vote, which she did.  In the run up to polling day they also suggested a tight and narrowing race, which it was.  But, the polling certainly didn’t get the overall result right and, if prediction is considered the ONLY purpose opinion polls, that is all that matters.  However, much of the prediction error laid at the door of polling would be more accurately attributed to the statistical modelling that aggregated polling results at the state level.  In short, the argument is that the national polls in US were broadly correct but that state polling was less accurate and it was this data that was used to predict the Electoral College outcomes in various statistical models.

In addition, there are complaints often voiced by academics that vote projection models and survey data often get lumped in with opinion polls as part of a general critique when these represent very different methodological approaches and aims.  All political numbers are not created equally, yet all can end up being tarred with the same brush within a general perception of ‘polling failure’.  However, while there is much truth to this more nuanced perspective on opinion polls, it does little to address the underlying problem once the perception of failure has been firmly established.

The second strand in the defence of polling is therefore more relevant here because it argues that much of the perceived problem with opinion polls is down to the way in which they are reported and interpreted rather than collected.  The central argument is that nuance is lost within highly politicised media environments and campaigns.  Reporting safeguards, the significance of context and caveats about the amount of error that is built into the process of polling are too often disregarded.  The value of opinion poll numbers should be in providing some clarity that allows people to see through the partisan fog and get a feel for the bigger picture or the direction of travel in public opinion.  Instead, political numbers have been weaponised and so opinion polls are more often used as ammunition in partisan debates that generate more heat than light.  Polling figures are not used to illuminate the wider context in which debates are taking place but are instead used as selective evidence in support of a partisan position – as if the numbers themselves have an independent power of persuasion.

As well as having a distorting effect in downplaying the importance of nuance and context (even gold standard polls and surveys tend to represent a single snapshot of opinion) it creates a zero sum mentality where the data is either right or wrong.  It is therefore using a fairly subtle device in a very crude way.  The purveyors of opinion polls have, to some degree, benefited from this over the years since it elevates their role from being the chroniclers of public opinion to being the arbiter’s of political debate.  However, what is required in order to rebuild the reputation of opinion poll data is an acknowledgement that the true public value of opinion polls is in their former role which may require taking a back step from the prediction business.

Individuals and organisations are currently working on technical solutions to improve the accuracy of polling in response to recent issues.  Many of these solutions are in the process of being implemented in response to the British Polling Council’s inquiry that was released at the end of March 2016.  However, even if these changes to polling practice are successful, it will take a long time to rebuild public trust in opinions polls in light of the perception of failure. It is therefore vital to review the way in which opinion polls are reported and used in future political debate so that they continue to be relevant and we avoid the danger of throwing the baby out with the bathwater.