Will the polls get the Scottish referendum right?

As the Scots goes to the polls Anthony Wells considers to what extent we can expect the outcome to match the predictions.

The Scottish polls at the end of last week and the weekend were broadly clustered around a small No lead. Perhaps a more likely route to a YES victory is if the polls are underestimating the level of YES support for some reason. Over the last couple of days I’ve seen several blogs or articles pondering whether the polls could be wrong, could they be underestimating YES or NO?

It would be hubris to suggest the polls couldn’t be wrong. Obviously they can. At most elections there are polls that perform better or worse than their peers, some of that is better methodology. When the polls are close most is probably just normal sample variation. That’s a matter for another time though, here I’m pondering more about the possibly that all the polls are wrong, the potential for a systemic bias with everyone a bit too yes or a bit too no. This is possible too – think of the way all polls overestimated Lib Dem support in 2010, or most famously how all the polls overestimated Labour support in 1992. How likely is that?

The Scottish referendum is a bigger challenge for pollsters than an election would be because it’s a one-off. In designing methodology for voting intention the experience of what worked or didn’t work at previous elections weighs heavy, and most companies’ weighting schemes rely heavily upon the previous election – if not directly through weighting by recalled vote, in using the data from the previous election in designing and testing other weighting targets. For a referendum you can’t take that direct approach, pollsters needed to rely more on modelling what they think is an accurate picture of the Scottish electorate and hoping it reflects the Scottish people well enough that it will also reflect their referendum voting intentions – it’s complicated because Scotland has a complicated electorate. Scottish voters have two Holyrood votes and a Westminster vote, and they use them all in different ways with different political loyalties. Within the space of a year Scotland managed to be a Labour stronghold at Westminster and to produce a SNP landside at Holyrood – using either election alone for weighting gives a rather different picture of what the Scottish electorate are like, even though you are trying to model the same population. Different companies have arrived at different methods of political weighting to deal with the issue – Survation, ICM and TNS weight by Holyrood recalled voted alone, YouGov weight by Holyrood recalled vote with a nod towards 2011 Holyrood voters who backed Labour in 2010, Opinium weight by Holyrood and Westminster recalled vote, Panelbase weight by Holyrood and European recalled vote, Ipsos MORI don’t use political weighting at all. Despite the variance they have all converged to produce the same sort of result, and that gives me some confidence – if there was a particular skew from being online or from using Holyrood recalled vote we would expect to see different results.

Continue reading

Could Scotland buck the trend and vote Yes?


The polls have been narrowing in Scotland’s independence referendum debate. Reading’s referendums expert Alan Renwick reflects here on whether the Yes campaign really could pull off an unexpected victory.

I wrote a piece for the Telegraph a couple of weeks ago pointing out that, if Scotland’s voters follow the patterns seen in most referendums around the world, they will vote No to independence on 18th September. The polls have consistently shown a lead for No. And opinion generally shifts in the direction of the status quo in the final stage of a referendum campaign.

Since then, however, the polls seem to have shown the reverse pattern: opinion seems to be shifting towards Yes, not No. So what is going on? Could Scotland really buck the trend?

The short answer to that is, yes, of course Scotland could buck the trend – it’s only a trend. While opinion generally shifts towards the status quo in the weeks before polling day, there are also exceptions. The evidence that we have so far certainly doesn’t show that Scotland definitely will buck the trend – the polls could easily shift again. Indeed, headlines about a narrowing of their lead are exactly what No campaigners need to mobilise their supporters and remind voters that this is not a game.

But what are the mechanisms at play here? Are there plausible mechanisms that could generate a shift towards Yes in these final weeks and an unexpected win for supporters of independence? Ever since I first wrote on this subject back in January, I have argued that there are two key mechanisms to consider: reversion point reversal and the anti-establishment bandwagon.

Continue reading