Stephen Fisher and Alan Renwick have developed a method for forecasting the outcome of the EU referendum based on current vote intention polling and analysis of opinion polling from previous referendums in the UK and around the world. Last month, in their initial forecast, they suggested that Remain had an 87 per cent chance of winning. In this second forecast this has now dropped to 73 per cent. 

A month ago we issued our first forecast for the EU membership referendum on 23 June . Based on an analysis of referendums in the UK and on the EU outside the UK, and on vote intention opinion polls we forecast that Remain had an 87 per cent chance of winning, and that Remain would get 58 per cent of the vote, plus or minus 14. This was in part based on our polling average (excluding don’t knows) of 55 per cent for Remain on 11 March.

Our current forecast suggests that the contest is a fair bit closer. Our polling average now puts Remain on 52 per cent. We now give Remain a 73 per cent chance of winning and estimate that the Remain share of the vote will be 54 per cent, plus or minus 13 points.

The key change here is the drop from 55 per cent to 52 per cent for Remain in the polling average. The main reasons for this are as much or more methodological than substantive.

Over the past couple of months the telephone polls have shown declining support for Remain (before and after excluding don’t knows). But there has been little or no trend in the online polls over the same period. So rather than both kinds of poll moving in lock-step, the much discussed gap between phone and online polls in levels of support for Remain (aka the mode effect, see here, here and here) has declined substantially.

Ultimately it is impossible to tell for sure which, if either, of these kinds of poll has been accurately reflecting changes in public opinion. It would be too bold to assume that online polls were getting it right and not phone, or vice versa. Instead we try to estimate the average pattern of change across pollsters. Assuming that pollster (house) effects have been stable over the past month and a half, averaging over some regression models with slightly different specifications suggests around a one point drop in support for Remain (excluding don’t know) over the last month.

Over the same period our estimate of the difference in the Remain share of the vote between telephone and online polls has shrunk from nine to five points. For our polling average, this means that whereas we previously adjusted telephone polls down and online polls up by 4.4 points for Remain, the adjustment is now just 2.5 points. In retrospect we think that our previous mode effect estimate for our first forecast was too large and so we have switched to a more conservative method for estimating the mode effect. Still most of the drop in the size of the adjustment is due to new polls over the last month suggesting that the mode effect has genuinely diminished.

In sum, the main source of change since our 11th March forecast ­– a 3-point drop for Remain (exc DK) in our polling average – is primarily made up of a (roughly) 2-point narrowing in the telephone vs online mode-effect adjustment. There appears to have been one-point drop in support for Remain in actual public opinion over the last month.

The other component of the forecast share of the vote that has changed is the amount by which we expect Remain to rise in the polls over the course of the campaign. Whereas we previously expected Remain to gain 1.5 points, we now expect only half a point. This purely reflects the difference between analysing historical polls over the final 75 days instead of the final 105 days. It does not mean that we were expecting Remain to have risen by a percentage point over the last month specifically. Although clearly the polls provide no evidence for any of the progress that Remain was expected to make.

As before there is still substantial uncertainty. With such a tiny (0.5 point) estimate for the direction of change now, the forecast is essentially that trends in the polls for the remainder of the campaign could go either way.

We are still expecting Remain to outperform the final polls in the actual referendum by about a point and half. Again that is a modest central estimate and accompanied by substantial uncertainty, meaning that it is only just a bit more likely that the polls under- rather than over-estimate Remain.

The forecasting method still makes no attempt to anticipate the possibility that supporters of one side turn out at a greater rate than supporters of the opposing side. Polling evidence so far suggests that Leave supporters are more likely to vote than those who favour Remain. One reason for us not doing anything about this (for the moment) is that some pollsters are already doing turnout weighting while others are not, and trying to account properly for which pollsters have done what is tricky. Turnout weighting will ultimately be a responsibility for pollsters and as the referendum approaches presumably more of them will be doing it.

While differential turnout has the potential to have a massive effect on the outcome, trying to estimate what that effect will be this far out is difficult. If this becomes a referendum that people care a lot about and nearly everyone turns out for, then, mathematically, there will little scope for different groups to turnout at very different rates. The lower overall turnout is the more differential turnout is likely to matter. Predicting the extent to which people will be interested and care about the outcome is important for estimating both overall and differential turnout. There are ways of going about this but all fraught with considerable uncertainty. So another reason why, at the moment, we are refraining from making differential turnout adjustments is that current patterns of reporting of likelihood to vote in the polls may not prove that reliable an indicator of future overall or differential turnout.

Thanks again to Jack Sheldon, Roberta Damiani and Johnny Runge for data gathering. We are still planning to write more about both our analyses of public opinion in the run-up to referendums and our forecasting method.

This post was co-published with Elections Etc.

About the authors

Dr Stephen Fisher is an Associate Professor in Political Sociology and the Fellow and Tutor in Politics at Trinity College, Oxford.

Dr Alan Renwick is the Deputy Director of the Constitution Unit.