On 17 April, the House of Lords’ ad hoc Committee on Political Polling and Digital Media published a report, following its inquiry into the effects of political polling and digital media on politics. At an event organised by The Constitution Unit, Lord Lipsey, who chaired the Committee, discussed the report with a panel that consisted of Baroness Jay of Paddington, a Labour peer who served on the Committee; Will Jennings, of the University of Southampton; and Martin Boon, a professional pollster. Dave Busfield-Birch offers a summary of their comments.

Following an inquiry that took evidence from a variety of experts, industry professionals, and ministers, the Committee on Political Polling and Digital Media published its report on the subject on 17 April. The Constitution Unit organised an event to publicise the release of the report, which consisted of a panel discussion (summarised below) and a lively and interesting Q&A session. Committee Room 2 in the Palace of Westminster was full when Jennifer Hudson, Senior Lecturer in Political Behaviour at the UCL Constitution Unit, introduced the panel, on which she served as Chair. Lord Lipsey and Baroness Jay of Paddington introduced the report on behalf of the Committee. They were then followed by Will Jennings, Professor of Political Science at the University of Southampton, and Martin Boon, who provided the perspective of a professional pollster.

Lord Lipsey

As chair of the Committee, Lord Lipsey noted that he had enjoyed working on the inquiry that produced it, although he did acknowledge that the report was ‘slightly unusual’ in one key respect. Normally, parliamentary inquiries examine government policy, and the recommendations in their reports are aimed at influencing it. This report, however, had focused its attention on the workings of the polling and digital media industries and it is they who are the targets of most of its recommendations. One recommendation that was intended to influence government policy called for the Electoral Commission to have a wider statutory role in regulating and monitoring polling during election periods.

Lord Lipsey then went on to offer some background to the report, saying that it had partially been prompted by the existence of three big polling ‘bloopers’ in recent British political history. In 2015, polls had widely predicted a hung parliament; instead, the Conservatives secured a parliamentary majority. At the next general election in 2017, the Conservatives experienced an unexpected result in the opposite direction: where polls had predicted an increased majority for Theresa May, the voters delivered a hung parliament and a government that now relies on DUP support for its parliamentary majority. Finally, the referendum on leaving the European Union produced a vote for Brexit that the polls had largely failed to predict. Lord Lipsey was careful, however, to point out that despite these three unexpected results, people should be careful of jumping to conclusions about the state of the polling industry. The Committee found no statistical evidence that polls are getting worse internationally. However, he did warn that the failure of polls to predict three otherwise unexpected results in succession would mean that pollsters should expect ‘not to get much sleep’ during the next general election campaign.

Speaking generally about how polling worked in the modern era, Lord Lipsey pointed out that the practice of getting an accurate poll with a representative sample had become more difficult. For example, refusal rates are higher now than they would have been in previous decades. This makes getting a good number of responses more time consuming. In addition, online polls tend to attract people who are generally interested in politics and therefore can offer a sample of the electorate that is not representative. Lord Lipsey also noted that social class was no longer a reliable indicator of voting intention. As the class system had ‘disintegrated’, a person’s economic background has become a less reliable predictor of how they will vote. One category that pollsters are starting to focus on instead is education, although identifying groups posed difficulties of definition; higher education can mean a number of things, for example. Age is also a category that is receiving greater attention as an indicator of intention.

Lord Lipsey then turned to the influence of polls, starting with the premise that people generally have an unrealistic view of what pollsters are capable of and that we often read too much into numbers that actually tell us very little. This, he said, is not the fault of pollsters. Secondly, he pointed out that there was little evidence to show that poll numbers themselves affect voting intention. However, this does not mean that polls don’t matter. In 2015, almost every poll predicted a hung parliament. This changed the nature of the election from a debate about the merits of the coalition’s austerity policies to a discussion of the ‘horse race’ and the possible coalition combinations of a hung parliament that never came into existence. Broadcasters, politicians, and voters need to focus more on policies, according to Lord Lipsey, rather than allowing numbers to drive the narrative during election periods.

Lord Lipsey then turned to the recommendations in the report, noting that the Committee had opted not to call for a ban on political polling during election periods, as is the case in a number of other European countries. However, he did note that if pollsters ‘keep getting it wrong’, then pressure for a ban may increase.

Instead of a ban, the Committee had recommended a change in the way polling is overseen, expressing a preference for an improved form of the self-regulation that currently operates. Although France has statutory regulation, Lord Lipsey expressed a lack of satisfaction in how the system operates in practice, which has led the Committee away from that line of thinking. As part of a new improved system, the Committee saw a more involved role for the British Polling Council, and Lord Lipsey reported that this had been responded to ‘enthusiastically’ by the Council itself. Amongst the polling industry, Lord Lipsey said that he had seen a recognition that a failure to ‘sort their own house out’ would increase the pressure on policy-makers to consider a more involved role for both government and parliament. Although Lord Lipsey accepted that improved self-regulation would involve higher costs for the polling industry, he said that this had to be viewed as an investment in its future, and urged the industry to take action in relation to the Committee’s recommendations.

Baroness Jay of Paddington

Baroness Jay prefaced her remarks by pointing out that she had been somewhat ‘naive’ and not particularly knowledgeable about the subject of polling prior to serving on the Committee. She did warn, however, that she felt everybody should be less interested in polls as a guide to the intention of voters and that we ‘should not be treating [polls] so seriously.’

Baroness Jay also expressed concern for how polls are reported, echoing Lord Lipsey’s remarks about how poll results tend to drive the narrative of elections.

Moving on to the recommendations of the Committee, she chose to highlight its view that the Electoral Commission should be more involved in polling during election periods, acknowledging that this would be an extension of their remit and would therefore require legislation.

Professor Will Jennings

Professor Jennings first addressed the question of whether or not there was a crisis in the political polling industry, and repeated Lord Lipsey’s warning that there was no empirical basis for asserting modern polling to be less accurate than it had been in the past. As proof of this, he stated that the average level of polling error had not changed significantly over the course of the past several decades. He also noted that polling is by no means a homogenous entity and that methodology and the industry itself are constantly changing.

Moving on to the issue of regulation, Jennings said that political polling is a loss leader, that is often used as much to boost a company’s profile as for anything else. Should regulation increase the expense involved in political polling, then some firms might find themselves opting out of it altogether.

On the recommendation that an advisory service needed to be introduced, Jennings was open to the idea, but cautious about how effective such a body would be. He was more supportive of an increase in training opportunities for journalists so that they could better understand how polling works and provide more informed reporting on poll results. However, even if standard media outlets improve their reporting, this does not solve the problem of how social media interprets polls. Given this new age of social media, he thought it was doubtful that a focus on formal regulations for print and broadcast media would make these problems go away.

Martin Boon

Martin Boon offered a personal view as someone who had spent over 20 years in the polling industry. He was generally approving of the report’s recommendations and recognised that the context for it being launched was one of a recent failure of polls to accurately predict election results. Although he explained that the job of a pollster was a very difficult one, he expressed recognition of the overall climate as one that required the polling industry to ‘learn its lessons’ following a period in which pollsters had ‘not covered ourselves in glory’. He compared the situation now to that following the 1992 general election, the outcome of which pollsters had also failed to predict.

Addressing the question of whether or not pollsters can be trusted, Boon said that it was ‘gratifying’ that the Committee had recognised the personal and collective integrity of political pollsters, who he said are strongly focused on obtaining accurate snapshots of public opinion and always strive for accuracy. Like Lord Lipsey, he noted that obtaining accurate results from a representative sample had become more difficult. By way of example, he said that whereas 4,000 phone calls would once generally be needed to generate 1,000 responses, the shift to mobile telephones and other factors now meant 30,000 calls were needed to generate the same number of responses.

Turning to the recommendations, he was complimentary, saying that they were ‘pretty much spot on’ and that he was surprised the Committee had not gone further by proposing formal state regulation or an outright ban. Boon said that the decision against advocating a ban was wise, saying that a lack of published opinion polls would create a vacuum that would be filled by politicians stating political support as a fact without any statistical analysis to contradict them. For that reason, a ban on polling reports during political campaigns would, in his view, be dangerous without a similar ban on politicians discussing levels of support for their party or that of an opponent.

Boon said that the call to revise guidance given to journalists and the need to conduct ‘post-mortems’ of polling outcomes once an election had finished were both worth pursuing. He was more cautious about the idea of an advisory service on polling design, which he thought would find it hard to respond to new methods, such as telephone and online polling, which had once been unprecedented innovations but were now accepted as standard practice. ‘Most difficult’ of all in this area was how the advisory service would deal with the design of individual surveys, whose questions are subjective. Experienced pollsters, he reasoned, were therefore the only people who could effectively deal with the issue of ‘good’ and ‘bad’ poll questions. However, he was aware that issue polling in particular raised matters of question framing and recognised that in some cases it was clear that questions had been designed to elicit particular responses. Boon agreed this was a cause for concern.

In conclusion, Boon said that the report was a fair one that recognised the difficulties faced by the polling industry, which he predicted would respond favourably to its recommendations.

The report of the Committee on Political Polling and Digital Media can be found here. Other publications of the Committee, including written and oral evidence can be found on the Committee’s pages on parliament.uk

About the author

Dave Busfield-Birch is editor of the Constitution Unit blog. He also edits Monitor, the Unit’s regular roundup of constitutional events. The latest issue, Monitor 68, was published in March.