|
Welcome to the 226th edition of The Week in Polls (TWIP), a polling newsletter that always welcomes new readers. So please don’t keep TWIP a secret; share it with your friends and colleagues:
This time, with the Clacton Parliamentary by-election result now in, it’s a good time to look at how the polls perform when it comes to Reform’s vote share.
This is followed by summaries of the latest national voting intention polls, seat projections from MRPs and the most recent party leader ratings.
Subscribers who pay for the full edition can also read ten insights from the latest polls and analyses, including:
The coalition preferences for the supporters of different parties;
What people think of the Burnham government’s priorities; and
How attitudes towards climate change have shifted through the recent heatwaves.
If you are not yet a paid subscriber, you can read all ten by starting a free trial now:
For more polling news as it comes in ahead of next week’s edition, you can also follow The Week in Polls on Bluesky.
Before all that, a couple of updates on last week’s piece about research into whether polls systematically underestimate support for parties on the right. First, there is a companion piece of research about measuring polling error, which makes the case for using Mean Absolute Error (MAE). Very attentive readers will know I’ve tended to prefer the Root Mean Square Error (RMSE) which places greater weight on larger errors. The piece also points out that “research using this approach has found that polling errors have remained relatively stable over time. The performance of polls in the last decade broadly aligns with what would expect based on polling records from the 1960s onwards.”
Second, I played down the significance in the UK context of the average error found by the research. One of the co-authors has, however, highlighted its significance in the US context.
In addition, you may have noticed a typo in last week’s edition. In case it wasn’t clear, when I wrote of a “disproportionate interest politicos in the US take in the details of US politics (myself included)” I of course meant that first ‘US’ to be ‘UK’. Sorry for any bafflement caused by that and I’ve corrected the online version.
My usual reminder too: please do hit reply if you’ve spotted a poll, focus group or new piece of poll-based research that you think should be in contention for coverage in a future edition. Thank you!
And with that, on with the show.
Are the polls overestimating support for Reform?
 |
Last week I talked about evidence showing that overall polls tend to slightly underestimate support for parties of the right. The Clacton by-election result prompts a return to that question in a different guise: the record of by-election opinion polls in attempting to get Reform’s support correct.
There was one proper poll for the Clacton by-election, from Survation. There was also a super-small sample follow-up from the same firm.
The full poll put the Reform vote for Nigel Farage on 73%, the super-small sample put it at 79% and the combined sample across both gave 73% again. Reality, however, gave Farage 63%, a full ten points lower than the full poll.
In fairness, what Survation got right was the big picture: Farage a long way ahead and Count Binface by far the most popular of the others, on course for the best-ever result for a joke candidate.
But still, ten points is ten points.
Of course, some, most or all of that could be explained by the poll being carried out well ahead of polling day. It may have been spot on at the time the poll was carried out, with people simply changing their minds a bit by the time they voted. Or it may have been out. We’ll never know for sure.
In that sense, the Survation poll was rather like the Greater Manchester Mayor polls which I described as right but lucky in an earlier edition of TWIP. There are understandable reasons for polling well ahead of polling day in such contests, and the pollsters have got the overall picture right. So yes, the pollsters have been right. But they’ve also been lucky that there hasn’t been a big late swing of the sort that turns early fieldwork from a handy convenience into the source of a big polling miss. The gap between polling and voting also means that the differences between the vote shares in the polls and the actual results can’t be clearly ascribed to polling error, as they may instead have been due to people changing their minds. If the polls have been wrong, that’s been obscured from us.
There is though also another issue, which is how Reform’s vote share in by-election polls has compared with the actual result in the four Parliamentary by-elections since Nigel Farage returned to lead the party:
Runcorn & Helsby - 1 poll (Lord Ashcroft) - Reform vote share 1 point too high;
Gorton & Denton - 3 polls (Find Out Now, Omnisis and Opinium) - Reform vote share 7 points too high in one, dead on in another and 1 point too low in the third;
Makerfield - 5 polls (Survation twice, More in Common / UCL Policy Lab, Convergent and Opinium, ignoring the half-reported unidentified additional poll mentioned by The i) - one 6 points too high, two 5 points too high, one 4 points too high and one 2 points too high; and
Clacton - 1 poll (Survation) - 10 points too high.
Note: in calculating the errors in the Reform vote share I have compared the published vote share figures from pollsters, which were rounded off to the nearest whole number, and the by-election vote share, rounded off similarly. If unrounded numbers for each poll were used against the unrounded result, it is possible that the occasional figure would be 1 point different. But the rounded figures are the published ones and so the reasonable ones to use in this context. Had the Reform candidate polled 11 fewer votes in Makerfield, all of those errors would have been 1 point higher (as his vote share would have rounded down to 34% rather than up to 35%).
It’s only four by-elections so the evidence base is small and fragile. But we have eight polls putting the Reform vote share higher than it turned out, one poll being dead on and one poll putting the Reform vote share too low.
That is starting to be suggestive of a more general issue with by-election polls and the Reform vote share.
It also fits with what happened in the 2024 general election. Looking at polls whose fieldwork was within a week of polling day, i.e. started on or after 27 June, there were 25 polls of which 20 were on the high side for Reform, three dead on and only two underestimating Reform. If you restrict this just to the polls carried out in the three days before polling day, it becomes 8 too high, one dead on and one underestimation.
As some of these polls were published to one decimal point and some rounded off to the nearest whole number, I’ve taken whichever form the pollsters chose and compared that with the equivalent format for the Reform vote share.
Again, the limitations of this evidence are important to emphasise, especially the fact that only one of those 25 polls (from Matt Goodwin’s PeoplePolling) was more than 3 points out on the Reform vote share - which is a pretty darn good performance by the polls.
But it does all mean we have a growing pattern in Parliamentary elections - both general and by-elections - of Reform’s support when the votes are tallied being less than it was in the polls in advance of polling day. This pattern needs to be treated with caution and caveats, and it’s not a certain prediction for the future either.
It is though a pattern that’s there and one worth paying attention to.
Elsewhere from me…
If you spot a factual error…
Borrowing an idea from Stuart Ritchie and others, if you spot a factual error in an edition of this newsletter, I will give you a free 12-month subscription to the paid-for version (or an extra 12 months on your existing one). Factual errors qualify but grammatical errors or disagreements over interpretation do not, and my decision is final. Messages about both of those are always welcome, though.
Voting intention and leadership ratings
Here are the latest voting intention polls from different pollsters, and more polls than not in the last week put Labour ahead of Reform. Whether that will last or whether this is a short-term new-PM bubble, we’ll have to see of course.
 |
The table is also online here, along with explanations about what is included. It is updated regularly throughout the week as new polls are released.
Next are the seat projections from MRPs, also sorted by fieldwork dates. Because they are published infrequently, most of these are somewhat dated.
 |
Finally, here is a summary of the latest leadership ratings, sorted by pollster name. We’ve now got a well-established pattern of Andy Burnham ahead, and with Nigel Farage and Zack Polanski forming the bottom two, and neither of their ratings (yet?) recovering in any meaningful way.
 |
The table is also online here with additional details and is updated regularly throughout the week as new data comes out.
For historical figures on both voting intention and leader approval stretching back to the 1930s, including Parliamentary by-election polls, see PollBase.
Catch up on the previous two editions
 |
|
My privacy policy and other legal information are available here. Links to purchase books or other items are usually affiliate links which pay a commission for each sale.
Quotes from social media messages are occasionally lightly edited for punctuation and clarity.
If you are subscribed to other email lists of mine, please note that unsubscribing from this one will not automatically remove you from the others. If you wish to be removed from all my lists, simply reply to this email to let me know.
How coalition preferences vary by party, and 9 other polling insights you shouldn’t miss
The following ten findings from the most recent polls and analyses are only for subscribers who pay for the full edition, but you can sign up for a free trial to read them straight away.
Interesting new polling is out from JL Partners about how coalition preferences vary between supporters of different parties. ...
Continue reading this post for free in the Substack app
| ||||||||||
