As an avid reader of political commentary I’m always keen for writers, especially those who make lots of predictions, to reflect on what they got right/wrong and why. Not many do it, perhaps because it seems self-indulgent, but when they do, like Stephen Bush, I always find it useful. Perhaps more importantly I think doing it will help keep me honest. If I know I’m going to have to publish this piece every year it acts as a further incentive to try and be as accurate as possible rather than let my biases get the better of me.
Elections
I’m starting with my strongest area this year. I predicted three elections and was pretty close to the outcome for all of them:
UK local elections: I said the Tories would lose 11 councils and 300 councillors. They lost 11 councils and 485 councillors (doing somewhat worse in London and southern district councils than I expected). I highlighted all the councils the Tories went on to lose in the preview, included Barnet, Wandsworth and Westminster, which was the main news story.
US midterms: In my first post on the midterms I said “if you put a gun to my head I think the Senate stays 50-50 with the Republicans winning Nevada and the Democrats winning Pennsylvania [and all other states staying with the incumbent].” And that the House would go Republican. This was all correct except for Nevada, which the Democrats held by less than 10,000 votes.
French Presidential election: when Marine Le Pen briefly pulled even with Emmanuel Macron in the polls I said I thought Macron still had a 70-80% chance of winning for three reasons (that Macron would actually do some campaigning which he didn’t during the first round; scope for Macron to pick up more of the Mélenchon vote than polls were showing; differential turnout). These turned out to be right.
While I will inevitably get elections more wrong than this in the future I think I have got better at predicting them since completely misreading the 2017 UK general election, which led to a lot of self-reflection. Since then I’ve been rigorous about focusing on data, both polling and historic patterns, rather than listen to any instincts I think I have about what voters will be thinking.
Boris Johnson’s Fall and the Leadership Election:
Here’s where my record starts to get more mixed. I was always convinced that Boris Johnson would not make it through 2022 and was one of the first to write about his demise last November before we even had the substack. But I was wrong in thinking he would lose the confidence vote in June (I put it at 70% likely). Knowing there were around 160 votes to get rid of him I assumed that other Tories, particularly Ministers, would rationally realise he couldn’t continue and switch. Instead, while 148 MPs voted against him, on the whole Ministers stayed loyal. It would have been rational to switch – they ended up having to dump him in a much messier way a few weeks later – but the lesson here is don’t try and play game theory with politicians. Political loyalty is an emotional and complex thing. A lot of Tory MPs think they owe their seat to Boris and knifing him was a much more difficult decision that I acknowledged.
On the contest itself I was broadly right about the candidate list, and correctly anticipated Ben Wallace wouldn’t run even when he was favourite in the betting. But I made two big mistakes. First I misread Sunak’s resignation. I thought he seemed dejected and beaten and his odd resignation letter reflected that. But actually it was positioning – it focused on economic differences rather than Partygate with the aim of trying to avoid looking like he was betraying Boris (which obviously didn’t work).
The worst error, and I think the worst I made all year, was thinking Penny Mordaunt and Liz Truss were fighting for the same MPs’ votes, which led me to overestimate Mordaunt’s chance of winning and underestimate Truss. This was just lazy on my part. I assumed because Mordaunt was a fervent Brexiteer and ERG member that she would be popular on the right. I hadn’t clocked at all that her book and her support for trans rights had positioned her as too “woke” for that section of the party.
By the time Johnson was thrown out and the contest started I had spotted and acknowledged the error and did say that I thought it would end up Sunak vs Truss (albeit with low confidence). I kept underestimating Truss though. And in my piece on the final two I said I’d bet on Sunak – which I did unfortunately – because, despite Truss being ahead in the membership polling, I thought it would become clear than Sunak was more attractive to the wider electorate. She did fall away in the polling amongst the general public, as I expected her to, but only after most members had voted so it didn’t have time to affect the result. Moreover, I underestimated how much Tory members would blame Sunak for Johnson’s demise, and how quickly they’d forgive Boris for his egregious behaviour.
The Truss Interregnum
I had a headstart on Truss as I’d worked with her a decade ago. My reading of her turned out to be pretty accurate:
“She is chaotic and eccentric, with a manic energy. She grabs hold of random ideas and forces everyone around her to spend inordinate amounts of time talking her out of them. Everyone, from officials to the other Ministers, found her difficult to work with. If you’d told me then that one day she’d be favourite to be Prime Minister I’d have laughed you out of the room.”
I also said
“Truss may turn out to be better than everyone expects, but there seems to me a much bigger risk of a catastrophic outcome for the party, such as another leadership election within a year, a group of MP defections to Labour, or an election defeat on the scale of 1997 when the Tories won 165 seats, and were out of power for 13 years.”
But then Camilla Long wrote a Sunday Times column that accused me of being sexist for saying this (without naming me) and lots of people I respect on twitter said I was being too dismissive of Truss’s political abilities and I started to doubt myself. This was exacerbated by my failure to predict Truss winning the contest. So when she started as PM I started forcing myself to look for the positives. My post on her cabinet selections correctly identified her main weaknesses but was still too generous about the choices she was making (including on Kwarteng). I even made myself write a post on how she could win an election, though it wasn’t very convincing.
Once it became clear what a disaster her budget had been I felt vindicated in my initial judgement and predicted she would not survive long. The lesson here isn’t anything as glib as “have the courage of your convictions”, after all most of the time when lots of smart people tell me I’m wrong it’s because I’m wrong. It’s generally a good idea to take that into account. But perhaps when I have access to additional information – in this case spending six months working with her – I should weight that higher versus disagreement from others. Though no one, not even her fiercest critics, would have expected things to unravel as quickly as they did.
The Bigger Picture
It’s harder to audit pieces where I didn’t make specific numeric or either/or predictions but the posts I’m probably happiest with this year are the two “Coming Storm” ones I wrote in August. While they focused on a prospective Truss premiership that finished earlier than anyone expected, the challenges outlined in the first post, which went up to this Christmas, have come to pass, and have been the issues to dominate politics (energy/NHS collapse/strikes). The second post, which looks at 2023, will hopefully end up being slightly too pessimistic on the economy and energy, though things are pretty bad, and it correctly anticipated the Tories would have to raise taxes, even though at that point they were arguing about how big tax cuts should be.
The wider point I made at the end of that post is a theme that I’ve been developing across multiple pieces:
“A proper analysis of why we’re here, and why our systems seem less resilient than those in other large, wealthy, countries, has to start with an honest appraisal of what’s gone wrong. That means acknowledging many of the trends in the development of the British state over recent decades have been in the wrong direction. We are horribly overcentralised; we invest far too little in infrastructure; have a housing market designed entirely around the interests of existing homeowners; are dependent in too many sectors on private companies prioritising profit over the interests of citizens; and yet are simultaneously reliant on a monolithic national health service that is barely coping. Our politicians aren’t being honest about any of these things.”
I looked at some of the reasons why this might be happening in a post earlier in the year on why British politics is broken but it’s a theme I want to keep digging into over the next year. My view is, increasingly, that some of the structural foundations of our political system need to be changed if we are to pull ourselves out of the doom loop we find ourselves in.
As for the next election I wrote a piece in February saying I thought a Labour majority was undervalued at the time which holds up well as we reach the end of the year:
“But even if either Sunak or someone else does surprisingly well, they’ll still have an almost impossible job. All of the fundamental metrics look terrible. People will feel materially poorer in 2024 than they did in 2019 due to inflation and low growth. On current projections they’ll be, on average, poorer than they were in 2008. According to Government analysis released this week NHS waiting lists will still be growing – 15% of the entire population will be on one by then. Crime is rising again too. These problems have all been exacerbated by Covid but they are tied to a lack of long-term thinking about economic productivity and public service reform. And they are pretty much unfixable in the time available to any new leader. It’s the perfect set up for a “time for a change” election with no obvious opportunity to repeat the trick Johnson used in 2019 by presenting himself as the change.”
A Labour majority is now, I think, valued about right at a 50% chance, with a hung Parliament at 40%, and a Tory majority at 10%. Tory chances of preventing that majority rest largely on Sunak’s personal popularity – which while deteriorating alread is still substantially stronger than his party’s – at a level of divergence we’ve not seen before. The reasons for this are something else I want to come back to early in the new year as I think they will underpin what happens in British politics next year.
Lessons
Out of all this I’ve drawn three key lessons which I’ll try to take into next year’s analysis:
1) The most important is that I still too often make the error of assuming groups containing people with very different views/experience to me will nevertheless think like me. Thus assuming Penny Mordaunt would appeal to right-wing Tory MPs or that the Tory membership wouldn’t quickly forgive Boris Johnson. Two ways I can improve here are a) to keep building wider networks that give me more insight (while avoiding the trap of being too influenced by whoever I’ve just been speaking to); and b) be more cautious about going against a consensus when I have limited insight into the group under analysis.
2) Where I do have privileged information, either from personal experience or from data others have ignored, then that should make me more confident to go against the consensus, as I did initially with Liz Truss.
3) I still have a slight tendency to overweight what’s happening now vs the bigger picture. For months I thought Sunak was favourite for the leadership and then a few negative stories about him made me think he wouldn’t run. Likewise I think I underestimated how quickly things would settle once Truss was gone. One of the hardest things about political writing is not overreacting to whatever’s in the news at the moment. I’ve got better at it but still work to do.
On the plus side I think my overall analysis of where British politics is at the moment has held up well during the year. And my use of data to make election predictions has been good. Do let me know if you think I’ve drawn the wrong lessons here, or missed an error I made during the year, either in the comments or my email.
************
This is the last post before Christmas so I hope all subscribers have a great break. Many thanks to you all, without you we wouldn’t be able to do this. If you’re struggling to think of a last minute gift for the politics obsessive or international relations analyst in your life then you can give gift subscriptions by clicking the button below….
Good to read this Sam. I love the fact that you're reflecting on and assessing your own performance. I remember it was one of Dom's big themes that cut through with me that pundits never got their old predictions held up to them.
(I try to incorporate a version of this open reflection at the end of Board meetings I chair - how did that meeting go, how did it feel, where did we waste time, how could we be better? Some people find it uncomfortable but too bad! Feedback forms may seem more candid but they never get acted on and are just make-work for clerks.)
I think at the end you're moving towards identifying some of your cognitive biases and trying to consciously correct for them. Maybe you could do this more explicitly, running through a list of the main ones and taking them in turn? You're saying you get swayed by recency bias, for example (a form of proximity bias). Be interesting to look at the top 10 cognitive biases [or whatever].
Have a good Christmas.
David
This is (another) interesting post both the honest content and analysis but also the reality of undertaking such reflection. When a school senior leader I tried to encourage more reflection but I wish I had made it more an even more formal and proper process. Too often initiatives come into school from individuals from other bodies all with good intent but the mix doesn’t always work. Better feedback/reflection I think would have helped and in turn reduced some workload and improved standards. I recall an Ofsted inspector at an INSET encouraging new ways to give pupils feedback and I so longed for a sept lead in the discussion to say “well we are not changing, we think hard what we do and we get it right” which many did.
You’ve made me think about how we reflect in our professional work
John Dexter