The Six Second Rule. How Opinion Polling is overused in policy development.
What can you do in six seconds? Walk thirty feet, hit the bad monster a couple of times with your sword, hide, or answer a question about a massively complicated area of public policy?
“A round represents about 6 seconds in the game world. During a round, each participant in a battle takes a turn.”
Six seconds. What can you do in six seconds? Walk thirty feet, hit the bad monster a couple of times with your sword, hide, blow the room up with a fireball or answer a question about a massively complicated area of public policy the aggregation of which will decide whether that policy proceeds. This is the six second rule.
And it is why the use of opinion polling as the front end for policy development is a bad idea. I understand that six seconds is plenty of time for your big paladin with a sword to chop lumps off the ogre. I, just about, appreciate that it’s enough time for the wizard to flip open his book and read some words while rubbing some herbs between his fingers (*kaboom*). And that in six seconds the dodgy guy with the knife can dive into some shadows from where he will jump you. But I really don’t think that six seconds is enough time to decide whether we should allow assisted dying, if we should cap the pay of business leaders and owners, or carry on stopping people from building new houses.
Yet that is increasingly how policy choices are determined. The world of clever people with principles discussing what the right policies are to make people healthier, happier and wealthier is gone. Now we have ‘science’. The analysis of options, any consideration of how a given policy might have second order consequences - these things are gone, replaced with ‘we can’t do the thing that would make everything better because it doesn’t poll well’. We are captured by the six second rule.
Back in the days when I was commissioning market research none of this existed. Quantitative research was expensive. You had to pay people (my Mum did this for a while, I remember sitting in the back of the Ford Anglia arguing with my siblings while Mum interviewed someone about soap powder) and it took a deal of time from the questioning to the results of the survey. Today you can whack out a survey question to your Internet panels and have enough data to post results within 48 hours. All for next to nothing.
Six seconds though. It is a load of time. Plenty of time to remove the head of that goblin over there or to pick the lock of a treasure chest. But is it enough time to choose whether we should allow the state to kill people who, maybe, want to be killed? Is it enough time to decide whether we should cap rents, restrict executive pay, or invade Denmark?
It is no criticism of today’s polling companies to observe that the Internet changed everything. And, for those involved in developing policy ideas, in the world of think-tankery, opinion polling became a lazy way to generate copy and content. You don’t have to do any thinking, you just run a poll about Steve’s crazy idea. This is great news for the polling business but terrible for the, arguably more important, business of getting public policy right. Because, damn it, sometimes the right thing to do isn’t the thing that the public would - in just six seconds - decide is the right thing to do.
The problem is, however, that this isn’t how the policy thing works. People have talked about ‘post to policy’, where the clever kids in a think tank get their crazy twitter idea adopted and there’s probably too little attention given to the now commonplace process of ‘media outrage to law’ (especially with distraught parents getting a law sponsored that, only tangentially, relates to why their child died). When the studious assessment of the problem directs those thinking about policy to ideas that - as with social care, housing policy, and tax - don’t ‘poll well’ we have a problem. And a bigger problem when policies we know are a bad idea - rent controls, maximum wage caps, nationalisation - are popular with these instant polls.
I once got a letter, an actual physical letter, from the finance director of a big client. The finance director was concerned about the veracity of some market research we’d done around a new pension investment product they were launching. To be brief he didn’t think what we’d found was true. Not because what we’d done was bad or wrong or even that it differed from what that finance director wanted. It was about sample size, the balance between qualitative and quantitative research and that the director wanted to be sure about the science of the findings. My response was brief: “what is truth?”.
We’ve been trained in the idea that, if a majority supports a given policy position, then that is true. Even when we know that something isn’t true, we still go along with the policy because it polls well. And when someone mentions that, just maybe, the emperor needs a different tailor the response is, so often, a combination of ‘this is my truth’ and ‘people when polled support the idea of getting sunbeams from cucumbers’.
When the YouGov or More in Common email drops into someone’s email box there isn’t a great deal of attention given. If you’ve a minute or two to spare, what you’ll do is open the email and click on some answers (there’ll be points to get after all). And you won’t give any answer any more time than it takes for that paladin to take the head off a goblin or two: about six seconds. So when the poll results come out - “poll shows public support decapitating goblins” - we don’t ask how much actual attention respondents to the survey gave to the question.
Public opinion is not truth. Yet so much important policy making involves the mistaken belief that things we know are true (see above on truth being often subjective) can’t be allowed to be true because the public - when polled - would rather they weren’t true. We are trapped in a world where everything gets determined by how a bunch of online people who aren’t paying much attention - six seconds of attention - react. YouGov, polling a load of six second pings from their panel, tell us that the public support bosses not being allowed to get paid more than ten times to lowest paid workers. The advocates of this policy aren’t doing so because they think Erling Haarland shouldn’t be paid over ten times what a cleaner at the Etihad Stadium gets paid. No, they’re advocating the policy because of those thoughtless, six second, responses to a YouGov poll from people who also aren’t thinking about Haarland or Phil Foden or Bernardo Silva but an anonymous suited fat cat.
When I was commissioning market research it was an expensive business. Today, for the multitudinous polling companies, it is cheap. You’ve got an Internet panel and sending them an email is almost free, certainly compared to the cost of a survey in the 1990s. As a result policy wonks gradually become polling wonks. We have seen considered reasoning get replaced by the aggregation of responses from people who have no context for a given policy and are going to give that question six seconds of their precious time. Then, when the responses are collated, the policy wonks decide what is the best policy. Not on the basis of understanding what works, not through the assessment of trade offs but because a bunch of not quite random respondents did or didn’t like an idea they’d considered for the length of time it took the barbarian to hit the orc twice with her axe.
I still think market research is useful and important but it is a poor substitute for “test and learn”. We know that, for example, very high tobacco duties and vaping bans result in a violent black market but because polls tell us these prohibitions are popular we carry on with the stupid policy. The same goes for other foolishness like rent controls, salary caps, and price restrictions. If you ask a bunch of people if they think limiting rises in the price of eggs is a good thing they’re going (after six seconds mulling it over) to tell you yes. Which is fine until the cap is introduced and suddenly there aren’t any eggs at the supermarket. Developing better policy doesn’t follow from opinion polling. Indeed the tendency to lean on cheap opinion polls as the foundation for developing policy has bad outcomes. Not only do we get unworkable ideas acted on because they are popular, we also get a set of wicked issues (social care, housing, health) that don’t get the attention they deserve because every substantive proposal for reform turns out to be unpopular.
Next time you see the results of some polling splashed in the newspaper or pinging into your social media feed, take some time (six seconds maybe) to remember that the poll is an aggregate of responses from poorly informed, unengaged people. And also that those people have given their response less time than a single Dungeons and Dragons combat round.