There you are, Bernard. The perfect balanced sample.

Bernard Woolley: The party have had an opinion poll done and it seems all the voters are in favour of bringing back National Service.

Sir Humphrey: Well have another opinion poll done to show that they're against bringing back National Service.

Bernard Woolley: They can't be for and against…

Sir Humphrey: Oh, of course they can, Bernard! Have you ever been surveyed?

Bernard Woolley: Yes, well, not me actually—my house. Oh, I see what you mean.

Sir Humphrey: You know what happens. Nice young lady comes up to you. Obviously you want to create a good impression, you don't want to look a fool, do you?

Bernard Woolley: No.

Sir Humphrey: So she starts asking you some questions: Mr Woolley, are you worried about the number of young people without jobs?

Bernard Woolley: Yes.

Sir Humphrey: Are you worried about the rise in crime among teenagers?

Bernard Woolley: Yes.

Sir Humphrey: Do you think there is a lack of discipline in our Comprehensive Schools?

Bernard Woolley: Yes.

Sir Humphrey: Do you think young people welcome some authority and leadership in their lives?

Bernard Woolley: Yes.

Sir Humphrey: Do you think they respond to a challenge?

Bernard Woolley: Yes.

Sir Humphrey: Would you be in favour of reintroducing National Service?

Bernard Woolley: Oh, well, I suppose I might.

Sir Humphrey: Yes or no?

Bernard Woolley: Yes.

Sir Humphrey: Of course you would, Bernard. After all, you told her you can't say no to that. So they don't mention the first five questions and they publish the last one.

Bernard Woolley: Is that really what they do?

Sir Humphrey: Well, not the reputable ones, no, but there aren't many of those. So alternatively the young lady can get the opposite result.

Bernard Woolley: How?

Sir Humphrey: Mr Woolley, are you worried about the danger of war?

Bernard Woolley: Yes.

Sir Humphrey: Are you worried about the growth of armaments?

Bernard Woolley: Yes.

Sir Humphrey: Do you think there's a danger in giving young people guns and teaching them how to kill?

Bernard Woolley: Yes.

Sir Humphrey: Do you think it's wrong to force people to take arms against their will?

Bernard Woolley: Yes.

Sir Humphrey: Would you oppose the reintroduction of National Service?

Bernard Woolley: Yes.

Sir Humphrey: There you are, you see, Bernard. The perfect balanced sample.

It's funny because it's true. And we know it's true because Ipsos re-ran the experiment in 2024. Keiran Pedley fielded both sequences to a representative quota panel of 2,158 British adults; half got the pro-priming questions, half got the anti. The pro half came down in favour of National Service. The anti half came down against. Pedley's takeaway, with the polling industry's customary understatement: "Public opinion is rarely set in stone."

But what's this got to do with you?

This week we've been advising on the design of a major UK statutory consultation. The kind where wording has to stand up to judicial review. If we get it wrong, a hostile claimant could re-field a counter-survey with neutral wording and produce a different headline figure—and the difference itself becomes evidence of priming. That's a Gunning-grade defensibility risk that would invalidate the results and waste a butt-tonne of taxpayers' money.

The instrument has to survive being argued against. And it has to do its second job too—every press write-up, every Cabinet conversation, every regulator brief leans on the headline figure. If the figure was generated by a primed instrument, the comms strategy is built on quicksand. That's the difference between announcing a successful programme and defending a lame duck.

Most CX teams get this, of course. Wise and dashing CX Corner readers know agree/disagree formats inflate support 5-15%. That one evaluative word—"forbid" versus "not allow", "welfare" versus "assistance to the poor"—moves marginals by 10-20 percentage points.

So why are we still seeing primed instruments shipped to subscribers and shareholders? Because most teams audit for the obvious mistakes: double-barrelled questions, undefined jargon, unbalanced scales—and miss the subtler ones. The ones Sir Humphrey just demonstrated.

Sir Humphrey's four biases

Worth explaining them, because we see them all the time in well-meaning surveys.

Acquiescence bias

Bernard says yes to everything. He's not stupid—he's being polite, and "yes" is the path of least cognitive effort.

Yes doesn't require an argument. No does. By the time the interviewer asks "are you worried about youth unemployment?", Bernard has no counter-case ready, and saying anything other than yes would feel argumentative for no reason.

Each successive yes is cheaper than the no it would take to break the pattern. Krosnick's full satisficing model formalises this. To answer a question properly, a respondent has to interpret (work out what's being asked), retrieve (dig up the relevant memories), integrate (combine them into a judgement), and map (translate the judgement onto the response options). Four steps, all cognitively expensive. Most respondents won't do all four—they'll satisfice, picking the response that looks acceptable based on cues in the question. The agree end of an agree/disagree scale is the cheapest cue available.

Construct-specific scales close the gap. "How reliable is the train service in your area? Very unreliable to very reliable" beats "I agree the train service is reliable" every time.

Framing and priming effects

The first five questions in each Sir Humphrey sequence frame the sixth. By the time Bernard reaches the policy question, his answer is almost predetermined.

People don't react to outcomes in the abstract—they react to whether the same outcome is framed as a gain (what we'll get) or a loss (what we'll have to give up). Kahneman and Tversky's prospect theory shows losses weigh roughly twice as heavy as equivalent gains. Same outcome, opposite frame, different answer.

The cleanest modern demonstration: a 2021 Findings paper by Ralph, Klein, Thigpen and Brown. They surveyed 600 US adults on the same transport policy, asked two ways:

• Gain frame: "Transportation policy should... a) make it easier for most people to drive for most trips, OR b) try to shift more trips toward public transit, walking, and bicycling." → 63% support for option b.
• Loss frame: "A central goal of transportation planning should be... a) to make driving convenient and to eliminate traffic congestion, OR b) to reduce driving." → 34% support for option b.

Twenty-nine-point gap from a single wording flip. The gap held across almost every demographic cut—gender, age, race, education, party, car ownership, traveller type. Only Very Liberal respondents (5% of the sample) gave majority support to the loss-framed version. Even car-free households wouldn't sign up for "reduce driving". The authors' line:

"Framing effects could spell the difference between defeat and victory in a referendum."

Social desirability bias

Bernard "doesn't want to look a fool"—Sir Humphrey gives the mechanism away in the setup.

Respondents don't answer in a vacuum. They imagine an audience—the interviewer, or an implicit one if the survey is online—and shade their answers towards what they think is the right thing to say. The pull is strongest on questions about perceived duty or virtue: do you exercise, do you recycle, did you vote (did you?). The gap between what people say they do and what they actually do, on items like these, can hit double digits.

The mitigations from Bradburn, Sudman & Wansink: indirect framing ("some people say X, others say Y, what's closest to your view?"), forgiving wording ("How often, if ever, …"), and self-completion modes over interviewer-administered. The web mode gets the discount for free. Telephone and assisted modes lose it.

Loaded vocabulary

"Reintroducing" implies a return to something good we lost. "Forced to take arms against their will" implies coercion.

Words don't just refer to things—they carry baggage. "Reintroducing" puts National Service in the same mental category as fluoride and seatbelt laws (good things we brought back). "Forced" puts it in the same category as conscription and detention.

Respondents process the connotation pre-attentively. They can't easily filter it out by an act of will, even when they notice it. The answer they give is to the question plus the loaded word, not the question alone.

The single evaluative word does the work—and the Schuman & Presser experiments above are the canonical proof.

These are the four likeliest culprits hiding in your own surveys. The good news: all fixable at draft stage. The bad news: most teams don't audit for them, because most surveys ship before anyone's looked at them with a critical eye.

Spring-clean your survey questions: six rules

1. Replace agree/disagree with construct-specific LikertsThe acquiescence fix above. "How reliable is the train service in your area? Very unreliable to very reliable" beats "I agree the train service is reliable" every time.
2. Force trade-offs Respondents will rate every feature 5/5 if you let them. MaxDiff (pick best and worst from short sets, repeated across overlapping subsets) and conjoint (pick between bundles of attributes) expose the actual hierarchy by making respondents give something up.
3. Anchor on specific incidentsFlanagan's Critical Incident Technique. "Tell me about a time when our service let you down" surfaces the operational detail that aggregate ratings hide.
4. Pilot for what would change the resultBefore you ship, draft two valenced (positively-and-negatively-loaded) versions of any decision-relevant question. If a hostile reviewer could rewrite it with opposite framing and get a different distribution, it's leading. Rewrite.
5. Look for the Silent Quitters Survey response rates skew towards the engaged-and-positive. If the people who'd be most useful to hear from are the ones who skip the survey, the survey isn't measuring what you think it's measuring. Look at non-response patterns and behavioural signals alongside the responses you do get.
6. Ask whether the survey should fire at all Survey fatigue is real, measurable, and rising. The ping that arrives when the customer can't engage with it is a tax on the relationship, not a measurement of it. Fire fewer, smarter, at moments where the answer will actually be useful. And for God's sake make sure you're mining the crap out of your unsolicited data.

This is the kind of thing we Wordnerds live for.

Big Steve and, well, normal-sized Ruth work closely with the behavioural science team at Durham University.

We could happily bore you to tears with more of this stuff if it's useful. Do let us know. Hit reply with what you'd want to read more on, where you think we got this wrong, what you'd add to the spring-clean list.

And—while we're being conspicuously generous with our time—if you've got a survey that's been bothering you, ping it through. We'll happily take a look. No charge for you, Reader. We're dead canny like that, you know?

Until next time, keep learning.

Pete

---

P.S. This week in the Wordnerds office, Stella has been uncharacteristically grumpy at the prospect of turning 24. Her thesis: Taylor Swift was 24 when she wrote "Welcome to New York". Stella has not written a global hit. She did, though, make a blancmange at the weekend, and marvelled at the spelling. Did you know they used to be white? And savoury? No, I didn't care either.

As someone rapidly closing in on the Meldrew Point, I could not be less sympathetic. Screw you and your impossible youth, Stella. And Happy Birthday!