What 26,119 pieces of customer feedback couldn't tell us: four days with Explain at the Northumbrian Water Innovation Festival

Analysing customer feedback is what we do every day. Utilities, housing associations, transport operators—organisations sitting on years of complaints, surveys and contact records, wanting to know what all of it actually says. So when Explain invited us to work alongside their design sprint at the Northumbrian Water Innovation Festival, the interesting question for us wasn't what the data would show, it was what it wouldn't.

Every July, the Northumbrian Water Innovation Festival brings organisations, customers and innovators together to work on the utilities sector's harder problems. This year Explain led Fans for Life 2, one of the festival's flagship design sprints: four days of deliberative research with 33 customers, shaping Northumbrian Water's customer experience strategy for the decade ahead. The group generated 45 ideas and refined them into five recommendations, presented directly to Northumbrian Water's Executive Team on the final day.

Our job ran overnight. Each evening we analysed 26,119 pieces of Northumbrian Water customer feedback—complaints, satisfaction surveys and root-cause records collected from the previous 12 months by Explain—and by the following morning the room could see which of its ideas were already visible in the experiences of Northumbrian Water's 4.5 million customers, and which were new.

The validation work did what we expected. What we didn't expect was how clearly four days in a room with 33 customers would map the edges of what feedback analysis can reach.

Three things the data couldn't tell us

Fourteen ideas that weren't in the record at all

Of the 45 ideas the group generated, fourteen had little or no trace in 26,119 pieces of feedback. Not weak signal—no signal. Feedback data is a record of what customers have already thought to tell you, in the moment they were asked, and only in the terms the survey allowed. It cannot contain an idea nobody has raised yet. Those fourteen weren't noise to be filtered; they were the map of where to look next.

The people who never appear in the data

According to gov.uk's Digital Inclusion Action Plan, around 23% of the UK population may struggle to interact with online services. Given that the survey and complaint data we analysed was all sourced digitally, that population was entirely missing from our dataset. If that holds, then no amount of analytical sophistication recovers them, because they were never in the dataset to begin with.

Our own behavioural segmentation pointed the same way from the other direction: around eight in ten customers in the feedback we analysed were either under time pressure or worried about getting something wrong. Recruitment that deliberately reaches people who don't self-select into feedback isn't a nice-to-have alongside analytics. It's the only route to those voices and underlines the additional value to Wordnerds of Explain's experience in deliberative research.

The "why" behind a theme

Analytics is good at telling you that a theme is large, who it affects and what it correlates with. It is much weaker at telling you why customers behave the way they do—and in the room, people simply explained themselves. That took a facilitator who knew how to ask.

What Explain brought

Recruiting 33 customers who genuinely reflect a 4.5 million-customer base, holding four days of structured deliberation without steering the outcome, and getting a group from 45 raw ideas to five recommendations they were willing to defend in front of an executive team—that's a specialist craft, and it isn't ours.

Steve Erdal, co-founder and Chief Scientific Officer at Wordnerds, said: "We analyse feedback for a living, so we're professionally aware of its blind spots—but you rarely get to see them this precisely. Fourteen ideas out of 45 with no trace in 26,000 records is a specific, measurable statement about what a feedback programme isn't capturing."

What Explain brought is the other half of that: their recruitment reaches people who never fill in a survey, and their facilitation gets to why, which is the thing our data can only circle. Watching them work changed how I'll talk to clients about the gaps in their own data.

Steve Erdal, co-founder and Chief Scientific Officer, Wordnerds

Analytics and primary research answer different questions

The useful framing isn't that one validates the other. Existing feedback tells you, at scale and with confidence, what your customers have already said. Primary research tells you what they haven't been asked. A programme running only the first will keep confirming a picture drawn from the people who volunteered to be in it. A programme running only the second gets depth without knowing how far it generalises.

Running both in the same four days, with an overnight turnaround between them, meant Northumbrian Water got recommendations to its Executive Team the same week they were generated—with each one already marked as established pattern, emerging opportunity, or genuinely new territory needing further research.

For anyone running a Voice of Customer programme in utilities or another regulated sector, the practical version is this: audit what your feedback data structurally can't see, then commission primary research specifically against that gap. It's a much cheaper question than "what should we research this year?" and it has an answer.