Who's liable when AI analyses your customer feedback: the software, or you?

Every insight analyst knows the moment. You're in front of the board (or an auditor, or the regulator), presenting a themed read of thousands of customer comments, and someone asks the five words that empty the room of air: “How do you know that?” How did the AI land on those themes? Would it say the same thing tomorrow? And if you're honest, you can't fully reconstruct how it got there. The model produced it. You're the one vouching for it.

Underneath that discomfort sits the question the whole industry is circling and would rather dodge: when an AI analyses your customers and gets something wrong, who carries it: the software, or you?

Here's the answer nobody selling AI is in a hurry to give you: your organisation does. A model can't be held to account in law; only a person or an organisation can. Your AI vendor (the company that sold you the tool) is responsible for whether it works. You're responsible for what you do with what it tells you.

Owning that liability is better than the alternative. Liability you understand is liability you can build a defensible position around, provided the AI you're leaning on can show its working. The one that can't is the one to worry about.

So this is a practical guide: where the liability actually falls, and how to make sure you're never the one left defending a black box. Our CEO, Pete, took on the wider shift in his CX Corner piece, Who is liable when AI gets it wrong? If you want the view from 30,000 feet, start there. This is the version for the person who actually has to answer “how do you know?” on a Thursday. That's our patch: at Wordnerds we build transparent, explainable AI for regulated sectors, so that question has a good answer.

Why the accountability lands on your desk

Part of what's changed is that the human never actually leaves. In September 2025, Gartner predicted that no Fortune 500 company will have fully eliminated human customer service by 2028, and that by 2027 half the organisations planning big AI-driven workforce cuts will abandon those plans. Automation keeps a person in the loop, and a person in the loop is a person accountable for the call.

Usually that person is the analyst: the one whose name is on the recommendation, presenting an AI's read of the feedback to a board that is trusting the analyst in the room, whatever the tool did. “It's clever, trust it” has never once survived a sharp follow-up question.

Which is why the black box is the real problem here. Any tool that hands you a conclusion you can't interrogate, defend or correct puts you in exactly the wrong spot. You were sold a shortcut and handed a liability. The good news is that the line between the two is clear, and staying on the right side of it is entirely doable.

So who's actually liable: the software, or you?

Precision beats “it depends” here. Liability splits three ways.

First, the AI itself carries none of it. Under English law an AI has no legal personality, so it can't be sued, fined, or hauled in front of a regulator. Someone real is always on the hook.

Second, your vendor is responsible for the tool itself: whether it's accurate, and whether it can show how it reached a result. That responsibility is tightening. The EU's revised Product Liability Directive (2024) now treats software and AI systems as “products” under strict liability, with integrators and importers liable alongside manufacturers. (It doesn't bind UK organisations post-Brexit, but it shows where the wind's blowing.)

Third, and this part never transfers: you're responsible for the decisions you take on the analysis, and your duty to your own customers, tenants or passengers. No directive moves that.

And in practice, the contract limits what you can claim back. In an analysis of AI vendor contracts, TermScout found 88% cap the vendor's payout at little more than the fees you've paid, while 92% keep the right to use your data beyond just delivering the service. So if the tool causes a costly mistake, that cost is yours. The AI can't be liable, because it isn't a person. You can. So don't get caught defending a black box.

Won't tighter regulation just move the risk to the vendors?

It's a fair challenge, and the strongest version goes like this: product-liability law is catching up with AI (that EU directive is proof), so surely the vendor ends up carrying more of the risk over time.

Three reasons it doesn't let you off the hook, and actually sharpens the point.

One: it's EU law. A UK housing association, transport operator or bank isn't covered. Here, product liability still runs on the Consumer Protection Act 1987 and ordinary negligence, and the AI-specific regime is still being worked out.

Two: even under the tighter regime, 88% of vendors cap their exposure. Strict liability on paper is cold comfort when the contractual ceiling is a couple of months' fees.

Three, and most important: product-defect liability is a different animal from your accountability. A regulator asking a landlord to evidence the tenant voice, or an FD asking why you cut a service line, does not care whose model produced the theme. The decision was yours. The pressure is rising on both sides at once, and the half that's yours never moves.

Why accuracy alone won't protect you

So if the liability is yours, what actually protects you? Most vendors point straight at accuracy. That's the wrong place to start.

Let's be honest about accuracy, because the industry usually isn't. AI is never 100% right. Neither is a human coder who's read four hundred comments before lunch. Get to 85% to 95% accuracy and you have more than enough to see how sentiment is shifting and decide what's worth acting on. Perfection was never on the table, for people or machines. The real question is whether your accuracy tolerance is written down and defensible.

Because a right answer you can't explain still can't be defended in the room. And the room is getting stricter. The FCA's 2024 review of Consumer Duty outcomes-monitoring told firms to move past raw dashboards to analysis with a clear causal chain and audit trail, and called out those who “repackage existing data” to look compliant. In social housing, the Regulator of Social Housing's Transparency, Influence and Accountability standard (April 2024) expects landlords to evidence that they understand tenants and act on what they say. Both regulators are asking the same thing in different accents: show your working. Even a number that comes out the same every time still needs an explanation a non-technical board can follow, and that's a separate job from getting the number right.

What a defensible position actually looks like

It starts with a tool that can show its working. And yes, we would say that, wouldn't we, so hold any vendor, us included, to it.

The engine does the heavy lifting, with sector-tuned models detecting and classifying themes automatically. What makes it defensible is the quality-control layer on top: every classification model ships with a plain-language description of the criteria it uses to decide whether a comment belongs in a theme. That's a surface a regulator can interrogate in plain language: who defined the theme, what the definition was, and why a given comment landed in it.

Consistency matters just as much. We use large language models to build and train those models, but your customer data never touches an LLM at classification time. Conventional machine learning applies the definition, so the same input gives the same answer every time. An LLM might not, and in a board paper or a regulatory return, that inconsistency is a serious problem.

Put together, that's a defensible position: a named owner, a documented accuracy tolerance, a model that's actively monitored (we've tested ours for bias since 2020), and an AI whose reasoning you can produce on demand. And we do it at regulated scale: our joint benchmark with Housemark read 135,000+ anonymised tenant comments from 18 housing associations against a single auditable frame, the UK's first qualitative TSM benchmark.

The real test: run it twice

Want to know whether your AI can show its working? Run it twice.

Take a batch of your feedback, paste it into ChatGPT or Copilot, and ask for the top themes. Then run it again. Then get a colleague to run it. You'll get meaningfully different answers each time: a slightly different set of themes, a slightly different story. That's fine for a discovery snapshot on a Tuesday afternoon. It's useless as the basis for a board decision you have to reproduce next quarter, or defend to an inspector in eighteen months.

Now ask the same of a specialist tool: same input, same classification, definition visible. If your AI can't pass the run-it-twice test, it can't show its working, and neither can you.

Get this right, and “how do you know?” stops being a threat

Picture your next TSM submission, board pack or Consumer Duty report. Someone asks the question they always ask: “how do you know?” This time you're not reaching for faith in a model you can't see inside. You point to the theme, the definition, the criteria, the audit trail, and the question becomes a chance to show your work.

And it isn't only about defence. Because the classification is consistent, you can see what's shifting in near-real time and act before it becomes a complaint trend or a headline. One housing provider we work with, bpha, cut repeat complaints by 32% once it could act on what its feedback was telling it. The same foundation that keeps you defensible also makes you faster, the payoff regulated teams keep missing when they treat governance as a tax rather than an edge.

The organisations that get ahead are the ones who faced the “who's liable?” question honestly, instead of hoping it wouldn't come up. Be one of those.

Book a diagnostic, and see the reasoning behind every theme on your own feedback. And if this is your kind of question, CX Corner is worth your inbox: our CEO Pete's fortnightly newsletter on turning customer feedback into decisions that stick, smart and honest and lightly sweary. One recent issue takes this very question on from the top down: Who is liable when AI gets it wrong?