How can you use AI to analyse online reviews?

Why is Morrisons' Trustpilot rating rising faster than Waitrose and Aldi?

Across the three supermarkets, Morrisons' star rating and sentiment on Trustpilot rose noticeably over the past year, and the standout difference is engagement. Morrisons replies to 96% of its negative reviews, typically within 24 hours. Waitrose replies to 62% of negative reviews, typically within a week. Aldi does not reply to its reviews at all. When a brand responds quickly, customers feel listened to, and that appears to track with higher ratings over time. Engagement is not the only factor, but in this Trustpilot data it is the clearest behavioural difference between the brand whose sentiment is climbing and the two whose sentiment is flat or softening. For any brand, the practical lesson is that responding to negative reviews, fast and visibly, is part of managing the rating itself, not just customer service after the fact.

What does review data reveal about customer loyalty in supermarkets?

Loyalty shows up in review data when customers self-identify, saying they have shopped somewhere for years or calling themselves a loyal customer. In the Trustpilot data, 15% of Morrisons reviewers talk about being a loyal customer, against roughly 7 to 8% for Aldi and Waitrose. Crossing loyalty against other themes shows what actually drives it: positive staff interaction, the in-store environment, store vibes, promotions and offers, ease of navigating the store, and value for money. These are the same themes that rank highest by volume across the data set, so when a brand does them well, loyalty and sentiment tend to follow. For Morrisons, strength on exactly these drivers helps explain why its sentiment is rising. The method matters as much as the result: loyalty is not guessed at, it is read directly from what customers volunteer in their reviews and benchmarked against competitors.

Why do Waitrose customers say they're leaving?

Waitrose has the highest proportion of reviewers talking about churn risk, customers saying they will never shop there again, and the reasons are revealing. Crossing churn against other themes shows that 46% of Waitrose reviewers who talk about leaving also talk about brand perception, the brand not being what it used to be. A further 13% of those talking about leaving mention expectations not being met. In other words, the issue is less about a single broken experience and more about a gap between the brand people thought they were buying into and what they now feel they get. That is a positioning problem as much as an operational one. Read this way, review data does not just flag that customers are unhappy; it tells you the story they are telling themselves about why, which is exactly the context a satisfaction score or complaints log leaves out.

How is AI changing e-commerce and online reviews?

Ross Hancock describes two shifts already underway. First, interactive advertising: instead of clicking an ad through to a store, you interact with the ad directly, by text or voice, and complete the purchase without leaving the platform. Second, agents: tools that find and evaluate products for you. Perplexity has launched a shopping agent in the US, and OpenAI's operator product is expected in the UK, able to take an instruction like "book the best hotel on TripAdvisor" and act on it. Crucially, Trustpilot's view is that the human stays in control of the actual purchase decision. That makes the experience signals in review data more important, not less, because those signals increasingly determine what an agent surfaces and what a human approves. As reviews get consumed out of context, by bots and interfaces rather than on your site, managing online reputation becomes critical.

What is the difference between Trustpilot and a review platform like Feefo?

Ross Hancock frames Trustpilot's core differentiator as openness: anyone can leave a review, rather than reviews being limited to verified, company-invited customers. Knowing more about a reviewer adds weight when you analyse the data, but the platform stays open by design. The contrast is with private feedback tools, where you ask selected customers directly: something like Medallia. Those approaches carry selection bias, both in who responds and in what they choose to say, so the picture is less representative of the real customer experience. Trustpilot's argument is that an open platform produces more honest, more representative feedback, which is exactly what you need if the goal is to understand and improve the experience. For analysis, that openness is the point: it widens the pool beyond the customers a brand chose to ask, surfacing experiences a private survey would never capture.

Can you link review insights to a financial measure like revenue?

Yes. Pete explains that Wordnerds spends much of its time building predictive models from feedback data, for example identifying customers likely to churn. Because the analysis lands in Power BI, it can be combined with quantitative data to see what drives a metric up or down, used as a lagging indicator to understand historically what moved a number, and then built into predictive models. Random forest analysis is a current favourite. Out of the box, these models often start at around 80 to 85% accuracy and improve as more data is added: the more data you feed them, the better they get. So review and feedback data is not just a qualitative nice-to-have; it can be tied directly to commercial measures like revenue or value at risk, segmented by whatever numeric data a brand already holds. That is where understanding the why behind a score becomes a forecasting tool, not just a diagnosis.

Full Webinar Transcript

Pete: Thank you so much for joining us. We've got a load of people joining us today. We don't take that for granted. We are always

Staggered and delighted at how many of you join us on Wordnerds webinars. We know you've got lots of calls on your time. So we very much appreciate you making some time to spend it with us today for unlocking the power of online reviews. So thank you very much for that. And it is my joy and pleasure to welcome friend of Wordnerds Ross Hancock to the Wordnerds webinars.

Ross is one of the principal product managers at Trustpilot. His specific area of expertise currently is their API. And in that capacity, he is super well placed to talk about how Trustpilot works with other organizations, how it fits into the ecosystem of customer experience. And we've had we've been dealing with Ross for a few months now, and we're really excited to welcome him today. We've got an inkling of what he's going to talk about.

And it's some it's some next level stuff and we're always excited by that. So thank you, Ross. Appreciate your time. We also have on the cast Stella. Stella is part of the graduate program at Wordnerds, she's one of our analysts. She's loaded with cold and feels utterly miserable. So she's putting on a brave face. If she looks a little bit less bouncy than usual, that's what it is. Stella

Comes here to do live data science demos because I'm yes, I know we're spoiling you. Nothing says Thursday lunchtime like a live data science demo. So more on that in a moment. If you don't know Wordnerds, Wordnerds are a customer feedback analytics platform. We have the pleasure and privilege to work with some of the UK and world's most data focused and customer-obsessed brands.

We work in four main areas in social housing. We work with a bunch of social housing organizations, largely because of the regulation there, forcing social housing organizations to do more with resident feedback. We work with people like Lloyds Bank and True Potential in Financial Services, in retail. We've got clients like the wonderful Sainsbury's, Marks & Spencer, B&Q, bunch of kind of high street brands that you'll be aware of.

And travel and hospitality, people like Dorchester Hotel, Hotelplan, some great brands like that as well. So welcome to those of you who are on the cast today from some of those brands. Always good to see existing customers here. And if you're new to Wordnerds, and I know there's a lot of you, thank you for joining us as well. We hope you enjoy it. Why are we here? Well, online reviews are an amazing resource.

They are a literal gold, well, not a literal gold mine, that would be the word. They're a figurative gold mine. I've been hanging around with my 17-year-old daughter too much. They're a figurative gold mine of lots of amazing data that you can use to your advantage in a bunch of different ways. Obviously, we at Wordnerds deal in the customer experience space, so we look at it through a customer experience lens, but I think as Ross is about to

Elucidate later on. There are other things that this information is useful for as well. But most companies, most brands that we talk to fail to utilize them to their max. And the point of today's webinar is to explore how can you how can you do more with online reviews, what can you do, who's currently doing it, what are your first steps, just to kind of kick this idea around. And we're going to do a bit of a now, next and future.

Of online reviews. We're going to look at, we're going start. Ross is going to take us through a little bit about Trustpilot. I'm sure everybody on the call today knows about Trustpilot. But he's going to give us a bit about the story so far. He's going to talk about the importance of content integrity at Trustpilot. But the really exciting stuff that I'm here for is how this amazing AI revolution is changing e-commerce as an industry.

And the implications for that on reviews and some of the opportunities that creates over the next few weeks, months, and years for him and his teams at at Trustpilot. So very much looking forward to that. And also how it places the importance of understanding your customers' experience at the heart of decision making in your organization. After that, we're going to talk a little bit about.

How brands are using online reviews right now. So Ross is going to do the future stuff, we're going to do the now stuff. You may have noticed that in the ads for this, we were meant to be joined by Ruth, who was going to do this piece. Unfortunately, Ruth can't join us right now. I am not Ruth. I am a poor facsimile of Ruth, and I will endeavor to deliver her slides in anywhere near the quality that she would do it. I will almost certainly fail. So for apologies, I beg forgiveness and forbearance, but bear with me through it.

I'm going to talk you through benchmarking, using reviews for positioning, how to work out what actions you can take from those reviews to improve your customer experience. And we've got a very short three-step process that I will outline so that we can get to the main bit that we're here for, which is Stella getting her hands on some actual data. We are Wordnerds, our idea of a good night in is a big stack of data and a bottle of wine. So

Stella getting a hands on some data. We've chosen for the purposes of the of the demonstration, real life retail example. Everybody shops at supermarkets, it's a point of reference we all have. So we've picked three supermarkets from different ends of the spectrum: Waitrose, Morrisons, and Aldi. And Stella's going to look at, we've downloaded some review data from Trustpilot. Thank you for that, Ross. And we're gonna look at

What that data tells us about how their customers experience those brands differently. If you were working for those organizations, what could you learn about your service delivery and product delivery from that data? What might you do next, et cetera, et cetera? So that's the plan for today. I hope that you guys are happy with that. We will whistle through it as fast as we can. There will, after that, be a very quick plug for Wordnerds. No such thing as a free lunch.

And we haven't even provided lunch, isn't that rude? It will be very quick and painless, I promise you. And then we'll go straight into QA. Please drop your questions in the chat at any stage. We will try and pick them up as we're going through, and we'll have a more formal QA at the end if we don't get to those things. Behind the scenes, we have Alex and Vic who are making sure that all the levers are being pulled and the whole Wizard of Oz thing is going on, so they will.

Pop up in the chat to help us with that. So thank you to those guys. Starting with before we get started, we like to have a little bit of audience interaction. We like to get a view on where you guys are. So I'm hoping that there should be a poll popping up in your on the right hand side of your screen now. If you click on polls, if you've already got it set to chat, you shall see it. So we're really interested in how you are currently analyzing your online review data. So if you have a look at the poll on the right hand side and

Click one of those answers. We will revisit at a later date or Vic will publish once everybody's kind of answered the questions where people are on that. It looks like we've got a good mix between people who are actively managing review data and people who want to dedicate more time to do this, which is a good start. So you're all aware of it and you're using it to different degrees, which is great. So it feels like the right audience for us. Thank you for that.

Ross, without further ado, over to you, mate.

Ross Hancock: Awesome. Thank you so much, Pete. Really appreciate it. Yes, Trustpilot is, as I'm sure, actually, given the poll that most of you know, Trustpilot is the largest independent platform for customer feedback in the world. We have over 300 million reviews, and that is growing at an incredible rate, at least 60 million reviews per year. And they said that's increasing at quite a rate.

We have reviews from all industries across all geographies from Uzbekistan to the USA. But what's interesting is actually where Trustpilot started. And it actually started with a washing machine, which might seem a bit weird. Back in 2007, our founder Peter, he went back to his family home. He was sat on the sofa and his mom comes up to him and says,

The washing machine's just broken. We need to buy a new one. I found this one online. Should I buy it? And he has a look and he goes, I don't recognize that business at all. I'm really not sure if you could trust buying it or not. And so it struck a chord with him and he realized, actually, I can probably solve this problem. If I can allow real users to provide feedback on businesses, then I can have confidence in buying online.

And I can help people like my mom. And so as a result, one of the byproducts of that, which is really hugely beneficial, is it allows businesses who are trustworthy and authentic to then be able to find the consumers for them. But underpinning all of that, for that to work, the data has to be trustworthy. It has to come from real people. And it has to reflect real experiences.

And at Trustpilot we invest heavily in ensuring that is the case through a large volume of humans that will moderate content, but also through a vast number of sophisticated technologies to detect and ensure that the content is trustworthy. And we look at a variety of different signals in order to ensure that from doing consumer verification, which is based on IDs to make sure that

A Trustpilot account is operated uniquely by a human, through to all sorts of signaling from location, but also through from patterns in time stamp, time stamping, cetera, through a variety of different things, some of which is proprietary and I can't share. But that ensures that the content on the site is as trustworthy as is possible. And I'm going say that because it's all of this stuff is

Is really important because we're seeing kind a major shift now in e-commerce, a major paradigm shift with the introduction of the AI. And if we think about kind of the history of this, when e-commerce came along and the internet was introduced, it meant that you could start buying, you no longer bought from the local shop or the local town, you're then buying from businesses that are on the other side of the world. And then when mobile came along,

It meant that you could buy anywhere, right? You could buy whether you walk into the shops, you can buy on the go, et cetera. The big change we believe now is coming in and we're seeing this in real time is with the introduction of AI. And at the very least, we're seeing that in two different ways. The first is the introduction of interactive advertising. If we think about the classic user journey in the e-commerce space,

You'd see a static adverts on a social site, for example. You'd click through. You then try and find the product in the shop. You'd select a variety of different, the size of the product, et cetera. Add it to your cart. Enter in your address details. Enter in your payment details and finally you make that purchase. But what we're seeing now is being developed and I'm sure will be released within months is

The ability to interact with adverts directly so you don't have to leave the platform which you're viewing that advert. So for example, you may see an advert on Instagram with, let's say, selling t-shirts. You could directly interact with that advert, either through text or through voice, so multimodal approach, saying, actually, is that t-shirt available in red? Can it be shipped to my location? Sure, let's go ahead and buy it. And that purchase would take place on that social.

On that social online store. The second thing that we're seeing is the introduction of agents. And you might have heard the term agents kind of flung around. The agents effectively can have an automated tool that allows you to, it can automate parts of your workflow. And in the application of the purchasing journey, what we see that as happening is an agent being able to find the products in which you want.

Without you having to search and do that yourself. And so one of the things that we have firmly in our belief, however, is that this isn't like Terminator. We're not just going to give control over to automated agents. We still believe that the human is still going to be firmly in control. And so if you mind moving to the next slide.

So as you imagine that customer journey where you may explore product to evaluate whether product is appropriate or not, we still believe that purchasing decision will be made by a human. It would be the point in which a human approves a purchase for an agent to make. And so what that means is it's still utterly critical and maybe even more critical because it's hard to get these signals to understand what the human's experience is because the human's experience determines whether they make the purchase or not.

And so that's why it's really, really, really valuable that you have data that actually reflects the true human experience so you can understand it and improve the experience and therefore improve the likelihood that the consumer will purchase.

Does that give you a good overview where we're kind of seeing the direction?

Pete: It really does. And I love it. I love the idea that you can all of a sudden buy directly from ads without having to go into people's websites. I think we're seeing with Gen AI the not that necessarily the death of search engine optimization, but the fact that you put a search in and the results just come to the top of the search engine without you actually having to go to people's websites, obviously you can click on them and go in and it's changing the way that we

That we basically consume information. Is something similar going to happen with e-com, do you think? Do you think do you think as, you know, as the people that provide the social proof, you're going to completely change the way that you have to deliver the results of the reviews because people are seeing these things out of context of your site or the e-commerce site?

Ross Hancock: Yeah, absolutely. Spot on. And to give a concrete example, Perplexity have released a shopping agent. I think it's actually only available in the States for a minute. But Perplexity for folks that don't know is a kind of similar type of a chatbot interface as per ChatGPT or Gemini, et cetera. But in the shopping agent application that they have, you can say,

You know, it's my daughter's birthday coming up. I need to find a present for her. She's kind of four years old, et cetera. And actually, if you've already interacted with the agent previously, it knows that information, right? So it can already hone down and personalize the content of what you see. So you say, no, birthday party coming up, et cetera. I can hit OK, right? And it's going to return me.

Different products that it thinks is appropriate for me to purchase based on consumer reviews, but also, as I said, the personalization aspect to it. And what it knows is based on, again, the interactions it's had with other people, what is an appropriate product for four-year-old and what is popular. But not only then automating that exploration and kind of evaluation stage, but also including the purchasing within that minute.

That within the interface. So you can simply click buy with Perplexity and then you can even select the attributes of the product as well. So we're seeing that being brought into different interfaces, into different parts of the customer journey, as opposed to that traditional search engine to online store through to a purchase. And so as you said, what's important is how can we bring in that social proofing

Into those applications to make sure that consumers are actually getting, they actually buying from businesses that are trustworthy. And so we do that through a variety of different ways, but one of which is then provide kind of Trustpilot data through an API so that those businesses, so that those platforms can consume that content.

Pete: Interesting. And I guess particularly if you're seeing it out of context, if you haven't got, you know, there's there's probably hundreds of decisions we make about a business and how trustworthy it is when we go on their website. If if you are divorced from that process, that adds more weight to the results of the review data that you're seeing out of context. So what's your advice for for brands about

Managing their their review data and putting their best foot forward so that when they are seen out of context particularly they can you know they can encourage a positive response.

Ross Hancock: Yeah, I think it's going to become as critical as ever to manage your online reputation. As these bots take control of that expression and evaluation, they are going to rely on review platforms and other types of signals as well. And there's no option that those bots will do that.

So managing your online reputation is just so critical. And obviously the way to improve that online reputation is to improve your business. And to improve your business, it's all about improving the customer experience, right? And the only way you can do that is truly understanding what the current state of that experience is.

Pete: That's really it's just fascinating I love it. I love this idea that there's gonna be a little bot following me around that knows more about me than and my purchasing journey than I do and will recommend me things, all these kind of dull middle-aged things that I buy these days. It will just put those things in my lap and I can consume these things. That feels I hate shopping. That feels like

Ross Hancock: I mean, what I find is fascinating is this isn't like, know, far out into the future. This is happening now. It is being piloted in different geographies and there's great consumer adoption. You know, as I mentioned, Perplexity, but other big vendors are doing this now. It is happening and it is being put into the hands of real consumers.

Pete: It's amazing how rapidly this AI revolution is changing every facet of life. It's unbelievable. We've been we've we did a hackathon at Wordnerds a couple of weeks ago to try and just automate processes that were annoying us. And the progress we made in one day on all kinds of things from illustrations to you know dull infosec processes was was just unbelievable. And so to see it affecting e-com this way is incredible. Ross, thank you. We're gonna have lots of opportunity for people to ask you questions at the end.

I think the it's a beautiful segue as well, by the way, talking about the importance of really understanding your customer. I we completely agree. And I think that's where we wanted to pick up the reins from you. Because certainly that a lot of the brands that we're talking about, they some of them proactively manage their their reputation. They do the kind of responding to reviews, but remarkably few of them

Do the analyzing reviews and understanding the content of them. So that's what we want to talk to you about today, picking up from Ross. And we believe that review data is a, as I've said before, figurative gold mine of all kinds of very useful information. But there is a challenge involved. A lot of the information can be old, there's far too much that people talk about.

All of the usual problems with text analytics exist in spade loads with review data. And the challenge for most people is where's the gold? Where's the stuff that's actually actionable? What is actionable depends on the question you bring to this. And people often bring different questions from generally what are our customers telling us that we didn't know, you know, what are the unknown unknowns here.

What do they think about something very specific? You know, if you've got a if you're a holiday company and you have a particular location, if you're a supermarket and you've got some new vegan sausages, what you know, is that are they good, are they bad, are they being well received? You sometimes bring hypotheses and questions to it. From a positioning perspective, we get a lot of people talking about whether or not they own a particular part of the customer mindset. We did some work for a high street retailer a few weeks ago.

They have always been the brand in their space that owned value for money. And because of, you know, a couple of years of double-digit inflation, they've had to put their prices up and they were very nervous that this might affect their reputation as being the value brand. And by looking at review data and comparing them to what their competitors' customers have been saying over the past couple of years, we've been able to plot on a graph exactly how that has shifted.

Over the last 24 months, which was really interesting. One of the things that we find is there is no doubt your survey data, your your complaints data, the information that you have directly from your customers within your business is the highest quality data that you have available to you. And that will definitely tell you when you have a problem. What it won't tell you is the context of that problem. We do a lot of work for train operating companies.

Every train operating company in the country has people complaining about late trains. If you look at a talk data set, it will say late trains as one of the big issues. The question isn't are people complaining about trains being late? The question is actually are you outperforming the market or underperforming the market? Are there fewer late trains on your network than everywhere else? Just because there's lots of people talking about it.

Doesn't mean that it's necessarily going in the wrong direction. It might be improved massively over where it was. And one of the joys of review data is it lets you understand the context of that issue. Is it an issue just for you, or is it a wider issue for the rest of the industry that you're actually overperforming it? One of the big things that Insights and Data teams are spending more and more time looking at as technology gets better is.

Is action. I think a couple of years ago the thing we heard most was, we're data rich and insight poor. I think now everybody has insights up the yin yang. We've got more insights than we know what to do with. And actually, which of those insights do we need to act on becomes a more and more pertinent question. And how do you work out how you act? What can we do to make our customers happier? And then the other thing that people often ask us is, we've there's a new brand on the scene, they're seen as a really good sort of brand for doing this thing.

How do we understand what they're doing and apply that to our customers? Which I think is a really wise question to ask. And depending on what question you're bringing, the data that you've got, the you will get a different answer and you'll need to do different things. But regardless of what you do, there is a very simple three-step process that allows you to do more with review data and it involves tools, process, and methodology. Very quickly.

When it comes to text analytics, there is a definite maturity model that we see. Everybody has started this journey doing manual coding. A few years ago, before this AI revolution, text analysis wasn't very good. Sentiment analysis was limited, and most people were reading this stuff, reading the verbatim, often five-bar gating it and doing this stuff by hand.

You will be shocked and surprised in 2025 how many people are still manually coding most of their customer verbatim. It's something that we see all the time. And if that is you, don't worry. Everybody is on a journey with these things. You are not alone. Like I say, more people that we meet now are still doing this than aren't. Once people have been through that manual coding phase, the next thing they tend to try is right, let's get one piece of software that will do all of this for us. Often that's a Qualtrics or a Medallia, it could be a

A stack like you know the Microsoft stack that has various tools that help you with this stuff. It's a perfectly rational response to this, and it will solve some of your problems. Qualtrics and Medallia are brilliant tools for generating customer feedback. They're great for automating processes and bringing in lots of different customer data into one place. Actually, the text analytics that they provide is quite what they were designed for, and it's quite limited.

We encourage everybody to have tools like that in their stack, but we don't encourage people necessarily to use that for certainly things like review data and for some of the wider customer data sources, call center data, things that your staff write about your customers, all of that needs to be taken into account as well. Wherever there are words in your organization from or about your customers, you need to be able to process that. So once people have tried that, they tend to then come up against a bit of a bit of a brick wall.

The next thing everybody tends to try is Gen AI, often Copilot, because that's the one that is bundled in with the Microsoft stack that most large brands are on. Can be ChatGPT as well. That stuff is getting better all the time. We won't belabor the point. We've done a couple of weeks ago, we did a completely separate webinar on can I use Copilot, can I use Gen AI to analyze my customer feedback data?

We will send out after this a link to it. It's really good, similar kind of setup to today. We go through some real life examples. Stella got her hands dirty with the data and compared Copilot to other methodologies of doing it. And we talked about why that technology is set up to do some things really well, but analyzing numerically and factually your customer data is not one of those things that it can do. That might change, I'm sure it will in the next few years. It was a really fun webinar and

If you are at a loose end or can't sleep, we encourage you to have a look at that after this. But Gen AI is not yet the answer. And shock horror, we are a specialist text analytics tool. Our conclusion is you should use a specialist text analytics tool. And we'll show you hopefully in the course of the next 10-15 minutes why that is. And when we say that, the things that we think are important about using a specialist tool are firstly.

A federated system of software providers that do very specific things very well. So, whatever tool you use, it needs to be integrated both in and out. It needs to sit under all of your frontline tools. Different parts of your organization already has software that they are happy with that manage complaints, that manage their social media, that manage their call centers. You know, this tool has to be able to pull in all of that information from that, but also out the other end as well.

Nobody wants another software platform to look at reviews on. And increasingly, the more we're doing this, the more we're reporting some of this stuff in Power BI and other BI tools, because that is where people in people's organizations already are with their quant data. And we find that quant data is great at telling you when there's a problem, but to find out why there's a problem and how to fix it, you require the qual data, you require the customer verbatim and reuniting all of that in your Power BI dashboards.

Is a really important step. And Power BI doesn't play nicely with text data. So that is a challenge. It needs to be configurable. You need to be able to listen for anything that you want at short notice. Gone are the days when software would just give you 10, 20, 30 things and say this is a delivery issue, this is a quality issue that is not helpful to anybody. You need to be able to get right into the minutiae of the data and find very specific problems.

And it needs to be accurate and transparent. One of the issues with Gen AI is Gen AI will tell you that something is a particular problem, but it doesn't tell you how it arrived at that decision. And often we get into very nuanced conversations about what makes something fit into a category and what doesn't. And it can be really sort of weird things. We did some stuff at the B&Q a while ago where they were interested in finding out, you know, whether or not people could find the toilets in store. As a company that sells toilets, the difference between

I couldn't find the toilet and I couldn't find the toilets, plural, is a really subtle but really important thing because they both have completely different meanings. And in that situation, they need to be able to train their AI in a totally different way to the way that most people kind of care about. So it's got to be accurate and transparent. No where it comes to words, humans are pretty weird. No technology is 100% accurate. And what you really need is technology that will tell you how inaccurate it is.

So you can decide the veracity of it and whether or not to trust it when you're making decisions based on the stuff that's coming out. The second step part of this is a three-step process for analyzing reviews. Obviously training AI to listen for stuff is great, but it's not really listening. It misses some valuable parts of it. The first thing it misses is the stuff that you didn't know to listen for. AI isn't great for that. It's

We call it zero shot topic analysis, telling you just what's in the data set. We use a lot of things like corpus linguistics for this. So what are the surprises? What are the unknown unknowns? What's in the your data that you maybe haven't seen before? What's new in this month that's growing? Most people most of the time know their big problems. What's the small stuff that's growing that is going to be a problem in six months' time that you've never come up against before? So how do you explore the data? How do you see the stuff you didn't know? There's then the classification piece, which we've spoken about.

You need to be able to set up a framework and more on how you set up a framework in a minute to put this stuff into buckets so you can see the size and the sentiment of those buckets. And then increasingly the game is about how do you prioritize that data at the end of it? How do you, you know, if you're a large organization, if you're a massive supermarket, the number of different things that your customers can tell you from problems in the car park to issues with specific products that you have is myriad.

And how do you get all of that information and work out what's the stuff I need to act on first? And again, it depends what question you're asking. And I think the answer to the prioritization question is around methodology. You need to be able to apply a bunch of different methodologies to this data. And by bringing in tools like Power BI after the classification process, it opens up this data set to being able to do so many different things.

We were talking this morning about project we're doing with the Housing Association, looking at customer feedback really early on in a tenancy agreement to be able to build a predictive model to see whether or not it's going to be a success, whether or not those people are going stay in that house. That's a methodology that is completely different to things like can I work out what is the thing that's driving my NPS score down? If I'm gonna change five things this year in my customer experience, do I know that doing that is gonna make my NPS score go up?

All of these things are possible. You just need to decide which methodology you care about depending on the question that you ask. And then you need to set up the process to deal with that methodology. The one that Stella's going to show you today, or one of the ones that Stella's going to you today, is a methodology we've been working on with Professor Alamanos from Newcastle University. He's a professor of marketing, he's done a lot of work on omnichannel customer journey.

Journeys in e-commerce so how how people buy online particularly complex journeys where there's a in-store presence and an online presence and there's two areas that he's interested in firstly what are the steps that people need to go through to be happy before they buy and you can see some of those on the left hand side he calls them experience values and each of these things breaks down into a lot of other smaller things and we can listen for signals for all of these things and measure them in real time.

On review data, looking at your customers and your competitors' customers. But then also he's interested in the customer journey. And increasingly this is a nonlinear customer journey. And I think some of the stuff that Ross spoke about about agentic AI and the way that you will be getting in-moment experiences out of an e-commerce context makes this nonlinear process even more nonlinear. And I'm really interested in how those two things work together.

But it's a process that starts with the kind of the planning and the thinking about stuff or being told about stuff. It goes through the whole kind of actual purchase journey. But then there's also a big post purchase part of this. So when people are using things, what are they then doing? Are they reviewing the product? Are they kind of experiencing issues and talking to the brands and kind of following up on that stuff? And he's gone into a lot of detail about this, and we've basically taken that methodology and applied it to the data.

So that's the theory. I promise you, Ruth does that so much more succinctly and better than I do. So I apologize for my delivery of it. It's all very up in the clouds though. And I think to give you a good example of actually what does that look like in real terms, I'm gonna bring Stella in at this stage to take our same, sorry, our Waitrose, Aldi and Morrisons example and show you what she's found in their data, which I'm excited about.

Stella: Yes, hello. So my plan is to take you all through a real life example that we've got from the Trustpilot reviews of the three supermarkets. So to begin, this is just the overview page on Power BI, which we've taken the reviews onto. And here you can see the difference in the number of reviews, the sentiment of reviews, just a very broad view.

Of all the supermarkets. And you can see at the bottom, the sentiment has been increasing over the past year, which we of course found interesting. And we thought, why is this? So we looked at the individual star ratings of each company. And you can see that Morrisons over the past year is a big reason for that increase in star rating and sentiment. And here we can just compare the overall star ratings.

And by time. And of course, there can be many different reasons that Morrisons star rating has gone up. If we look, Aldi's has stayed relatively similar, same with Waitrose, if that hasn't gone down. And so yeah, it could be many different reasons. We trained themes to see if any of them stick out, but an a reason we found was the difference in the engagement in the Trustpilot pages.

So Morrisons 96% of the negative reviews are replied to and typically within 24 hours. If we compare this to Waitrose, 62% of their negative reviews are replied to typically within one week. And then finally, if we re compare it to Aldi, that they don't reply to their reviews at all. And we found that interesting, as it might be a reason why.

When engagement goes up, star rating may go up in the fact that customers feel listened to, things like that. It could be a reason for this change. So to look a bit deeper, we can look at what matters to people. So on this right hand side are themes that we as Wordnerds have trained, as Pete mentioned in that customer journey to see what people care about.

And this is ordered by volume and you can see the emotional effort, which is things like they feel anxiety or stress or any emotional feelings towards their shopping or journey. That's the top mentioned thing for customers, followed by praise for the in-store, monetary themes, personal interaction, self-interaction, so the interaction that customers

F fine in the store, negative staff interaction. These are all clearly things that customers care about and that across the data sets that the customers are talking about the most. We can also look based on negative sentiments. So what's driving negativity across the three supermarkets, which is where things like contact center experience, negative staff interaction, delivery collection, communication, where they come in and drive negativity.

Among all data sets, which obviously is interesting, but you want to know how you can benchmark against competitors. So, what is what is one supermarket doing that others aren't, which is where we have this graphic of the customer journey. So here with the categories of the themes that we've trained, we can see the volume and the sentiment at each stage in the customer journey. The size of the circle is the percentage volume.

Of comments in each brand. So the bigger the circle, the more people are talking about it as a whole, just so we can compare more accurately with different volumes across the different supermarkets. And so you can see that Morrisons leads on most categories in this data set. If we start with starting, you can see that store vibes, which of course vibes are very important.

Everyone knows, everyone loves vibes, are very high volume and high sentiment in the data. If we compare this to say Waitrose who have lower volume and lower sentiment, you can start to see that as we found these are things that people care about, you can see why Morrisons may be high sentiment in this particular category.

Next we can look at the shopping, which Morrisons once again. This is probably a more important category because it's where the main things happen. You can see that things like positive staff interaction, the in-store environment again, value for money, these are all drivers of positivity within Morrisons. And again, if we compare it to Aldi first of all, you can see that their negative staff interaction outweighs their positive staff interaction by quite a lot.

Interestingly, their value for money is lower than Morrisons, which I wasn't expecting because obviously they're a cheaper supermarket. But and then also if we look at it for Waitrose, obviously their value for money is even lower, their negative staff interaction is higher than their positive staff interaction, and a lot less people are talking about the in-store environment. You can start to paint a picture on why Morrisons is increasing their sentiment and

What people care about and why people are leaving higher reviews. Finally, if we look at the following up stage, which is the after stage, we can see that a lot of people talk about being a loyal customer, Morrisons. So for this, this is people self-identifying as a loyal customer. So saying I've shopped here for years or actually saying I'm a loyal customer. And you can see that it's high and positive in Morrisons. And if we compare this with Waitrose,

You can see that their loyal customer base is actually quite a lot down, which again is interesting because I think more Waitrose you think of as being something that people are loyal to. So we can see in here that actually it's not as big of a mention as you would think, which might be something to work on. So next we can dive deeper into this loyal customer.

And if we look at the loyal customer as a key driver, on this right hand side are crossover themes. So what are people talking about when people self-identify as being a loyal customer? And this you can see what really drives people's loyalty. And in this it's positive staff interaction, in-store environment again, store vibes again, promotions and offers, navigating the store, value for money. So it's things like this that we can relate back to those top.

Volume themes at the start and you can start to see if people care about those themes and Morrisons are doing them well. That's a likely a case why their sentiment is higher. Diving deeper again, we can look at the cross tables, which is one of my favorite graphics, where you can basically cross. So for here I've done the competitors as the rows and the themes as the columns. And we can look at when people self-identify as loyal customers, where is it coming from?

So, what proportion of people at Morrisons are saying they're a loyal customer? So 15% in Morrisons talk about being a loyal customer compared to seven, eight percent at Aldi and Waitrose, which is interesting. And then if you look over here at churn risk, which is people talking about I'm never shopping there again, or I will never shop there again, you can see that it's kind of inverse relationship. So Aldi and Waitrose is much higher than Morrisons.

So a lot less people talk about churning at Morrisons. So interesting, Waitrose has the highest proportion of people talking about churn risk. So what I did, I did another cross-table on people at Waitrose talking about churning and brand perception, which is people talking about the brand as a whole, their standards, their idea of the brand. And interestingly, 46% of people that talk about leaving talk about

Brand perception in Waitrose. So from that you can gather that people are talking about not the standards not being what it used to. It's they used to like the brand, but it's not doing they're not as good as they used to be, which is where you can see this expectations, which is they're not matching my expectations basically, which 13% of people talking about leaving or churning mention their expectations. So that's something for Waitrose that

You can identify in the data of an issue and maybe a reason why the negativity around Waitrose is such a thing. Finally, we can look and this is where we can get more actionable insights. So this is what a negative what's driving negativity in each data set. So for Morrisons, we can look at what's driving the negativity in the data set and

This is things like contact center experience, delivery, collection communication, negative staff interaction. So these are the things that we would focus on if we were giving recommendations to Morrisons, as in where is the negativity coming from? Interestingly, if you compare it with Waitrose, Morrisons is a lot more external. So the contact center experience, the delivery communication, it's not necessarily in the store.

If we look at it versus Waitrose, you can see that things like payment experience, product in information, returns experiences a lot more internal in the actual store. And these are things that we would start to recommend. We'd start to look deeper into why people so negative about the payment experience, things like that, that they could then turn into actionable insights. So you're able to see what your competitors are doing well or less well that if you

So yeah, you're able to benchmark. So that's me done. My conclusion is Morrisons seems to be the best and I'm gonna get all my shopping from there in the future. So that's me done.

Pete: Thank you, Stella. Fussy, well done. You absolutely smashed that and you wouldn't even notice that URL. So well done. People never put a millennial in charge of naming themes, otherwise, you get themes like store vibes. So that's my learning from this whole experience. Thank you for that. Like I said, that now follows a party political broadcast on behalf of the Wordnerds Party. I will keep this very brief. If you like this stuff.

We can do this for you. We can do you a competitor benchmarking report fully managed by the nerds, your brand versus competitors, competitors. We can pull in your own data as well. I think where Stella was talking about, you know, if you change your if you're Morrisons and you change your kind of contact center stuff, you know, and that will have the biggest influence on satisfaction. There is a step there that is like, right, what do you do to solve that issue? What's the next level down? What is the problem with the contact center?

For that stuff, your own data is really, really helpful. So bringing all of that into one place and getting even deeper and even more actionable than that is great. So we can help you with that. What you get is an interactive Power BI dashboard, the like of which Stella just showed you, a key takeaway deck where we'll pull out some of these interesting things, show the evidence for it, and give you recommendations. And then we tend to run an online stakeholder workshop where you and other interested people in your organization.

Come together, we talk about what we've done, we discuss the kind of the pros and cons and you know the implications of it. And it's a great way of kind of socializing this stuff and bringing everybody along with you. If that's interesting, we can do that for about, ooh, sorry, we can do that for about six thousand pounds. There's a bit of a thing on the side of the screen saying book a chat with Pete. Obviously there's a bunch of other stuff we do as well. We've mentioned predictive modeling, all that kind of thing. Even if you just want to chat

Tell us what you're up to, ask some questions. You've probably spotted it. I love talking. So feel free to book a chat and it'd be lovely to hear from you. Thank you, Vic, for pushing that up. Right. That was it. I told you we'd keep it brief. There's loads of questions coming up in the chat. Thank you so much for that. I think the first couple, I think, are for Ross. So I think the first one is have you got UK stats for number of Trustpilot reviews, Ross?

Ross Hancock: Yeah, we have around about 120 million reviews in the UK, and that's growing at around about between 20 and 25 % per year now. And we expect that to grow exponentially.

Pete: Fantastic. And I think when we spoke about it a couple of months ago, I think one of the things that you're interested in is how you use this stuff across brands as well, to spot patterns within verticals and how consumer behavior is changing. And you know, is it worth just mentioning a little bit about that?

Ross Hancock: Yeah, absolutely. Yeah, yeah, 100%. I mean, there are a couple of applications for that. Yeah, really understanding how consumer behavior is changing, how that differs across geographies. It's also, and we're going to see this more and more, we're to see that customer feedback is going to be a key input into investment decision making as well. So the investors for your companies will be using review data, customer feedback data to manage their capital allocation across their portfolio.

So it's become increasingly important, not just to ensure that you are delivering a better customer experience and growing the business, but also for investment purposes as well.

Pete: Thank you. That's really helpful. Next question. Thank you, Kerry, for this one. What kinds of businesses are adopting this AI? Is it just retail or broader than that? Could it plan me a holiday? Ooh, good question.

Ross Hancock: Yeah, it's absolutely

Broadened that you're spot on. Retail is just one example. And actually, holidays is the whole travel industry is likely to be one of the first implementations. We expect OpenAI to be releasing their operator product in the UK in the next couple of months. And one of the core use cases of that will be to

Will be for travel purposes. You'll be able to ask it, hey, I want to go to Italy. Book me the best hotel as rated on TripAdvisor. And it will go ahead and do that. The way it does it is by effectively taking over your screen, your browser. It will screenshot the screen and do some image detection in order to navigate through that website.

So it will look through the TripAdvisor website, find the highest rated hotel, let's say, but then also consume kind of review data and that type of stuff in order to personalize that as well. Maybe that you value a great breakfast. It will able look at the feedback from the customer reviews and then take that appropriately.

So that is absolutely happening, including the purchasing experience as well, all within the interface.

Pete: Fantastic. Yeah. And it really underlines what you're saying about customer experience and how you need to really understand this and be proactive about finding out what your customers are learning about you at the point that they're consuming this stuff. There's loads more and we're running out of time, so I'm gonna push on. Paul, thank you for your question. Can you link the findings to a financial measure, i.e., improve revenue or value at risk? I can answer this.

Absolutely. So we spend a lot of our time building predictive models for people looking at from their feedback. Can we identify people who are going to churn? What's the chances that they're gonna leave your brand? We can we can because this is in Power BI, we can take any kind of quant data as well and look at what are the drivers compared to those metrics. So what things make spend go up, what things make spend go down, anywhere where you've got

Numeric data, we can use that to segment this feedback stuff. And you can do that as a lagging indicator, look historically at what's what's driven that and understand a deep dive of kind of what makes that makes that number happen. And then obviously the holy grail is building predictive models. Again, we use all kinds of techniques for that. Random forest analysis is one of our favorites at the minute. It's not perfect, but most of the time we're building models where

Out of the box, they start at 80, 85% accuracy and can improve from there. The beauty of this stuff is the more data you get, the better it gets. So, yes, using this to predict revenue is obviously a key thing that lots of people are interested in. Kerry, in your Power BI dashboard, Stella, I'm coming to you for this. In your Power BI dashboard, can you see the verbatim of what the reviewers are saying?

Stella: So yes, there are ways to see verbatim within the Power BI, but at Wordnerds we usually use go into the platform and we can see the verbatim there. So short answer, but yeah, you can kind of do both.

Pete: Absolutely. And why why do you tend to go on the platform?

Stella: Because I like the platform you can basically go deeper into the platform, you can cross it against other themes so you can get really into what a 10 customer's saying and you can also see what else is present in that comment so you can see what other themes are present, what else are they talking about.

Pete: Interested.

Thank you. Just for my own satisfaction. We've got time for one more before we go. Ross, this feels like it's for you. What's the difference between Trustpilot and Feefo? In the past, I felt Trustpilot was more open platform rather than needing to be a verified customer, invited by the company, et cetera. Is this changing? What makes Trustpilot better, stroke different as a platform?

Ross Hancock: Yeah, sure. Mean, Jimmy, you're absolutely spot on. Our core differentiator is that we are open. Anyone can leave a review. That said, if we know more information about the consumer, then that holds more weight in terms, not in terms of the trust score, which is displayed, but in terms of ensuring that it's more valuable to you when you do your custom analytics. The other core part is that it is open.

And what we do see, I'm not suggesting this isn't the case for Feefo, but on other platforms and you actually see this, the core part is if you see this, if you do some customer analytics on, say, feedback from a private tool, let's say it's something like Medallia or something like that where you're asking, you're directly asking the customer for feedback. You will not necessarily, one that will not necessarily be representative of that customer's experience.

Because there's inevitably some selection bias that goes on there. But also the information which the customer provides also tends to be biased as well. So we found that being open is absolutely core and critical and the key differentiated for us. And it provides you far better, can far more honest feedback as well in order to improve. I'm not sure Pete, you might even have some examples of when you've seen some differentiation between.

Private feedback and then you look at the public feedback and actually that one covers kind of big, big changes. Think, yeah.

Pete: Find all the time that brands get really frustrated because the way a customer feels doesn't always represent the reality. So they spend millions of pounds changing some core element of their service. But because they haven't communicated that often to the client well enough, the client complains about something and thinks that nothing's being done about it. And almost always it's a comms issue, not an actual service delivery issue. So what they're saying doesn't necessarily re reflect the

The actual situation, which I think is a version of what you're saying.

Ross Hancock: Yeah, so the core purpose is for us to give a true and accurate representation of how the customer is experiencing a certain product or service. And as I said, being open is the best way to achieve that.

Pete: I love it. Thank you. We are at time. I could talk about this all day. I am off to set up agentic AIs to go and research golf clubs for me because that's how interesting I am these days. Ross, thank you so much for your time. It's been fascinating as ever. We really appreciate you spending your morning with us. Stella, you smashed that. Well done. And thank you. Thank you, everybody, for joining us. Like I said, we really appreciate your time. You know where we are if you want to get in touch.

We will be sending some resources out afterwards, including the link to the other webinar on Gen AI. Do check that out. It's really fun. We will also be asking you for your feedback. Thank you very much. It's 12 o'clock. Go and have some lunch. Thank you again for your time.

Ross Hancock: Thank you.

Stella: Bye everyone.