How do you analyse customer feedback in Power BI?

How does Wordnerds get customer feedback into Power BI?

Wordnerds uses a three-step framework. First, classification tags every sentence of tenant feedback with a sentiment score and the themes that matter to you, turning unstructured text into structured, quant-like data. Second, a semantic model translates that classified data into a structure optimised for reporting in Power BI. Third, visualization brings the feedback together with your existing quantitative data so teams can find, prioritise and share insight in a tool they already use. Under the bonnet, the data is uploaded to the Wordnerds platform where classification happens, passed through Amazon Redshift via an aggregator, fed into the semantic model, and presented in Power BI. It is the same tenant feedback throughout, now enriched with classification so a Power BI user can slice it by theme, sentiment and time alongside their other reporting.

How does the Wordnerds sentiment score work?

The sentiment score runs from 0 to 100, where 50 is average. Wordnerds uses a large language model to grade feedback across five steps, from very negative to very positive. Because a single comment can carry both positive and negative points, the model averages sentiment within each comment, and that figure is then averaged across the whole data set. Scores tend to form a bell curve around the 50 mark, so as a rule of thumb anything over 50 is good and anything under 50 needs work. In the live demo, filtering the dashboard to damp and mould returned a notably low score, which is exactly the kind of signal that tells an insight team where to look first. The point of a transparent, explainable score is that you can see how it is built rather than trusting a black-box number.

What is a semantic model and why do you need one?

A semantic model is a translation layer. Stephanie Clish illustrates it with Doug, the dog from the film Up, whose collar translates dog-speak into something humans understand. In the same way, a specialist classification tool structures feedback in a way that suits that tool, and the semantic model translates it into a structure optimised for reporting and analysis in Power BI. There are three challenges to building one in-house. First, you need expertise in the classification platform's data structure, and the people building the model are often not the end users of the analysis platform. Second, there is frequently a long backlog for BI development time. Third, if different tools classify different sources of feedback, you may need multiple models. Once the model is built, the rewarding part begins: bringing qualitative and quantitative data together in one familiar place.

What makes a customer insight actionable?

Steve Erdal's definition is deliberately plain: an actionable insight is an insight you can action. Flipping it over is more useful, because three things stop an insight being actionable. First, there is no obvious next step. "Customers don't like trains being late" leads nowhere; you have to keep digging, in Wordnerds' words, "keep on going until you find the action." Second, you cannot measure it. Without a number, you cannot prioritise which issue matters most or prove an intervention worked, whether that is volume, sentiment, or a correlation with the TSM score or ombudsman escalations. Third, nobody cares about it, which is why you attach the voice of a real customer, a verbatim that makes the issue land. When an insight has a next step, a measure and a human behind it, you can intervene, know what you expect to change, and show how you improved customers' lives.

How many themes can Wordnerds apply to one piece of feedback?

Within the TSM categories, Wordnerds has 141 different themes that can be attached to a single piece of feedback, and a comment is tagged into as many of them as it needs. Long, multi-point feedback might hit several themes; a short comment might hit one. New data is tagged automatically, which is what makes this workable at volume, something a human manually checking 141 aspects of every comment never could. The theme bank is not an off-the-shelf list: Wordnerds built it specifically around the Tenant Satisfaction Measures when the TSMs came in, training each category with a human in the loop. Customers can also train their own themes for issues specific to their region or questions, so the bank flexes to what each organisation cares about rather than forcing everyone into the same fixed set.

Can you combine Power BI feedback data with your own data warehouse?

Yes. Because the enriched data lands in Amazon Redshift, customers can connect to it and pull that data anywhere they want, including back into their own data lake to combine it with service-performance and other operational data for a more rounded picture. For the Power BI report itself, Wordnerds is in a beta for customer access: one user from your organisation connects to the Redshift data lake, Wordnerds sends the PBIX file containing the semantic model, and that user refreshes the data so the dashboards live in your own Microsoft tenant and can be shared widely. Because the PBIX file then sits with you, structural changes are currently handled as a manual step with Wordnerds rather than self-serve. The principle throughout is an open data approach: the insight goes where your teams already work, not locked inside another platform.

Full Webinar Transcript

Sarah Wilson: So hey everybody, thank you so much for joining our webinar. This is a super exciting one for us because Power BI has been something that we've been asked about for so long. So I am personally thrilled we're finally doing this webinar. Super exciting. I'm gonna take you through an agenda, but I thought I'd just start by introducing our speakers today. That's me. Awkwardly, apparently I only have one talk.

I do love the letter print. I'm Sarah Wilson and I work in the account management team here at Wordnerds. And I'll be your host today, so I'll be taking you through the sessions and the agenda. Good, everyone can hear me. That's always nice to hear. Then we roped in Steph. We've given Pete a week off, which I'm sure everyone's relieved about. And we've roped in Steph from product. So Steph's taken us through some of the challenges that she sees from clients.

And then we've brought Steve back as well. Think some of you will remember Steve from the series of webinars he ran last year. He's now heading up a newly formed team at Wordnerds, which is exciting. With all things reporting, yeah, it sounds like my nightmare, but he's just, yeah, doing amazing things. And so, yeah, welcome back, Steve.

So today's agenda, we've got about 40 minutes planned. We always say this stuff works much better as a dialogue and not a monologue. So please, we're going to have interactive points throughout the sessions. Please do let me know your questions, your burning questions. I want to put these guys to the test. That's what they're here for. I to make them sweat. Steph's first webinar, so yeah, we're looking for some tricky questions for her ideally.

So yeah, we're gonna start with setting the scene. That's gonna be me taking you through the problem and the framework. And then Steph's gonna jump into the framework in more detail. So the different parts of the problem and what you're ultimately looking to achieve. And then Steve's got a really great kind of live interactive demo. We like to test these things and we don't wanna do anything that's pre-rehearsed through Steve. So it's all in the platform. He's gonna take you through a kind of a full journey.

Which is really a great bit. I think I learned quite a lot from that. Then we've got time for Q &A, like I said, tends to be a really juicy part of the webinar. You know, this is a community of housing people want to make sure that we're learning from each other. And yeah, we like to steal everyone's ideas and pass them off as our own. So definitely give us all of your suggestions, your comments, your questions in the chat, as much interaction as possible would be great. Setting the scene, like I said, we're going to

Just to see where you're at with Power BI, how you're feeling about it, where you are in your journey. No wrong answers as we always say, like feel free to be as honest as you can. I think we're all here to learn right and doesn't matter how long you've been doing this. Yeah, there's still there's still plenty to learn with this tech as it's continually evolving. And like I said, any questions just pop them in the chat as we're going through. We do have a time at the end for questions, so we'll get to them all and try and give us many answers as we can.

At the end. So who are we? So we are Wordnerds. I can see tons of names I've heard of before, lots of familiar faces, lots of clients actually, which is really nice. So thanks so much for joining. But just in case you're new here, I think we've got some new names. We're Wordnerds. We're a customer feedback analysis specialist. Got some amazing brands that we're looking to work with and call clients.

We started out predominantly in the retail space, doing a lot with M&S, Sainsbury's, and then we moved into the social housing space with the change in regulations and the TSM's. And we also have been making inroads recently into financial services and travel and hospitality. So they're our main kind of sectors that we work across. Why do they all choose to work with us? So obviously they're all super different with super different use cases. But despite that, they've all got a really similar desired outcome. They all want to turn their role

Feedback into these deep actual insights that we keep hearing about that can be easily found, prioritized, and shared through the business. I think most importantly that they're actually confident in. For our experience of working with clients, we're seeing that Power BI is really seen as this silver bullet. Think like I said, we've been asked about it for a long time now, and that's because it's already embedded in your business. It's already within the wider teams, within the BI team and the wider business for all the quant data.

So obviously it seems like the next step to just say, well, just add in the call data in a similar fashion and call it a day. So you're probably thinking at this point, why did I sign up at all? This all sounds super straightforward. I'll just do that. Ultimately, though, I think we all know it's not quite as straightforward as that. As we always said at Wordnerds, if we to treat words like numbers, it'd be really easy. But unfortunately, this thing's like regional differences, sarcasm, nuances, loads and loads of misspellings.

And it's really hard to treat words like numbers in that way. So in this webinar, we're going to unpack specific challenges around doing that and how to work with call data for seven years. We've figured out the framework to make reporting this verbatim in Power BI possible. And by we, I obviously mean the product and data boffins. I had absolutely nothing to do with this. Yeah, just to make that clear, much to everyone's relief.

So introducing the framework. So Steph's put together this lovely kind of three step process to understand your customers in Power BI. You've got to start with the classification piece, getting everything to topics and working out what fits in what category. And then you've got to take that classification, translate it into a language that Power BI understands. So that's the semantic model side of things. And then once you've done that, get it into an output that you're comfortable with, that you can share through the business, that you can dice as much as you want.

And that's the visualization piece and making it kind of appealing for people in the business to understand.

So that hands me over to Steph. Steph's gonna take you through the framework in a bit more detail, challenges, outcomes. Yeah, so over to you, Steph.

Stephanie Clish: Thanks, Sarah. Hi everyone. So picking up where Sarah left off, let's dive straight into that first section of classification. And this is where it tends to start for a lot of the organizations that we see. You're doing a great job of gathering customer feedback, but you're left with a whole load of messy, unstructured text data and no good way of gleaning insights from it.

And so the goal of classification is to create a system where every single sentence that you're receiving from your tenants is tagged with a sentiment score, as well as lots of categories that are meaningful to you. And this gives us two outcomes. It makes the data easier to get insights from, but it's also structuring in a it in a way that's more like quant data. And that's good because that's what Power BI is designed for.

So, as Sarah mentioned, we've been doing this for many years now, seven or eight years. We've spoken to hundreds of organisations and distilled what they want into these three things. So they want accurate classifications, ones that are transparent so you can see the workings out. And they want them to be as broad and also as narrow as required so that they can perform deep dives and get to the root cause of their problems. And achieving these three things enables them to.

Trust the classification process and also be confident that they're making good decisions.

And we see housing associations go on a bit of a journey trying to find actionable insights from their qual data. They trial a bunch of different options, most of which end up falling short for what they actually need. And it often starts with manually tagging. You definitely don't need me to tell you that this activity is a massive time sink and puts pressures on the team. And generally, unless you've got a large group of people doing this.

You just can't keep up with the volumes of data that are coming in, meaning that lots of it's left on the shelf, unclassified. Teams also have a massive number, they sort of massively reduce the number of tags that they're applying to the data so that they can make it an achievable task, but this prevents that ability to do those deep dives that I just spoke about before. So they get super frustrated. They have so much rich data coming from the customers.

The customers are spending time giving them this data and they feel like they can't even scratch the surface with it. And at the same time, we all have generative AI at our fingertips now, and we see lots of people wanting to use the likes of Copilot to surface insights from their qual data. We did a webinar on this a few weeks ago, sort of showcasing the major problems with this technique. So if didn't catch it, it's on our website. But the TLDR version is that.

Gen AI are probabilistic systems. So they can't give insights teams that precision they need so that they can confidently surface, size, and prioritize their issues. More recently, though, there's been a move from business intelligence tools to offer LLM technology solutions like text analysis, and they function straight out of the box. These can be relatively quick to set up, which makes them super tempting. But at their heart, they're what we call black box AI.

Because they don't understand your context and the nuances that can exist between the themes you need and they don't show you their workings out. So you can't understand and therefore trust how and how well they're classifying your data.

Hopefully that gives you a bit of an understanding of the potential pitfalls of trying to achieve the goal of getting your data classified. In our experience, to do this well, and I'm a little bit biased, a specialist tool is definitely the best way. But with LLM technology being more accessible these days, and obviously it's super exciting, I don't think there's a tech team on the planet right now that doesn't want to go up working with it.

And so we often hear from the people we're speaking to, well, can we just build ourselves? And the answer is well, depending on the tech team that you've got, possibly. But at Wordnerds, we spend every day building and maintaining a platform solving these sorts of problems. I think we've got something like 142 different technologies, a team of 30 people that help us do that. And so it's really a it's a full time job.

And it would definitely take a bunch of resourcing and upkeep, which might not be suitable and take your teams off track. Whichever specialist tool you go with, though, once you're classifying the data, there's just one more step before you can start getting your data into Power BI. And I'm going to explain it with this guy. Does anyone know who he is? Pop it in the chat if you do. Bonus point if you can get his name.

Ricky's got it.

Okay, we've got some people with some good film taste. Yeah, you've got it. So this is Doug from the film Up. As you can see, Doug is a dog. Dogs can't speak English, but when Doug wears his magic collar, it translates dog speak so humans can understand him. And this is exactly what we need to do for Power BI. So if you're using a specialist tool for your classification, the data is structured and optimized in a way that's suitable for that platform.

The semantic model translates the data from your classification tool into a structure that's more optimized for reporting and analysis in Power BI.

There's three challenges to sort of building that semantic model. First, you need expertise in the classification platform's data structure. And often, if you're doing this in-house, the team building the semantic model aren't the end users of the analysis platform. And that makes it a little bit tricky to learn. Second, we're hearing from lots of organizations that also now more than ever, there's a huge backlog for BI development time.

So this might make it sort of a long time coming to get built. And then finally, if you have different tools classifying different sources of your customer feedback, that might require sort of multiple bot models to be built. And that's what a semantic model looks like, by the way. And once you've built this.

Set of phases are done and you can move on to the exciting stuff, which is bringing all of that quant data in with your core into one place so you can find, prioritize, and share your insights through the business. Most importantly using a tool that most of the departments in the business are familiar with. So hopefully that's been helpful. I'll pass back to Sarah now. Before I do, if you if you've got any questions, drop them in the chat and we'll pick them up in the QA.

Sarah Wilson: Amazing, thanks Steph. Yeah, I love that analogy. I don't know if anyone else does, but I think that's fab. So thank you so much for explaining that so succinctly. We're going to hand over to Steve now for the dashboard development side of things and the reporting. I have already teased that it's a great session Steve, so no pressure at all. Over to you.

Steve Erdal: Thank you so much, Sarah. I made no such promise. I want to make that very clear from the start. Thank you so much, Steph. That was really fascinating. I so you've got your data set up. You've classified it as Steph's described, so you can group it. And you've turned it into a version that you've translated into a version that the Power BI can understand. You still have a big job ahead of you. How do you then turn that into insights?

That will allow you and your wider team to actually improve customers' lives, what we tend to call actionable insight. And what we're going to do now is to look at how we take what Steph's described, take that foundation and actually create the actionable insights on top of that. There is gold in your customer feedback data. I was thinking actually, Steph, if I'd known what I was doing.

I should have used the prospector from Toy Story 2 and kept the Pixar theme going, but I've just got a regular gold prospector here. But the next step after you've got your data in a state that Power BI can understand and use it is then start to digging down layer by layer to find that actionable insight. First, though, you have to understand the jobs to be done.

Having the right question at the top is a bit like knowing where to dig. And those questions do vary. In this particular session, we're going to be looking at this, these first couple. So, what affects TSMs? And obviously, what are the key numbers that your senior leadership team need? But you might be looking at what is the root cause of, for example, complaints being escalated to the ombudsman, what is the what are the cause of them, what is generally going wrong and what can be done about it.

Often customers are also looking for positive things to take to the team because the feedback that we get tends to be more on the constructive criticism side, but often pulling out positive stuff can be really pot really great for the team. And if you're an insights, you'll know the the old question of tell me something I don't know as well. So these are areas that you might want to start looking, and that helps you to then know where to drill down to find that actionable insight.

A few other things to consider before we dive into the actual demo. Think about how do you get the most impact from the fewest possible visualizations on BI. I think if if you're like me, you're you get really into visualizations and you love what what BI can do. And I think often lots of visualizations.

Can make people who are less comfortable in this area feel slightly overwhelmed. How can you give them the most possible value with the least number of visualizations? Are you planning to democratize your data? So is this something that you're curating and you're passing out to the relevant people? Or do you want your frontline teams, your management, your senior resource team to be following their own curiosity and going through the data? Understanding that is really important because that will change how you go about presenting it.

And then finally, how do you go about imposing a narrative on this? I got into data because I loved stories. And you'll have heard the concept of data stories, but I think it's it's so important to think about story structure, think about beginning, middle, and end to the to the points that you're trying make, because that is the best way to properly chime with people to properly help them to understand and to want to act on it.

So with those things in mind, I'm now going to turn to the demo part of the presentation. Always a minorly nerve-wracking moment. This data that we're going to show you is, as I share my screen, this is synthetic data. This is data that's been created by an AI. So forgive it if the actual verbiage is a little bit a little bit stilted at times.

But obviously, we couldn't show any actual housing associations data. So this is just synthetic data. Right to the homepage. And I think the first thing to spot here is where we're we're getting into those big numbers for senior research teams. So looking at volume, at sentiment, at your overall TSM score. There's obviously challenges here with this particular organization. We've then got that across the different levels of sentiment and over time. On this side, we have those classifications that Steph was talking about. So

As she described, we've broken it down into different groups of people suffering the same issue. And that then allows me to compare these by volume, to look at them by sentiment. So I can see what people really like, helpful staff, all the way down to what people are really struggling with. I can also with this, then if I if I spot something that's interesting, I can click on it. So dampen mold is looking quite high.

Click on that, I can see what the TSM score is when you're just looking at damp and mold, what the sentiment is, how that's changing over time. There's clearly been some kind of damp and mold-based incident in July. And hopefully you can see already. Sorry, Sarah, do you have your hand up?

Sarah Wilson: So sorry, Steve, we just had a couple of questions. It might be helpful just to contextualize before we go through the rest of the data. Of course. Sorry, I can see Steve's as well. Had a comment that we can't see his screen. Can everyone else see it OK?

Yeah. OK, great. So just the questions, the questions, Steve, sorry to interrupt you mid-flow. I was asking the flow of data. So where's the data gone now that it's in Power BI? And also, can you explain the sentiment scale just before I move forward? So is it out of 100? Does it go to minus? How does the sentiment score work?

Steve Erdal: Brilliant. Thank you. Two great questions there. I'll do the second one first. So the score is out of 100. So 50 would be average. We break it down into five steps of sentiment from very negative to very positive using a large language model. We then kind of average that across the end of all the comments because some people might be saying positive and negative things within the same comment. And then we average that across the whole data set.

So this score here would be a obviously it's it's focusing on damp and molds. This is a very low score. The score goes up to from zero to 100, but scores tend to it tends to be something of a bell curve around the kind of 50 marks. So we tend to say over 50 is good, under 50 needs work. In terms of the journey data has gone on, Tim, with another great question. So as so the data has been.

Uploads to the workners platform where that classification piece that Steph's talked about has been done. The enriched data has then been passed through we use Redshift, but through Amazon Redshift, but through an aggregator to allow you to then to pass it into the semantic model that Steph just talked about. The semantic model has then presented this onto Power BI. Hope that makes sense, but it's still the same data.

It's just enriched now with the data, with the classification that Steph showed us earlier.

Sarah Wilson: Really helpful, thank you.

Steve Erdal: Not at all. Please do keep those questions coming. Sorry for for not spotting them, but please do keep them coming. Brilliant. So this is our top level. This is our homepage. We've got our big numbers, and hopefully you can already see we're starting to pull layers away. We're starting to see individual things. We knew nothing about this housing association 20 seconds ago. We now have a few potential suspects, a few potential hypotheses that we can go and explore in the data.

After this step, we tend to look at drivers of TSM. This is the area where we were talked about. It's like what can we do to improve our TSM scores? And the great thing about the classification that Steph's shown you is that you can then turn it into hierarchies, turn it into turn it into groups, which then allows you to look at specific challenges. So hopefully if these will be familiar concepts to you guys, they're designed to mirror the TSM questions that you will have to answer.

So maintenance repairs, safety, communication, complaint handling estates. What we've been able to do with the classification piece that Steph talked about is we can now see where our biggest challenges are with regard to TSM areas. And the particular area might jump out to you. So maintenance repairs, for example. I can then see what themes within that data are are creating this number at the top. So I can see.

The specific things that people are talking about. If something jumps out at me down here, so for example, here we've got the problem is getting worse, people talking about deteriorating issues. I can then click on that, and this will show me all the themes that overlap with that. And again, our old friend Dampen Mole is coming up there as something where people are talking about it worsening. And again, hopefully you can see we've just pulled off another layer. We've gone down from our top level.

Through the maintenance and repairs being the key sort of TSM issue you want to look at, finding a specific issue within that, and then seeing what is most likely to overlap with that issue. Now sometimes you might want to turn that into a journey process, particularly with things like repairs, where you have, excuse me, where there's a sort of linear progress that people make through it, from making an appointment through to the waiting for it, the repair itself.

The conduct of the operative, the the kind of the overall quality of the work, and then follow-up after. Hopefully you can see here the y-axis is our sentiment. So we can see it going down and up as through different parts of the process. And the volume is represented by the size of this bubble here. You see something interesting. I can then click on that. And that will then show what the kind of the key issues are that are coming out about that. So we can see.

There's an issue with incomplete work while people are waiting. And doors in particular seems to be an area where customers feel like they're waiting a long time for an appointment. Again, pulling back another layer, trying to get to that point where we are finding actionable stuff that we can actually do something about. The final level of that we tend to use this is called a cross-table. And this brings us to Aldwin's question.

Which but which he provided before the event, which is really much appreciated. About how you go about then overlapping our quant data with this kind of stuff. So you'll have loads of metadata about your customers that maybe you don't want to pass in through this system. So what we've done here is overlapped different categories, different themes. So we've got fixtures and fittings done this side, and then

Whether the customer felt like the the the contractor was reliable or unreliable. Now this could be you could use this to look at area, look at look at the area where the property is. You could look at type of property, demographic information about the customer, tenure, length of tenure. Our customers use all these kind of different extra information to add richness and value to.

This kind of work, but the idea is similar. We're looking at the intersections between different challenges to allow us to get a sense of where do we start, where do we go first in order to do stuff. In this particular case, you might want to look at just the custom the contractors that customers are finding unreliable. I'll just click on that to just show you up here. What this is giving us is a list of which of the fixtures and fittings.

Our customers most likely to say, I found this service to be unreliable. And we can see as a proportion of that of overall data in this issue, hot water has come at the top. So customers are most likely to talk about unreliability of contractor when hot water is involved. That's something that you can then go and action. You can talk to your contractor in that area about that issue. And obviously, you can also then overlap that with all that metadata that will allow you to better understand it.

We can continue drilling down, getting to the final level, which is the actual customer. So we're looking here at the unreliable customers, not people customers feeling that people don't aren't saying doing what they say. And here you can see hopefully that we're we're we've got right down now into the specific verbatim from the customers. This is synthetic data to reiterate, but

Getting that process from the very, very top, the big numbers, the scene, the things the senior leadership team need to know, tracking them over time, understanding how they're changing. Getting down into our TSM categories to see, okay, well, what where within the TSM framework are we, do we have the biggest challenges? Then down into our customer journey, how that's how how it feels for the customer across that period, finding those intersections and finally getting into.

This is what a customer actually said. Delighted to answer any questions about this process at the end, but generally that's how we set up our dashboards, thinking about that through line, thinking about that narrative, going from the very top right into here's an individual customer with this problem. I'm gonna hop back onto the deck now, which hopefully you can see again.

So wonderful. Congratulations. We have a dashboard. This is highly exciting news for any data person. The question now is so what? Because that doesn't matter. That dashboard alone will not make things happen for your customers. We sometimes think of this as the top levels of the dashboard are about hypothesizing in the way that we found Damp and Mold earlier. It's about finding potential areas that you might want to explore.

The cross-table and things like that are more about understanding. It's more about finding what the kind of the key issues are at that kind of next level down. You still then need to deliver that to your customers. And there's a couple of things that we kind of talk about with regard to this that we thought would worth mentioning. We think this is true whether you are presenting it in the dashboard, whether you write reports that get sent out, whether you present it to your senior leadership team.

In a presentation, I think there are a few things that you still need to do in order to get what that kind of holy grail of this kind of work is, which is actionable insight. Now that's a phrase you've probably heard before. It is ubiquitous in our industry. It's sort of become the name for the thing that we produce. People don't tend to do a very good job of defining it. What actually is an actionable insight? Well.

I hope you're all ready to have your minds absolutely blown. Prepare yourselves. I'm gonna drop a real truth bomb here. An actionable insight is an insight that you can action. I know, I know, hold your applause. It's it's some high level stuff here. I think in terms of what that actually means, flipping it on his head is quite useful. So you get an insight. What makes what makes an insight that you can't action? What how how does how does that work?

And I think there's for us there's three things that mean that you can't incite an action that you've got. You can't incite sorry, you can't action an insight, apologies, if you don't know what the next step is, if the next step isn't obvious. So we're at train companies, customers don't like trains being late is not an actionable insight because there's nothing to be done off the back of it. Oftentimes when we're to go back to that gold mining analogy, how you know it can send you down rabbit holes, it can send you.

We have a saying in our insight and innovation team at Wordnerds, which is keep on going until you find the action. That might be in terms of time and space, in terms of how things are changing or what area this is happening in. It might be different overlapping themes. It might be going back and looking at kind of unsupervised data to get a sense of but until you can say, and therefore we should do this.

Or and therefore have you considered doing this, or therefore we should stop doing this. Until you get to that next step being obvious, what you've got is not an actionable insight. The second thing that you need, and this one's fairly obvious, but the second thing, you can't action an insight if you can't measure it. Now that is something that you know, because if you've got if you can't measure an insight, you've got no way of prioritizing which issue is the biggest and the most important.

And when you go and try and fix that problem, you've got no way of knowing whether you succeeded. So, in order to make it something that you can intervene on, you need to be able to show when I've intervened on it, this is what we expect to change. And that might be volume, fewer customers describing an issue. It might be sentiment, people are fairly happier about it. It might be a correlation with something like the TSM score. So you might be able to say, this is accounting for a big drop in our TSM score. So when we fix it, this will improve.

Or it might be a correlation with another factor. It might, for example, escalation of the ombudsman, we're expecting them to go down. But if you don't have that number, then you've got no way of demonstrating the efficacy of the intervention that you want to make. Finally, and this is this is a harder one to talk about, but you also can't action an insight if you don't care about it. And we find this sometimes not in housing. I don't think we've ever experienced it in housing, but

In other industries, we've had customers where we found this really great insight, but the bottom line was they didn't care enough to look to change it. I don't think that's problem here. But I and I think there's a there's a limited amount that an inside professional can do in that space. But I think there are some key things that you can do to make people care. And the key one for me is give the voice of the actual customer. So find a verbatim that elucidates the point.

And in the same way that you can't call an axe an insight actionable if you can't measure it. If you don't have this is an individual, this is a real person who suffered this issue, I don't think you I don't think you're providing the best possible level of understanding and persuasion that will allow people to buy into your narrative about the issue. So you need to have an an obvious next step, you need to be able to measure.

The issue that you've described. And you need to find an individual for whom this is a problem. When you've got all these three things, whether you're doing this via via the dashboard, you're reporting on it, or you're presenting it to the SLT, once you've got those three things, you have an actual insight. And when we put those together,

In our reporting, it tends to look something like this. We're going back to our hot water example. So we can see here we've got our numbers. We go, we can show the numbers of the different issues. We've got a potential next step. So you know, so here's something that we can look at. We tend to put that in, we don't tend to order people, you must do this. We tend to talk about it in terms of hypotheses, but we have an obvious next step. And then finally, we have the

The real customers for whom this has become a problem. And that's the bit that ultimately makes people care about and think about this is something that this is a real person that I can help and I can make their life better. So this overall in total is what we would call an actual insight. And what allows you to do, it means you can intervene, you know what you expect to change when you intervene, and you can understand how you're actually improving your customers' lives.

Thank you so much for listening. I will pass back to Sarah. Please do keep questions coming. Delighted to explain anything that wasn't clear first time.

Sarah Wilson: Amazing. Thank you so much, Steve. That was brilliant. I think my favorite part is going from the top numbers to then seeing the actual verbatim and like you say, that data stories piece, which we know is so important. So thank you so much for that. So on to the juicy bit now. We've had a few questions in the chat. Do keep them coming. We're just going to get onto the Q &A now and a couple of potential next steps. So yeah, another couple of minutes to ask questions in the chat. There you go. Repeat that on my slide for you, please. Ask questions.

And we are going to send across a full resource packs and a few things we mentioned in the chat. So things like the co-pilot webinar and we've got a few different resources around Power BI. We've got a couple of kind of options that we put together of like a bias guide that way you can see the different models that Steph talked about. So you'll get a full resource pack and the recording after the call. I think the thing that we want the most is some feedback, please. We've got a mixture of people on the call. So I think both that kind of.

People who want to build it and people who want to buy it. So, you know, a mixture of Power BI analysts and also CX professionals who just want the insights, you know, yesterday. What I'd say is just get in touch with us, you know, whatever route you're taking, wherever you are on that journey, we've been there, we've done it, you know, we've made the mistakes. So do get in touch with me and we can have a chat about where you're at, what you're trying to achieve. And yeah.

How we can help ultimately. We've got, like I said, tons of resources, tons of contacts, lots of experience in this sector. So yeah, do get in touch with me. We have got a meeting in April as well too. We always start the month with like a big internal kind of content brainstorm. Yeah, and I like to do as little work as possible ideally. So please do let me know all of your topic suggestions, know, anything that you've got as a burning problem now. We know that submissions are coming up again. You know, what are you thinking about? What are you worried about?

What are you trying to figure out as the year goes on? Just let me know on email. Think most of got my email, so drop me message across. Now for the questions. So let's go back. I can see that Steph's starting to sweat. So one for Steve. This popped up when you were in the middle of your BI layers. Another one from Alderman likes to keep you busy. So how many layers and themes would you generally attach to one piece of feedback?

Steve Erdal: It's it's another great question, I'll I think we have within the TSM categories, I think we have 141 different themes that could be attached to a piece of feedback. And obviously when we talk about pieces of feedback, some feedback is really long form and goes through multiple points and some feedback is really short and has just you know has one key issue that it's talking about. So as as many as it needs is the answer to how many are are attached to an individual piece of feedback. But

Any new data that will go through this project will automatically be tagged into however many of those themes it should be in. Now obviously that's something that if you're manually tagging, you can't ask an individual person to go through 141 different aspects of the of the thing. But because we've trained these themes, it's possible. It's also worth saying that we allow our customers to train their own themes. So if there's a specific thing that you care about that isn't in our theme bank, you can absolutely

Get that sort of train up a small ai model to go through your data and tag into that aspect so so the sky's the limit ultimately on that but that's the sort of level of that's the sort of level that we found useful with regard to something as as big and complex as TSMs I think in housing because there's so many different types of issue and different types of things that can go wrong or indeed right we need that sort of and that's the sort of level that we start

Sarah Wilson: Really helpful, thank you. To be clear as well, when Steve's talking about the theme bank, that isn't just an out of the box solution. That's something that we've developed through the TSM's ourselves. So we sat down at the start of, you know, when the TSM's came out and we worked out the different categories and then we've all trained those ourselves. So it has that human in the loop interaction. And then like Steve says, we have that ready-made theme bank based on the TSM's, but we know that you're going to have your own questions.

Specific to your region, your geography, your questions, and you can tweak it however you'd like to have your own themes as well.

Right, let me go back down the questions.

So I think this is one for Steph. So with the rise of LLMs, are you thinking about using them in Wordnerds to create the action plans or at least suggest them off the back of the insights from all the customer feedback?

Stephanie Clish: That's a good question, Andrew. So yeah, we've got lots of sort of threads we're pulling in terms of how we can use kind of upcoming technology to help insights teams just to speed everything up. Like we we know that everyone's sort of time poor. But I think the key the key thing to mention that Sarah and Steve kind of just mentioned it before is

Like having that context and having that human actually know which things are the right things to do. So it I think for us it's about working with technology, but having that transparency and being able to allow the our users to sort of choose the right things. So yeah, it's but we're definitely looking at sort of different areas of technology. I think just also trying to

Find new ways of surfacing sort of better insights. So once you've kind of got those first two stages set up and you've you've got everything on Power BI, we're now doing some discovery with prediction. So, you know, trying to actually pinpoint, for example, in housing, which which complaints might be likely to be escalated to the Ombudsman, or which tenancies are are likely to

Become unsuccessful. So we're yeah, we're testing things like that at the minute. But yeah there's there's definitely lots of threads to pull I could I could talk a lot more about this but I don't I don't want to take up any of the time. Hopefully that's answered your question. If it hasn't, yeah, right write back and I'll I'll try my best.

Sarah Wilson: I think the good thing about this topic is that there's so many sub topics we can cover and we are going to do a series of webinars. So don't worry about it being a kind of catch all. You must have all the answers. Because I think that we, like I say, are going to run a series of these. Another question from Rory. This is a good one. Something that I've also noticed. I'm recently back from MatLeave. I came back last year. And I noticed the same thing, Rory. Recently migrated our context data within Wordnerds, so the new version, and found that it's a lot more accurate. How was this achieved?

We've actually, we've got a guy who's responsible for this in the data science team, but I'm sure that Stefan, Steve will be able to give very full answers. So who wants to take it?

Stephanie Clish: Yeah, I'll I'll say that. That's so yeah, like one of the most difficult things kind of mentioned this whole like maintaining and making sure the platform is working to its best potential is around understanding which technologies to use alongside our sort of bespoke things that we've got in place. And this was just a case of us building some new technology to

Make context themes more accurate. So yeah, we ended up switching out old context themes to that to that new technology. And that yeah, Steve has experience of a lot more experience of building themes, but they they seem to be you can you can do a lot more things in terms of what you're trying to classify now. Lots of behavioral stuff and things like that.

Sarah Wilson: I think my favorite part is the accuracy check-in. So I'm sure there's a few of you guys on the call that have seen it. I think before when you trained a thing, you weren't really sure. But now you're 100 % sure. Are you going in the right direction? It tells you when you've made enough effort, when you've had enough tries. And it's brilliant. So yeah, really good question. Thank you. One from Jack. So yeah, local lad. How would I connect to our Qual data with something like you guys showed?

The kind of technicalities around the floor.

Stephanie Clish: Yeah, so in terms of I mean it depends where your call data is, Jack. Sorry, I'm not sure whether you're a customer already or or not, but so the idea is you you find something to classify it. So a lot of our our our well our customers use us, and then you yeah, you go through those stages in the framework so that enable you to then get it into Power BI to start.

Building those those dashboards from in terms of if your core data is sort of I guess a bit everywhere you've got different survey tools doing different things it's helpful to sort of pull all of that into into one place and then use one tool to classify it so you've got that consistency and you can start to compare issues sort of on a on an even keel. I hope that's answered your question.

If it's not, yeah, let me know.

Sarah Wilson: We've got a question from a client. So, hey, Charlotte, Plymouth. At least I'm hoping it's you. I'm assuming it's you, fingers crossed. Is it possible to incorporate platform data into our own data warehouse so we can combine with service performance data to show a more rounded picture of how we're doing?

Stephanie Clish: Hi Charlotte. Yeah, it definitely is. So the benefits of us getting the data out into the likes of Redshift means that you can connect to that and you can pull that data and basically send it anywhere you want. So we do have some customers who want to put all of this back into one data data lake so that they can bring it all together and makes it more useful for them.

Yeah, that's definitely possible. Give us a shout. We can we can talk you through how to do that.

Sarah Wilson: Fab and there's a couple more. We've got, the end product Power BI report reside in the customer Microsoft tenant? If change is needed to it, would we contact you via a help desk?

Stephanie Clish: That's a good question, Stephen. Yeah, so we're in a bit a bit of a sort of beta release with this in terms of customer access. So what we're doing at the moment is one user from your organization would connect to the sm the Redshift data lake. We'd send you the PBIX file, which contains that semantic model, and then that user would be in charge of refreshing the data there and but they'd

It would essentially then live in your in your Microsoft account so that you can share those those dashboards reports more widely.

Sarah Wilson: Perfect.

Stephanie Clish: And sorry, yeah, to answer your second part, but the yeah, in terms of changes, yeah, it would it would kind of be a one of those manual things because the PBIX file would would sit with you. So we'd we'd be able to drop you another.

Sarah Wilson: Fab, thanks Steph. And then Steve, I think this could be a good one for you. So last question, do you find transactional surveys give better actionable insights than the TSM surveys?

Steve Erdal: Okay, so this is a wonderful and multifaceted question. I think that obviously with TSMs, there are specific things that you have to ask, and there's specific order that you have to ask them in and so on. And with transactional servers, you have a bit more leeway, a bit more freedom. We tend to find that obviously because it's it tends to be fresh in fresher in people's minds, in a transactional thing, in transaction transactional situation, that can then lead to more depth.

What I would say though is, and I think this is something that I would would would be my kind of number one takeaway for anything that we were doing at Wordnerds, is that people are weird. And they will give you really interesting advice about your communication channels in transactional surveys. They will give you really positive stuff about your staff in the middle of complaints. So oftentimes we find whilst looking at individual

Surveys is really important and really valuable. It can also be useful to have oversight of all this data together and being and grouping by those classifications so that everybody talking about you know damp and mold, regardless of whether they're talking about a transactional survey or a TSM or a complaint or an email or a call,

The often these and often by the way, these groups are siloed in housing associations. So there'll be one team that will be looking at transactional surveys, one team that will be looking at TSMs and so on. If you can unite those things, everybody gets more value out of it because whatever you whatever you care about, customers will be using all these different sources of contact. They'll be using all these different ways of contacting you. And the key thing, the gold, might be in any one of those groups.

So I would say that there's that there's pros and cons to both of them. I think there is probably more depth in transactional service that we've seen. But the key thing for me is to try and unite these data sets, excuse me, so that you can start to see so that however a customer chooses to express themselves and however whatever channel they choose to use, the right team gets access to it and gets an understanding of what the issue is.

Sarah Wilson: Thanks so much, Steve. That's really interesting. I think we're leaning into the complaints section now, aren't we, with expressions of dissatisfaction, however made. So yeah, super interesting. We do tend to find that transactional surveys are on the whole more positive than the TSM because it's based on someone's perception that's really hard to change. Whereas like you said, transactions are specific experience that they've had recently. Yeah, really interesting.

Any more questions or I think that's we've answered them all. We've really flown through them, haven't we? They were all brilliant. Thanks guys. Very thought provoking. Great, well, it looks like that's it then. Finished five minutes early so you can go and grab your sandwiches. Like I say, I'd really appreciate feedback. So please email me directly, give me a call. As you can probably tell, I absolutely love to chat. So yeah, just drop me a message. Let me know your feedback, any questions that you've got.

Or anything you'd like to see next because like I say we are running a series of these so look out for the next invite. Thanks so much guys and thanks Steph and Steve you did an amazing job. Bye guys.

Steve Erdal: Questions

Take a moment.

Stephanie Clish: Pai