‍

Your customers are constantly sharing valuable insights through surveys and feedback channels, but are you truly capturing the full value of this information and are you confident in the accuracy of your analysis?

We conducted a live experiment to tackle the pressing question many businesses are asking: "Can I use ChatGPT or Copilot to analyse my customer feedback?"

Here's what you can learn in this webinar:

1. Understanding the AI Analysis Challenge

"We're not intimidated by AI, but we need to understand its capabilities and limitations," explains Sarah Wilson, Account Manager at Wordnerds, as she frames the central question facing customer experience professionals.

With over 12,000 housing association survey responses our synthetic dataset, we demonstrate why generative AI tools like LLMs often struggle with customer feedback analysis:

• They provide plausible-sounding summaries but fabricate statistics with high confidence
• They struggle to verify their own claims when challenged
• Different queries about the same data yield contradictory results
• They lack the human context needed for accurate classification

"When you take feedback analysis to the board or senior leadership team that turns out to be wrong, you look like an idiot," Pete emphasises. "You need confidence in your data."

2. The Differences between Copilot and Specialist Feedback Analysis Tools

"Gen AI is really good at tasks where there's no wrong answer, but really bad at deterministic tasks where there is either a right answer or a wrong answer," explains Pete.

In this webinar Pete will take you through the evolution of customer feedback analysis, showing how specialised tools combine the best of AI and human expertise to:

1. Classify feedback with 85-95% accuracy using human-guided machine learning
2. Quantify exact volumes and sentiment for specific issues
3. Prioritise which problems will drive the greatest impact on satisfaction scores
4. Predict which complaints might escalate to regulatory authorities

"What allows you to solve problems is getting all the way down to specific issues, the things that are driving them, and the verbatim supporting that," Pete explains.

3. The AI Showdown

Stella, Wordnerds' insights and innovation analyst, conducts a real-time comparison showing how Copilot and a specialised analytics platform handle tenant feedback data differently:

Stella’s demo reveals:

• Copilots overconfident answers: Copilot claimed to find 3,700 repair mentions but couldn't verify with specific examples
• Being able to understand the volume of your issues: The Wordnerd's platform revealed precisely which repair issues were most discussed (contractor behaviour at 41%, quality of repair at 40%)
• Getting to actionable details: The platform enabled drilling down to specific verbatim comments about long repair times across 843 verified mentions

The result? Different approaches of feedback analysis lead to vastly different results.

In this event we cover:

[00:00 - 03:48] Introduction and overview of WordNerds, a customer feedback analytics platform serving social housing, financial services, retail, and travel sectors.

[03:48 - 06:03] Setting up the challenge: "Can I use Copilot or ChatGPT to analyse my customer feedback?" - exploring the limitations and expectations of generative AI tools.

[06:03 - 08:03] Explanation of the webinar structure and introduction to the synthesised housing association dataset.

[08:03 - 12:42] Pete explains the challenge and dataset: 12,289 rows of TSM (Tenant Satisfaction Measures) survey data for a fictional "Acme Housing" association.

[12:42 - 16:14] Audience participation to select a topic for live analysis, with "repairs" being chosen as the focus area.

[16:14 - 21:09] Demonstration of Copilot's initial analysis, showing how it provides plausible summaries but inconsistent numerical data about maintenance issues.

[21:09 - 25:48] Revealing Copilot's limitations: inconsistent results, inability to verify data claims, and tendency to fabricate statistics with high confidence.

[25:48 - 30:33] Technical explanation of how generative AI works with sentence embeddings and large language models, and why this approach struggles with deterministic data analysis tasks.

[30:33 - 36:20] Introduction to WordNerds' alternative approach: combining AI with human context to create structured data layers that enable accurate analysis and visualisation.

[36:20 - 41:07] Live demonstration by Stella comparing Copilot's analysis of repair issues with WordNerds' platform capabilities, showing the benefits of structured data visualisation.

[41:07 - 44:44] Q&A session addressing topics like multi-source data analysis, AI prompt engineering, and the limitations of generative AI for business intelligence.

[44:44 - 49:09] Overview of WordNerds' services, from initial consultation to proof of concept and subscription options.

[49:09 - 58:31] Extended Q&A on competitor approaches, the role of human oversight in AI analysis, and how specialised tools complement rather than replace data analysts.