4 Lessons from AI Proof of Concept Failures in Customer Service
Advances in Artificial Intelligence (AI) have delivered big wins for many B2C companies. AI technologies have enabled biometric identification, voice recognition, predictive analytics and, of course, chatbots. For contact centers, key AI applications include workflow automation and agent decision support. What all these systems have in common is their reliance on accurate, actionable data and robust models. But the path of a successful AI Proof of Concept is often a bumpy one.
When an AI Proof of Concept Fails
A Proof of Concept (POC) is generally understood to be a project designed to demonstrate the feasibility of a proposed concept to solve a business need. However, many AI Proofs of Concept never make it to production. In fact, more than 25% of companies report a 50% failure rate for their desired AI projects, while one study reports that 85% of AI projects ultimately fail to deliver on their promises to business.
When a company’s AI project is focused on customer service, it risks a double whammy if it fails: not only the wasted time and resources, but also embarrassment and frustration which can severely damage the brand and lead to higher customer churn.
Some of the world’s biggest players have learned this the hard way. Facebook’s M project, for example, was an unmitigated and expensive flop. It was initially conceived as a text-based version of Siri or Alexa that would assist users with scheduling appointments and dealing with customer service departments. After a patchy POC, it was then scaled back, and given orders to jump into Messenger conversations with timely, helpful suggestions. Unfortunately, M proved unequal to the task, offering various inappropriate interjections.
However, we are now in a period when many early adopting companies – which took their first abortive steps in AI five to seven years ago and failed – are now trying again. Now they’re smarter, having learned from their mistakes, and with recent technological advances, they’re willing to give it another go.
Learning clear lessons from failed AI Proofs of Concept in customer service is critical to avoiding further disappointments. Here are four mistakes to avoid when embarking upon an AI POC.
1. Mistake: Not Setting Clear Goals
C-suite executives are the driving force behind 33% of AI projects and are usually highly optimistic about the impact that AI will have on company processes and productivity. However, if there is no real underlying technology need, the project is doomed from the outset. Company leadership must set clear goals and get comprehensive buy-in from the CTO, CIO and key staff at lower levels of the company.
Lesson learned: it’s crucial to identify and explain the goals of the project in terms of specific use cases and measurable KPIs. For example, many customer-facing organizations are now aiming to drive higher levels of call deflection and therefore need AI systems geared towards intent prediction as well as self-service. But if the scope of the project is too wide, performance inevitably suffers.
2. Making the wrong choice
An AI POC can refer to an evaluation of the suitability of an independent software vendor’s product or of a solution built in-house. Many companies – often blinded by effective marketing – end up overpaying for a third-party solution that doesn’t fully meet their needs or is a total disaster. Conversely, they can place too much of a burden on their own people to deliver a sophisticated solution. They can also become victims of their own success. When a vendor offers a customer service AI product with a consumption-based pricing model, if the bot ends up handling more sessions than expected, costs can quickly spiral out of control.
Lesson learned: the ability to properly evaluate an AI Proof of Concept depends on having access to transparent explanations and clear reasons for AI-based decisions. Known as Explainable AI (XAI), this approach aims to break down AI “thoughts” so all stakeholders can better determine the successes and failures of the AI model, allowing for a more robust POC evaluation.
3. Getting Lost in Translation
Winston Churchill popularized a phrase said to have been coined by George Bernard Shaw: “Americans and British are one people separated by a common language.” While many chatbots are built to serve specific customer communities, they can still face numerous communication difficulties. Aside from the nuances of each dialect, chatbots must wade through missing punctuation, abbreviations, numbers replacing letters, local or generational slang and so on.
Lesson learned: it’s vital to manage expectations. Chatbots work best for short conversations geared towards simple issues. The typical breakdown across industries is that short customer episodes account for around 60% of contacts and these are the interactions that will deliver the greatest successes.
4. The Devil is in the Data
One of the highest barriers to success for an AI Proof of Concept is the lack of curated data needed to train the system. The data often exists, but cannot be accessed or analyzed efficiently, or is lacking in accuracy, relevance or reliability. If data is missing values, features inconsistent units of measurement, or includes duplicate values, the machine learning model will fail to produce a successful AI POC.
Lesson learned: it’s essential to develop a robust process for company and customer data collection, one that can scale to include new categories of information. Make sure data can be shared among all departments efficiently, and that it meets all quality requirements. An emerging approach in customer service is to crowdsource the expertise of contact center agents by analyzing their inputs and tagged images relating to each customer interaction.
Selecting or building an AI solution is a notoriously time-consuming and labor-intensive process that’s fraught with potential problems caused by everything from entrenched company culture to technological limitations and linguistic challenges. Setting clear goals, evaluating the AI Proof of Concept properly, addressing language barriers, and streamlining data will go a long way toward ensuring POC success and eventual deployment.