AI Proof of Concept Fails in Customer Service: 4 Lessons Learned

Share on facebook
Share on twitter
Share on linkedin
Share on email
Share on google
AI Proof of Concept Fails in Customer Service: 4 Lessons Learned

It’s a sobering statistic: 85% of AI Proofs of Concept fail to deliver on their initial promise. This article explains the four main elements that are often neglected in a PoC, and explains how to avoid them to increase your chances of successfully implementing AI technology into your customer service system.

AI in Customer Service

Advances in Artificial Intelligence have delivered big wins for many B2C companies. Customer-facing AI technologies have enabled many features including:

  • biometric identification
  • voice recognition
  • predictive analytics
  • chatbots

For contact centers, key AI applications include workflow automation and agent decision support. What all these systems have in common is their reliance on accurate, actionable data and robust models. But the path of a successful AI Proof of Concept is often a bumpy one.

What is a Proof of Concept?

A Proof of Concept (PoC) is generally understood to be a project designed to demonstrate the feasibility of a proposed concept to solve a business need. However, many AI Proofs of Concept never make it to production. In fact, according to a 2020 IDC survey called “IDC FutureScape: Worldwide Utilities 2021 Predictions”, the majority of companies reported significant failures with their AI projects, and for 25% of the organizations, the failure rate has been as high as 50%.

When a company’s AI project is focused on customer service, it risks a double whammy if it fails: not only wasted time and resources, but also embarrassment and customer frustration which can severely damage the brand and lead to higher customer churn.

When AI Proofs of Concept Fail

Some of the world’s biggest players have learned this the hard way. Facebook’s M project, for example, was an unmitigated and expensive flop. It was initially conceived as a text-based version of Siri or Alexa that would assist users with scheduling appointments and dealing with customer service departments.

After a patchy PoC, it was then scaled back, and given orders to jump into Messenger conversations with timely, helpful suggestions. Unfortunately, M proved unequal to the task, offering various inappropriate interjections.

However, we are now in a period when many early adopting companies – which took their first abortive steps in AI five to seven years ago and failed – are now trying again. Now they’re smarter, having learned from their mistakes, and with recent technological advances, they’re willing to give it another go.

Lessons Learned From AI PoC Mistakes

Learning clear lessons from failed AI Proofs of Concept in customer service is critical in order to avoid further disappointments. Here are four mistakes to avoid when embarking upon an AI PoC.

Lesson 1: Setting Clear Goals

C-suite executives are the driving force behind 33% of AI projects and are usually highly optimistic about the impact that AI will have on company processes and productivity. However, if there is no real underlying technology need, the project is doomed from the outset. Company leadership must set clear goals and get comprehensive buy-in from the CTO, CIO and key staff at lower levels of the company.

It’s crucial to identify and explain the goals of the project in terms of specific use cases and measurable KPIs. For example, many customer-facing organizations are now aiming to drive higher levels of call deflection and therefore need AI systems geared towards intent prediction as well as self-service. But if the scope of the project is too wide, performance inevitably suffers.

Lesson 2: Robust Evaluation

An AI PoC can refer to an evaluation of the suitability of an independent software vendor’s product or a solution built in-house. Many companies – often blinded by effective marketing – end up overpaying for a third-party solution that doesn’t fully meet their needs or is a total disaster. Conversely, they can place too much of a burden on their own people to deliver a sophisticated solution.

They can also become victims of their own success. When a vendor offers a customer service AI product with a consumption-based pricing model, if the bot ends up handling more sessions than expected, chatbot costs can quickly spiral out of control.

The ability to properly evaluate an AI Proof of Concept depends on having access to transparent explanations and clear reasons for AI-based decisions. Known as Explainable AI (XAI), this approach aims to break down AI “thoughts” so all stakeholders can better determine the successes and failures of the AI model, allowing for a more robust PoC evaluation.

Lesson 3: Consider Language Barriers

Winston Churchill popularized a phrase said to have been coined by George Bernard Shaw: “Americans and British are one people separated by a common language.” While many chatbots are built to serve specific customer communities, they can still face numerous communication difficulties. Aside from the nuances of each dialect, chatbots must wade through missing punctuation, abbreviations, numbers replacing letters, local or generational slang and so on.

It’s vital to manage expectations. Chatbots work best for short conversations geared towards simple issues. The typical breakdown across industries is that short customer episodes account for around 60% of contacts and these are the interactions that will deliver the greatest successes.

Lesson 4: Streamlining Data

One of the highest barriers to success for an AI Proof of Concept is the lack of curated data needed to train the system. The data often exists, but cannot be accessed or analyzed efficiently, or is lacking in accuracy, relevance or reliability. If data is missing values, features inconsistent units of measurement, or includes duplicate values, the machine learning model will fail to produce a successful AI PoC.

It’s essential to develop a robust process for company and customer data collection, one that can scale to include new categories of information. Make sure data can be shared between all departments efficiently, and that it meets all quality requirements. An emerging approach in customer service is to crowdsource the expertise of contact center agents by analyzing their inputs and tagged images relating to each customer interaction.

Building a Successful AI Proof of Concept

Selecting or building an AI solution is a notoriously time-consuming and labor-intensive process that’s fraught with potential problems caused by everything from entrenched company culture to technological limitations and linguistic challenges. It is imperative that a company addresses the four issues discussed above. Following these guidelines will go a long way toward ensuring AI PoC success and eventual deployment.

Andrew Mort, Content Manager

Andrew Mort, Content Manager

Andrew Mort brings extensive experience of writing compelling B2B and B2C copy, including press releases, thought leadership articles and marketing content.


How Visual Assistance Resolves the Challenges of Delivering A Personalized Tenant Experience
Business Operations

Benefits of Remote Visual Assistance Technology For Property Managers and Tenants

Find out how property management organizations are leveraging remote support to enhance tenant experience as part of multiexperience journeys.

How to Optimize Customer Service Costs with Visual Assistance
Business Operations

How to Optimize Customer Service Costs with Visual Assistance

Quality customer service is expensive, but you can’t cut it! Optimize customer service costs and offer better service with visual assistance.

How Augmented Reality is Poised to Disrupt Field Services: Four Scenarios
Business Operations

Augmented Reality Field Service: The Enhancement Across Four Operating Models

Learn how AR-driven visual assistance plays a crucial role in field service improvement in four different operating scenarios