AI Proof of Concept Fails in Customer Service: 4 Lessons Learned

TechSee explains what to be aware of when developing an AI Proof of Concept in customer service

It’s a sobering statistic: 85% of AI Proofs of Concept fail to deliver on their initial promise. This article explains the four main elements that are often neglected in a PoC, and explains how to avoid them to increase your chances of successfully implementing AI technology into your customer service system.

AI in Customer Service

Advances in Artificial Intelligence have delivered big wins for many B2C companies. Customer-facing AI technologies have enabled many features including:

  • biometric identification
  • voice recognition
  • predictive analytics
  • chatbots

For contact centers, key AI applications include workflow automation and agent decision support. What all these systems have in common is their reliance on accurate, actionable data and robust models. But the path of a successful AI Proof of Concept is often a bumpy one.

What is a Proof of Concept?

A Proof of Concept (PoC) is generally understood to be a project designed to demonstrate the feasibility of a proposed concept to solve a business need. However, many AI Proofs of Concept never make it to production. In fact, according to a recent IDC survey, the majority of companies reported significant failures with their AI projects, and for 25% of the organizations, the failure rate has been as high as 50%.

When a company’s AI project is focused on customer service, it risks a double whammy if it fails: not only wasted time and resources, but also embarrassment and customer frustration which can severely damage the brand and lead to higher customer churn.

When AI Proofs of Concept Fail

Facebook M’s flop

Some of the world’s biggest players have learned this the hard way. Facebook’s M project, for example, was an unmitigated and expensive flop. It was initially conceived as a text-based version of Siri or Alexa that would assist users with scheduling appointments and dealing with customer service departments.

After a patchy PoC, it was then scaled back, and given orders to jump into Messenger conversations with timely, helpful suggestions. Unfortunately, M proved unequal to the task, offering various inappropriate interjections.

However, we are now in a period when many early adopting companies – which took their first abortive steps in AI five to seven years ago and failed – are now trying again. Now they’re smarter, having learned from their mistakes, and with recent technological advances, they’re willing to give it another go.

Four mistakes to avoid in an AI PoC

Learning clear lessons from failed AI Proofs of Concept in customer service is critical in order to avoid further disappointments. Here are four mistakes to avoid when embarking upon an AI PoC.

1. Lack of clear goals for the AI project

C-suite executives are the driving force behind 33% of AI projects and are usually highly optimistic about the impact that AI will have on company processes and productivity. However, if there is no real underlying technology need, the project is doomed from the outset. Company leadership must set clear goals and get comprehensive buy-in from the CTO, CIO and key staff at lower levels of the company.

Lesson learned: it’s crucial to identify and explain the goals of the project in terms of specific use cases and measurable KPIs. For example, many customer-facing organizations are now aiming to drive higher levels of call deflection and therefore need AI systems geared towards intent prediction as well as self-service. But if the scope of the project is too wide, performance inevitably suffers.

2. Third-party or in-house: making the wrong choice

An AI PoC can refer to an evaluation of the suitability of an independent software vendor’s product or a solution built in-house. Many companies – often blinded by effective marketing – end up overpaying for a third-party solution that doesn’t fully meet their needs or is a total disaster. Conversely, they can place too much of a burden on their own people to deliver a sophisticated solution.

They can also become victims of their own success. When a vendor offers a customer service AI product with a consumption-based pricing model, if the bot ends up handling more sessions than expected, chatbot costs can quickly spiral out of control.

Lesson learned: the ability to properly evaluate an AI Proof of Concept depends on having access to transparent explanations and clear reasons for AI-based decisions. Known as Explainable AI (XAI), this approach aims to break down AI “thoughts” so all stakeholders can better determine the successes and failures of the AI model, allowing for a more robust PoC evaluation.

3. Human to AI: getting lost in translation

Winston Churchill popularized a phrase said to have been coined by George Bernard Shaw: “Americans and British are one people separated by a common language.” While many chatbots are built to serve specific customer communities, they can still face numerous communication difficulties. Aside from the nuances of each dialect, chatbots must wade through missing punctuation, abbreviations, numbers replacing letters, local or generational slang and so on.

Lesson learned: it’s vital to manage expectations. Chatbots work best for short conversations geared towards simple issues. The typical breakdown across industries is that short customer episodes account for around 60% of contacts and these are the interactions that will deliver the greatest successes.

4. The devil is in the data

One of the highest barriers to success for an AI Proof of Concept is the lack of curated data needed to train the system. The data often exists, but cannot be accessed or analyzed efficiently, or is lacking in accuracy, relevance or reliability. If data is missing values, features inconsistent units of measurement, or includes duplicate values, the machine learning model will fail to produce a successful AI PoC.

Lesson learned: it’s essential to develop a robust process for company and customer data collection, one that can scale to include new categories of information. Make sure data can be shared between all departments efficiently, and that it meets all quality requirements. An emerging approach in customer service is to crowdsource the expertise of contact center agents by analyzing their inputs and tagged images relating to each customer interaction.

Building a Successful AI Proof of Concept

Selecting or building an AI solution is a notoriously time-consuming and labor-intensive process that’s fraught with potential problems caused by everything from entrenched company culture to technological limitations and linguistic challenges. It is imperative that a company addresses the four issues discussed in this article by:

  • setting clear goals
  • evaluating the AI Proof of Concept properly
  • addressing language barriers
  • streamlining data

Following these guidelines will go a long way toward ensuring AI PoC success and eventual deployment.

Subscribe to Our Content Updates

Get the latest on Customer Service Innovation in a brief occasional email.

    Thanks for subscribing

    Subscribe to Our Content Updates

    Get the latest on Customer Service Innovation in a brief occasional email.

      Thanks for subscribing
      This website uses cookies. We inform you that this site uses own, technical and third parties cookies to make sure our web page is user-friendly and to guarantee a high functionality of the webpage. More Information Accept

      SURVEY 2021

      How to calculate the true cost of a truck roll