Advances in Object Recognition Builds Trust in the Virtual Technician

TechSee explores how advances in image based object recognition can improve virtual technicians accuracy level

Gareth – the perfect robot?


“Gareth, the dishwasher seems to be clogged. Please fix it.”

“Affirmative Sir, will check and repair.”

“Great, I’ll be out on the deck.”

 It’s a beautiful day in 2030 and I’m relaxing with my entertainment system. I sit up suddenly.


Sniff, Sniff. “Gareth, are you cooking something? I smell something burning.”

“Negative, sir, just flushing the dishwasher hose, sir.”

“It really smells like gas in here, let me see…. Oh no, Gareth, what did you do? That’s the oven hose you’ve disconnected, not the dishwasher. Gas is leaking…FIRE! FIRE!”


Object recognition: only 3.5% error rate

Deep learning has achieved high levels of accuracy in recent years due in part to sophisticated advances in computer vision technologies, specifically in the area of object recognition. 

Since 2015, object recognition has attained an error rate of 3.5% – even lower than the 5% baseline human rate – meaning that today’s machines surpass humans in the ability to recognize objects. However, sometimes even the remaining 3.5% of errors can be critical.


Image recognition errors examples


Object recognition errors: Silly or serious?

Neural network mistakes can often be funny. For example, the Multimodal Recurrent Neural Network proposed in 2015 by Karpathy and Fei-Fei famously mistook a toothbrush for a baseball bat, and wrongly identified a soccer game for a tennis match. 

Why is object recognition important?

Sometimes errors can be dangerous – even the smallest visual error made by an autonomous car or a robotic doctor can be disastrous. When it comes to a virtual technician, errors in object recognition can be humorous such as when the virtual technician mistakes a cable for a snake, but can also be destructive. Hardware devices can be ruined, software can be damaged, and dangerous situations such as electrocution can occur.

There are several reasons for neural network errors. Computer algorithms lack human common sense. Therefore, the machine may fail to deduce certain logic, such as that a baby cannot lift a heavy baseball bat. Lack of computer learning is another reason.  If an algorithm was not trained with sufficient data, or more simply, did not see enough images of babies, toothbrushes or baseball bats, the correct objects will not be detected.

For this reason, deep learning modules often display an accuracy level.

What is an accuracy level?

An accuracy level is a number which represents the percentage of successful recognition. This number is based on the machine’s ability to correctly match the object to an existing class, or category of items in which the network was trained. 

For example: 

  • animals, 
  • food, 
  • hardware, 
  • road signs.

How can object recognition enhance the accuracy of the future virtual technician? 

Using image based object recognition can help to improve the accuracy level of virtual technicians, and reduce costly mistakes. What is the optimal way to use object recognition to achieve an increased accuracy level?

Data is the key

The best method of increasing the machine’s accuracy level is by extensive data collection. For a virtual technician to accurately recognize the exact models of a wide range of devices, cables or ports – hundreds of thousands of labeled images of each of these items are required.

However, collecting and tagging masses of data can be a long, costly and painful process for an enterprise. 

Data crowdsourcing + object recognition = increased accuracy level

One of the most efficient methods to execute this activity is via crowdsourcing. Imagine that a company has several thousands of customer service agents, and that each of them captures 2-3 images during every technical support call, tagging each image with the device model and specific technical issue. In a very short time, the enterprise will build up a massive data set that can be used to train algorithms to achieve high accuracy levels.


Next step: Continued optimization with image based object recognition

Once the algorithms achieve an acceptable level of accuracy, the enterprise must begin to focus on ensuring ongoing optimization of their algorithms. This can be done through small-scale testing. 

When an agent captures an image taken from a customer’s smartphone, the virtual assistant can be given an opportunity to recognize the device using object recognition and diagnose the customer’s technical issue. The agent then confirms or corrects the diagnosis, and in this way, allows for continuous learning and further improves the accuracy of the algorithms over time.


Increase object recognition accuracy levels with a 3-pronged approach

In today’s age of smart homes and plethora of digital devices on the market, achieving high accuracy levels for object recognition is essential for the effectiveness of the future virtual technician.

Over time, advancements in the following areas will ensure increased object recognition accuracy levels: 

  • Ongoing acquisition of data 
  • Continuous learning
  • Constant optimization 

No longer will computers mistake a cable for a snake, a toothbrush for a baseball bat, or an oven for a dishwasher… 

As a result, virtual technicians will be trusted with a wider range of technical support operations.

Subscribe to Our Content Updates

Get the latest on Customer Service Innovation in a brief occasional email.

    Thanks for subscribing

    Subscribe to Our Content Updates

    Get the latest on Customer Service Innovation in a brief occasional email.

      Thanks for subscribing
      This website uses cookies. We inform you that this site uses own, technical and third parties cookies to make sure our web page is user-friendly and to guarantee a high functionality of the webpage. More Information Accept

      SURVEY 2021

      How to calculate the true cost of a truck roll