Advances in Object Recognition Builds Trust in the Virtual Technician

over coming errors in image recognition

“Gareth, the dishwasher seems to be clogged. Please fix it.”

“Affirmative sir, will check and repair.”

“Great, I’ll be out on the deck.”


It’s a beautiful day in 2030 and I’m relaxing with my entertainment system. I sit up suddenly.

Sniff, Sniff. “Gareth, are you cooking something? I smell something burning.”

“Negative, sir, just flushing the dishwasher hose, sir.”

“It really smells like gas in here, let me see…. Oh no, Gareth, what did you do? That’s the oven hose you’ve disconnected, not the dishwasher. Gas is leaking…FIRE! FIRE!”


Deep learning has achieved high levels of accuracy in recent years due in part to sophisticated advances in computer vision technologies, specifically in the area of object recognition. Since 2015, object recognition has attained an error rate of 3.5% – even lower than the 5% baseline human rate – meaning that today’s machines surpass humans in the ability to recognize objects. However, sometimes even the remaining 3.5% of errors can be critical.


Image recognition errors examples


Object recognition errors: Silly or serious?

Neural network mistakes can often be funny. For example, the Multimodal Recurrent Neural Network proposed in 2015 by Karpathy and Fei-Fei famously mistook a toothbrush for a baseball bat, and wrongly identified a soccer game for a tennis match. Other errors can be dangerous – even the smallest visual error made by an autonomous car or a robotic doctor can be disastrous. When it comes to a virtual technician, errors in object recognition can be humorous such as when the virtual technician mistakes a cable for a snake, but can also be destructive. Hardware devices can be ruined, software can be damaged, and dangerous situations such as electrocution can occur.

There are several reasons for neural network errors. Computer algorithms lack human common sense, therefore, the machine may fail to deduce certain logic, such as that a baby cannot lift a heavy baseball bat. Lack of computer learning is another reason.  If an algorithm was not trained with sufficient data, or more simply, did not see enough images of babies, toothbrushes or baseball bats, the correct objects will not be detected.

For this reason, deep learning modules often display an accuracy level – a number which represents the percentage of successful recognition – for its ability to correctly match the object to an existing class, or category of items in which the network was trained. For example, animals, food, hardware, road signs, etc.

How can the accuracy of the future virtual technician be enhanced to reduce costly mistakes?


Data is the key

The best method of increasing the machine’s accuracy level is by extensive data collection. For a virtual technician to accurately recognize the exact models of a wide range of devices, cables or ports – hundreds of thousands of labeled images of each of these items are required.

However, collecting and tagging masses of data can be a long, costly and painful process for an enterprise. One of the most efficient methods to execute this activity is via crowdsourcing. Imagine that a company has several thousands of customer service agents, and that each of them captures 2-3 images during every technical support call, tagging each image with the device model and specific technical issue. In a very short time, the enterprise will build up a massive data set that can be used to train algorithms to achieve high accuracy levels.


Continued optimization

Once the algorithms achieve an acceptable level of accuracy, the enterprise must begin to focus on ensuring ongoing optimization of their algorithms. This can be done through small-scale testing. When an agent captures an image taken from customer’s smartphone, the virtual assistant can be given an opportunity to recognize the device and diagnose the customer’s technical issue. The agent then confirms or corrects the diagnosis, and in this way, allows for continuous learning and further improves the accuracy of the algorithms over time.


In today’s age of smart homes and plethora of digital devices on the market, achieving high accuracy levels is essential for the effectiveness of the future virtual technician. Over time, with ongoing acquisition of data, continuous learning and optimization, computers will no longer mistake a cable for a snake, a toothbrush for a baseball bat, or an oven for a dishwasher… and virtual technicians will be trusted with a wider range of technical support operations.

Subscribe to Our Content Updates

Get the latest on Customer Service Innovation in a brief occasional email.

    Thanks for subscribing

    Subscribe to Our Content Updates

    Get the latest on Customer Service Innovation in a brief occasional email.

      Thanks for subscribing
      This website uses cookies. We inform you that this site uses own, technical and third parties cookies to make sure our web page is user-friendly and to guarantee a high functionality of the webpage. More Information Accept

      REPORT 2021

      Transform Your Customer Service from a Cost Center to a Profit Center