Improving Virtual Technician Performance With Deep Learning

Explore with TechSee how computer vision can upgrade your virtual technician

Over the past few years, deep learning has driven significant improvements in computer vision accuracy and performance. Deep learning, the most advanced form of AI, enables independent learning of massive data sets. Unlike classic methods in which a human expert needs to define features (rules and attributes), deep learning can learn straight from data without human intervention, with  minor guidance (supervised learning) or without guidance at all (unsupervised learning).  In some fields, deep learning achieves far greater results than classic computer learning methods.

Let’s take a look at how we can use some of these computer vision technologies to build our virtual technicians of the future.

Object Recognition

Object recognition is a technology that enables finding and recognizing objects within images or videos. Object recognition includes several tasks, such as:

  • classifying that the image has a specific object
  • localizing the object in the picture
  • distinguishing the object from other objects
  • identifying parts within the object.

Since 2015, deep-learning-based object recognition has achieved amazing results, with the error rate dropping below 5% (the human level). This means that in several fields, today’s machines can recognize objects even better than human beings can!

This incredible accuracy makes object recognition a core technology for the future computer vision-powered virtual technician. To diagnose an Internet connectivity problem, for example, a virtual technician  needs to identify the router (and recognize the specific model), its parts (such as indication LEDs and back panel), and its cables; it also needs to localize all these objects to understand their context in order to diagnose the problem.

TechSee displays how virtual technicians can use computer vision to identify devices

Image 1: Identifying Modem and Printer with Object Recognition

Image to text:

Image to text is the process of using ‘deep semantic alignments’ to create textual descriptions. The technology allows the network to describe what it sees on an image in a simple sentence.

A machine develops this capability by:

  • recognizing objects and their locations within an image
  • converting this information into text
  • creating a meaningful contextual sentence to describe the image.

Using this technology, customers will be able to upload images of their equipment, after which a virtual technician with computer vision will  automatically describe in a sentence what it sees. For example, “A DLINK 5323 modem with a red light and ADSL cable disconnected”.

Such a description may help a human agent in a contact center diagnose the problem more quickly. Alternatively, it could also help by using Natural Language Processing (NLP) to automatically retrieve textual explanations of how to solve a problem.

TechSee provides an example of why creating a textual description of an image helps AI

Image 2: Textual description for a TV screen issue

Visual Similarity

Visual similarity enables finding similar images to the image that is provided to the machine. This capability is essential, as sometimes it is hard to explain an image in words, and the easiest way to a solution starts with  finding similar ones.

Visual search engines and websites such as Pinterest use visual similarity  to provide their users with similar images (with related objects, colors, or patterns). Visual similarity will allow virtual technicians to use computer vision to take images of a technical issue and search for similar issues within massive visual data sets of captured technical cases.

Explore with TechSee how computer vision helps virtual technicians find similar devices with related issues

Image 3: Finding similar issues with a specific router

Motion Detection

Motion detection is a technology that allows tracking moving objects in videos in real time. Motion detection is a fundamental capability in autonomous cars for ‘pedestrian detection’ capabilities; and in the security domain to identify moving persons detected by security cameras.

Motion detection is a key component of the future virtual technician, enabling it to provide instructions and feedback to customers in real time . For example, in order to instruct a customer on how to install a printer, a virtual technician needs to identify the customer’s hand movements and provide him with real-time instructions accordingly: for instance, “Now put the papers in the tray”, “Now hold the network cable. Not that one, the other one”.

TechSee’s computer vision technology can allow virtual technicians to track a moving object in real-time

Image 4: Motion detection in troubleshooting process

Face Recognition

Face recognition is recognizing faces within images. This is one of the most common tasks in computer vision. Deep learning has made significant improvements to machines’ facial recognition capability, especially in challenging lighting conditions, angles, and backgrounds.

“Good morning Mrs Brown,” chirps the virtual technician

With facial recognition, a virtual technician will be able to perform the critical task of recognizing customers, and greeting them accordingly:

“Hello Mrs. Brown, I see that you’re having trouble with your washing machine”.

Companies can also use facial recognition to offer biometric warranties, ensuring that customers get service for their devices without forcing them to save receipts and warranty documents.

Is Computer Vision the Way Forward for Virtual Technicians?

The progress of computer vision with deep learning will foster the creation of  ‘artificial eyes’ for the virtual technician of the future, and help us, the consumers, manage the growing number of smart devices that we’re setting up in our smart homes.

Liad Churchill, VP Product Marketing
Artificial Intelligence and Deep Learning expert, Liad Churchill, brings depth of knowledge in marketing smart technologies.

Subscribe to Our Content Updates

Get the latest on Customer Service Innovation in a brief occasional email.
Thanks for subscribing

Subscribe to Our Content Updates

Get the latest on Customer Service Innovation in a brief occasional email.
Thanks for subscribing
This website uses cookies. We inform you that this site uses own, technical and third parties cookies to make sure our web page is user-friendly and to guarantee a high functionality of the webpage. More Information Accept