How Augmented Intelligence Helps the Manual Grading of Video Responses
Manually scoring video interviews is a time-consuming and cognitively demanding task. Learn how Augmented Intelligence helps evaluators in grading video responses.
Share
Smart screening with recorded video interviews helps you uncover hidden talents that you might otherwise have missed. It goes beyond traditional screening and resumes as it introduces you to thousands of candidates in minutes—each with personal and unique stories. It provides a win-win situation for both our customers and candidates.
Our Smart Interview on Demand for example has helped our clients conduct several thousands of interviews in the last year alone. Most of these interviews are scored using AI algorithms which are trained to evaluate a candidate’s competency using Natural Language Processing (NLP) capabilities.
However, a set of responses need to be evaluated manually for a different set of reasons including
- Customers not choosing to evaluate candidate response through AI
- Candidate(s) not choosing themselves to be evaluated through AI algorithm.
Manually scoring video interviews is a time-consuming and cognitively demanding task. Often, an evaluator has to spend several hours every day to shift through a large number of candidates’ video responses.
The answer to this problem is Augmented Intelligence.
According to Gartner, Augmented Intelligence is a design pattern for a human-centered partnership model of people and Artificial Intelligence (AI) working together to enhance cognitive performance, including learning, decision making, and new experiences.
Figure 1: A snapshot of how Augmented Intelligence can help evaluators execute video interview grading efficiently.
Augmented Intelligence is a design pattern for a human-centered partnership model of people and Artificial Intelligence (AI) working together to enhance cognitive performance, including learning, decision making, and new experiences.
SHL’s Intelligent Player is an Augmented Intelligence solution designed to help the evaluators when they go through a set of recorded interviews. It will help evaluators in analyzing the responses, focusing their attention on important snippets, and improve their efficiency and turn-around time. Let’s look at some of the features of our Intelligent Player:
- Time-synced transcript: Human’s reading speed is almost 2x as compared to their listening speed. In fact, an average human reads around 250-300 words per minute, while the recommended talking speed for high comprehension is 150-160 words per minute. This makes reading more effective than listening. We provide a time-synced transcript along with recordings so that evaluators can read the candidate’s responses much faster with the flexibility to click and jump to any part of the video and vice-versa.
- Phrase highlighting: It is important for the evaluator to focus his/ her attention on key parts of the response/conversation. We have built a custom NLP algorithm to highlight specific phrases in a candidate response with two essential properties i.e., Atomicity and Diversity. Atomicity ensures that these phrases are self-contained, and Diversity ensures that the algorithm chooses different phrases that can provide an efficient summary of the response.
- Word-sentiment highlighting: This feature aims at highlighting the category (or polarity) of certain words in the phrases or sentences used by the candidate to answer questions. Word sentiment has been classified into three groups in our case: Positive Affect, Negative Affect, and Tentative Affect. Each of the groups is highlighted in different colors.
Reading is more effective than listening. An average human reads around 250 to 300 words per minute, while the recommended talking speed for high comprehension is 150 to 160 words per minute.
We have trained our NLP algorithms for both phrase and word sentiment highlighting on thousands of data points. These data points were labeled with the help of three different experienced raters. Our algorithms achieved human parity in highlighting different aspects of the candidate response.
Our deployments with selected customers have shown hundreds of hours saved for evaluators, directly resulting in significant cost savings.
What is next?
We continue to innovate on the technology for Intelligent Player and bring more Augmented Intelligence features that will help evaluators in doing candidate ratings effectively. If you have any feedback, please get in touch with us, we would love to hear it.
If you want to see the SHL’s Intelligent Player in action, please book a demo with one of our product specialists.
