On the Sofa with Karim Badr: Science, AI, and Assessment
SHL’s Research Scientist, Karim Badr, talks about how science, AI, and assessments intermingle and the role of AI in promoting fairness in hiring.
Share
This month we bring a science viewpoint to our blog by speaking with one of our team of scientists who work on developing assessments at SHL. Learn how scientific models can predict outcomes, what to consider when developing an assessment, and how AI can help bring fairness to hiring.
Could you tell us a bit about your role at SHL?
I work as part of our Science team, mostly focusing on developing new psychometric assessments and embedding new technological advancements into them. I have a psychology background and a passion for data and technology, so working in a company where science and technology are at the heart of everything really makes my role enjoyable.
Within the Science team, I work on next-gen assessments and products, where we research and test new technologies that have come to the market like AI, or industry developments that may impact assessments and how they are used. The role includes a lot of data analysis and looking for trends and insights that the data can provide, as well as a fair share of designing innovative data-driven products. This is only possible because one of the great assets that sets SHL apart is the sheer amount of data we have—a true delight for any scientist!
We also help in undertaking job analysis for organizations, understanding what the job entails and what the key behaviors that the person should demonstrate on the job. We then research how we can actually design an assessment that can measure those behaviors while keeping the candidate at the heart of this process.
Can a scientific model really predict people outcomes?
The simple answer is yes! The interesting part about people science is that it can never be exact. Unlike the physical sciences such as physics or biology which are more deterministic, people have too much variability to ever get the science 100% right. What we can do, though, is develop data-driven scientific models that can predict certain behaviors and characteristics.
It is one thing to analyze data and say ‘people who do X are more likely to do Y’, but it is a whole other beast to develop the right tool that can give you the data to confidently say that X and Y are related. That is both a challenge and a delight for us, and we are always after maximizing our validity evidence and ensuring we are measuring what we want to measure when developing these assessment models.
We never take anything to market that has not been developed properly, and we always do our due diligence and study our models until we have absolute confidence in them, ensuring they are objective and do not discriminate against any part of the population. Then we keep monitoring the data and aim to continuously improve where we can in terms of accuracy and efficiency.
What should people consider when deciding how to assess candidates?
One of the first things that should be asked is ‘what you are trying to assess or measure?’ This sounds simple, but organizations often get caught up in picking an assessment or solution just because they have heard it mentioned or seen some flashy demos without actually thinking if it is something that they could use or provides useful outcomes.
SHL has numerous assessments measuring lots of different things but not every one of them is suitable for every organization—each business has its own goals, ways of working, and priorities, so it is important to check that you are selecting the right assessment for your context.
Part of what our team does is work with our consultants to help customers understand what each assessment is measuring and match that with their requirements, so they know they are evaluating what is important to them. We can then ensure that the insights are actionable and help our customers reach their goals, whether it is improving hiring efficiency, selecting the right person for the right job, or developing employees.
It is one thing to analyze data and say ‘people who do X are more likely to do Y’, but it is a whole other beast to develop the right tool that can give you the data to confidently say that X and Y are related.
What is your favorite solution at SHL and why?
My favorite solution at SHL is Smart Interview on Demand, which is our tool that enables high-volume interviewing of candidates on demand in their own time, without the need for an interviewer to be present.
Using transparent AI to review responses and conduct interviews offers huge time savings but what may surprise some people is that it is just as fair and accurate as traditional interviewing methods, if not even more so. Like with any new technology, explaining the reasoning behind it is key—what drove this solution was the need to create the same experience with every candidate and be truly fair and objective. This is almost impossible with real people as if you are interviewing many candidates, it is very unlikely that the interviewer will be the same, and even if they are, they would naturally not be able to conduct the exact same interview with every candidate. If the candidate knows this reasoning, they are much more likely to engage with the process.
Like with any AI, it learns from the human, and human data can be full of bias that the AI can pick up on and learn itself. Our job is to continuously monitor what data we feed these models and make sure that they are not having any adverse impact on particular groups.
What are some of the trends you are seeing when it comes to assessments?
It is probably not going to come as a shock to anyone, but AI, especially generative AI, is revolutionizing the way we hire people, the way we retain people, the way we do our jobs, and the way we communicate. That does not mean it will take over everything, but if there are tasks that AI can automate or do more efficiently, then I think we should be open to using it. That is the approach we are taking with our work—we are testing and looking at ways AI can improve the assessment process while ensuring there is still a human-in-the-loop element to all of this.
Another trend that I am continuously observing is the shift from recruiters focusing on credentials such as qualifications and past experience to a more skill-based approach and looking at “soft skills”. I think calling them “soft” really does not do them justice as often they are the hardest to master.
This is where assessments can provide great insights to help measure these hard-to-measure skills and help understand a candidate’s potential, readiness, and fit for an organization.
What does the future hold for talent assessment?
I think there will be even more regulation about how we use AI in assessments. This is something as a team we are always keeping an eye on in collaboration with our expert Legal team as we are starting to see more legislation and ethical guidelines on the use of AI. I think it is important to know about them as things are evolving constantly and I am reassured that they are being put into practice.
At SHL, we ensure that innovative products are legally defensible and that any AI that we are using in our assessments and products is explainable. This means we can provide details of how the AI is predicting its scores, so it is not hidden behind a black box or an unknown quantity to the user. It is important that when developing AI-based tools or using AI, you can understand why it has come to the conclusions it has—otherwise how can you trust it?
Are you interested in bringing scientifically proven talent insights to your organization?
Contact SHL today to learn more or see how we can provide talent intelligence to your talent acquisition or talent management.
