As the move towards skills-based hiring intensifies, the role of assessment tools is becoming increasingly important. As employers focus on being more inclusive, it is critical to understand how the science behind assessment tools works to support this.
My guest this week is Kristin Allen, Senior Manager of Psychometrics at SHL. SHL has an applied research initiative, SHL Labs, driving talent technology innovation. One of the key research pillars is Diversity, Equity and Inclusion, and SHL has recently published the first results from its neurodiversity research program.
In the interview, we discuss the following:
• The role assessment plays in helping employers build diverse workforces.
• Fully inclusive talent solutions
• Using assessment to reduce bias
• Job analysis and validation
• Transparency
• Disability inclusion
• Using language that encourages candidates to ask for accommodations
• Spiky profiles
• Emerging best practices
• What does the future look like?
Listen to this podcast on Apple Podcasts.
Transcript
SHL Solutions (0s):
Support for this podcast is provided by SHL From Talent Acquisition to talent management. SHL Solutions provide your organization with the power and scale to build your business with the skilled, motivated, and energized workforce you need. SHL takes the guesswork out of growing a talented team by helping you match the right people to the right moments with simplicity and speed. Their quip, recruiters, and leaders with people insights at an organization, team, and individual level, accelerating growth, decision making, talent mobility, and inspiring an inclusive culture to build a future where businesses thrive because their people thrive visit shl.com to learn more.
Matt Alder (Intro) (1m 10s):
Hi there, this is Matt Alder. Welcome to Episode 472 of the Recruiting Future Podcast. As the move towards skills-based hiring intensifies, the role of assessment tools is becoming increasingly important. As employers focus on being more inclusive, it is critical to understand how the science behind assessment tools works to support this. My guest this week is Kristin Allen, Senior Manager of Psychometrics at SHL. SHL has an applied research initiative, SHL Labs, driving talent technology innovation.
Matt Alder (Intro) (1m 52s):
One of the key research pillars is Diversity, Equity, and Inclusion, and SHL has recently published the first results from its neurodiversity research program.
Matt Alder (2m 3s):
Hi Kristin and welcome to the podcast.
Kristin Allen (2m 6s):
Hi, Matt. Thank you so much for having me. It’s great to be here.
Matt Alder (2m 9s):
An, absolute pleasure to have you on the show. Please could you introduce yourself and tell us what you do?
Kristin Allen (2m 14s):
Yes, so my name is Kristin Allen. I am a Senior Manager on the Psychometrics team in the science function at SHL. I have a Ph.D. In Industrial and Organizational Psychology and I’ve been with SHL for over 10 years. SHL helps businesses by leveraging the power of people, science, and technology and our talent solutions help organizations leverage their people’s potential so they can maximize their business outcomes. Here at SHL, I lead a team of scientists on our Psychometrics team and we are responsible for developing, validating, and maintaining our portfolio of talent assessments. We are also involved in applied research through SHL Labs. SHL labs is a new initiative which combines our expertise across multiple disciplines to innovate for the future of talent and technology.
Kristin Allen (3m 2s):
One of those three pillars of SHL Labs is diversity, equity, and Inclusion, which is a key focus area for my team and what I’d like to talk more about today.
Matt Alder (3m 10s):
Fantastic stuff a absolutely really wanna kinda get into the kind of research that you are doing SHL Labs, particularly around diversity and neurodiversity. So really interesting question for me. What role does assessment play in helping employers to have diverse workforces?
Kristin Allen (3m 29s):
Yeah, so this is a really great question. Implementing inclusive talent programs can help to reduce unconscious bias because they can objectively and efficiently and effectively identify talent. So we strive to create fully inclusive talent solutions that can benefit organizations in support of achieving their diversity goals and the hiring outcomes that they want with the diverse talent pool. Our approach is to reduce common biases so that organizations can make the best and the fairest decisions through the use of talent assessments. And this allows those decisions to be made based on objective data rather than human judgment, which we all know can be biased. So when assessments are properly developed and validated, they can objectively identify the right people who have the potential to be successful in the job and remove the unconscious bias that’s so common in decision-making.
Matt Alder (4m 18s):
That’s really interesting stuff and I can see how that really sort of helps with the unconscious bias. How careful do you have to be in designing assessments to make sure that they achieve those kind of objectives?
Kristin Allen (4m 33s):
So it’s really important that assessments are developed properly. And the two things to look for there are job analysis, so making sure that the assessment is fit for purpose or that it establishes that it’s job relevant. So what’s being measured in the assessment is relevant to the job. And then the second piece there to consider is validation. So does the assessment performance on the assessment, is it related to performance on the job? So if you score better on the test, are you more likely to perform better on the job? If yes, then it’s a good predictor of success and useful tool to inform talent decisions.
Matt Alder (5m 8s):
And what kind of questions should employers be asking the people who design their assessments to make sure that they’ve got those kind of robust methodologies in place?
Kristin Allen (5m 20s):
Yeah, so details about job analysis and validation should be available in technical manuals from assessment providers. But there are a number of other things that you can think about beyond what’s in a technical manual to consider when you’re considering using an assessment. So one thing to to to think about is Disability Inclusion. There are a couple different facets of this. First is the assessment platform accessible to candidates with disabilities. Audits can be done on assessment platforms to ensure that they’re accessible to all and identify any other enhancements that can be made. So things like best practices would involve screen reader compatibility, offering different color contrast options for those who might have visual impairments, and more.
Kristin Allen (6m 4s):
So really simple things that can be done just to ensure that that platform is giving the best, most accessible experience for all candidates. The next thing there is to consider the assessment content and making sure that that’s both accessible and inclusive for candidates who might have disabilities. So employers can be sure to use Inclusive language in the recruitment process. And this is important because it encourages candidates to ask for accommodations when they’re needed. So, by employers ensuring that all have a fair chance to demonstrate their job-related knowledge, skills, and abilities through an assessment helps to ensure that there’s no bias in that process. Further knowing what types of accommodations can be offered and the employer is willing to offer will help to provide a fair and Inclusive experience for all candidates.
Matt Alder (6m 53s):
Now one of the things that you’ve been working with is a specific neurodiversity research program round assessment. Tell us about that and the scope of it and how it works.
Kristin Allen (7m 7s):
So this is a really exciting initiative that we have going on at SHL. So I’m gonna start with some background because this has been in the works for a couple of years now. SHL has a long history of commitment to Disability Inclusion. We’re members in the valuable 500, we’re partners with the organization called Purple and we are accredited as a Disability Confident Employer. But over the years we’ve watched the global commitment to disability inclusion gain momentum and we have been seeing increasing amounts of questions from our clients asking about accessibility in accommodations for our assessments. Recently, we’ve seen a large increase in this type of question in regards to neurodiverse talent. So when these questions started coming in at greater frequencies, you know, a couple of years back, we looked to the research to advise our clients, but we found that we were turning up with more questions than answers in the published research.
Kristin Allen (7m 58s):
So to give you a sense of the global impact of SHL, our assessments are used in 150 countries in 40 different languages, 25 million assessments per year are completed SHL assessments. So this means that a candidate is taking an SHL assessment every second of every day. We know that it’s estimated that 15 to 20% of the general population globally has a neurodivergent condition. So our candidates or our clients can bet that their candidate pools include neurodivergent talent. So we recognize this is a really important consideration for all organizations. Given the scope of our impact, we thought there is not enough research.
Kristin Allen (8m 39s):
The research is lagging behind practice. Our clients need answers. So who better to take on this research than us? We felt that we were uniquely positioned to research the evidence-based best practices for serving the neurodivergent community. We’ve got a long history of commitment to ensuring that our assessments are fair and accessible and we want to help our clients attract and retain neurodivergent talent. So in 2019, we launched SHL neurodiversity research program. This research program was launched with the goal of informing evidence-based practices for the field of talent assessment. And we want to get to the point where we can create selection tools that provide a fair and inclusive experience where all candidates have the opportunity to perform for the best of their abilities.
Kristin Allen (9m 25s):
Over the last three years, we’ve been engaging with academic research partners and client research partners on a number of research studies that are focused around two key questions. So the first is how do individuals with neurodivergent conditions perform on assessments? And the second is learning more about how do they experience or react to that, those assessment experiences and how can those be improved?
Matt Alder (9m 50s):
And tell us a little bit about how you doing the studies and actually kind of getting to the results before we talk about some of the results that have come out.
Kristin Allen (9m 58s):
So we are working with a lot of different partners. So we’ve got an academic research partner team at Florida International University that’s helping us run one of our studies. We have a website called SHL Direct, which is where participants can go to take practice assessments and we’re able to collect some data there. On that site, we have an opportunity for participants to optionally disclose any sort of neurodivergent conditions or disabilities for research purposes only and we can use that data to study impact on differences in scores and differences in reactions. We also are working with some client partners who can activate a similar disclosure form and applicant reactions form with their candidates.
Kristin Allen (10m 42s):
So to the extent that our candidates and our research participants feel comfortable disclosing that they have a neurodivergent condition, we can start to look at general population reactions and score differences or neurotypical compared to neurodivergent.
Matt Alder (10m 54s):
What are the results? Tell us about the kind of best practices that employers should be adopting when it comes to assessing Neurodiverse Candidates.
Kristin Allen (11m 6s):
Well, that’s a big question. So, far we’ve completed five research studies and there’s a lot of information there to digest, but I’ve got a couple key takeaways that I can share with you today. So, the first one is that a one size fits all approach just doesn’t apply. We know that there’s a wide range of conditions that fall under the umbrella term of neurodiversity and each need to be considered separately. So we found clients asking us is this assessment appropriate for neurodiverse candidates? But the answer really isn’t that simple. So within neurodiversity, there’s so many different conditions that may have different experiences and different impacts with assessments. Even within conditions, as employers are looking to make accommodations for candidates.
Kristin Allen (11m 51s):
They can’t assume that what you did for one candidate, let’s say with autism will be the same accommodation that works for another candidate with autism. So a really individualized approach is important. While the assessment content and the platform should be as consistent and inclusive as possible for all candidates, it’s important to take that individualized approach to understand how the experience might be affected by each individual’s neurodivergent condition and how modifications can be made to help that candidate best demonstrate their abilities.
Matt Alder (12m 26s):
One of the things that you mention in the findings is neurodivergent talent is likely to have a Spiky profile. Can you talk us through that a little bit?
Kristin Allen (12m 39s):
Yeah, absolutely. So what we found in our research so far and what’s out there in the published research does support this idea of a Spiky profile, which means there are areas of strengths and there are areas of challenges or opportunity. And so that is what we have been finding in our research so far as well. We have studied score differences between neurodivergent and neurotypical individuals in the areas of cognitive ability, personality, and behavioral competencies. And we found some really interesting findings. So the first couple of studies that we did were focused on cognitive ability. There we actually found on three different types of cognitive ability tests that there were no meaningful score differences, meaning that neurodivergent and neurotypical individuals were scoring very similarly on those types of cognitive ability tests.
Kristin Allen (13m 26s):
So that was really encouraging. We also found that the neurodivergent participants tended to actually complete them faster, which is interesting because when we talk about accommodations, oftentimes the default response is, “Okay, give the candidate more time if they need an accommodation.” Right? And so in this instance, we were showing that you know, time might not be the thing that’s making the difference in the experience where we actually found that those participants were completing those tests slightly faster. Now, when we looked at things like personality, we, again, I wanna caveat all of this by saying we’re scratching the surface on this research. We’ve got five studies under our belts, our sample sizes are small, and most of them, so this is the beginning of a long program of research.
Kristin Allen (14m 12s):
So, take what I’m sharing here today, just knowing that it’s preliminary findings and that we’re gonna continue to try to replicate this in further studies, but I will share what we know so far.
Matt Alder (14m 22s):
Absolutely.
Kristin Allen (14m 22s):
For personality, we did find that there are strength in a number of areas. So things like creativity, innovation, strategic thinking, learning, applying expertise in critical thinking. These are those areas that emerged as strengths for this particular sample of neurodivergent individuals. Similarly, we found there’s some challenge areas. So things like delivering results, planning and organizing, decision-making, and resilience came out as some of the areas where the neurodivergent participants scored a little bit lower. So we, we are recognizing that there are clear strengths of this talent pool, but there’s also areas that may be more challenging in terms of their assessment scores.
Matt Alder (15m 7s):
And in terms of the types of accommodations that employers could make during the assessment process, what are we talking about? Big changes, small changes? What are the kind of things that you are finding could be useful?
Kristin Allen (15m 23s):
Yeah, so that’s an area of research interest to us. So we have just completed a study with that involved extensive interviews with autistic participants. And one of our goals in that process to truly understand their experience with our assessments was to ask about their willingness to disclose a neurodivergent condition as well as what types of accommodations might be useful across different assessment types. So we’re just beginning to uncover their certain things that would be useful. It’s gonna be a focus area for us in 2023 in some of our new studies. But what I can share with you is that there’s huge value in gaining feedback directly from the neurodivergent community.
Kristin Allen (16m 3s):
So it’s this whole idea about nothing about us without us and the importance of involving the voice of the neurodivergent community directly in this research. We found that some of the suggestions that our participants made for accommodations were actually small changes that had the potential to make a big difference. So things like word choice and being really careful about using inclusive words, providing clearer and more detailed instructions about what to expect in the assessment. And things as simple as allowing for breaks in between assessment components that are paired together in a single session. So, these are examples of things that require little effort and no cost to implement but could significantly improve the candidate experience.
Kristin Allen (16m 48s):
So we are trying to uncover those features that can help or hinder the experience so that we can learn how to improve it. Ultimately with the goal of developing our assessments in a way that relieves the burden from the candidates. So they are not required to disclose that they have a neurodivergent condition and that potentially a need for accommodation. We know from our research that participants, even in a research setting, so even in a low stake setting are very hesitant to disclose a neurodivergent condition. When we look at our research forms the number of people in our research samples that are disclosing neurodivergent conditions are much smaller than the global estimated prevalence of those conditions.
Kristin Allen (17m 34s):
So we know that there’s a hesitancy to disclose and we know that that must be exaggerated in a high-stakes selection setting. So if you’re applying for a job, you’re going to be likely even more hesitant to disclose that condition because you might worry that it would impact the hiring decision. So, here we just wanna recommend that employers are using positive language, that inclusive language and explaining the benefits of requesting accommodations. Letting candidates know that there is no judgment and we are aiming to provide an accessible and inclusive experience for you. We are able to offer accommodations and are willing to do so. Just to kind of open the door for them.
Matt Alder (18m 16s):
Absolutely. You mentioned some of the things that you’ll be focusing on in the research program as it progresses. Tell us a little bit more about what’s next for this research.
Kristin Allen (18m 26s):
So we have big plans for 2023. We have recently grown our internal team to include more scientists that are working on this research. We are also adding new academic and client research partners to our research program for 2023. So we’ve got a number of new studies planned. We’re looking to expand our research to include more assessment types and also a broader range of neurodivergent conditions. We’ll be looking at things like disclosure rates, the impact of Inclusive language to try to get some really tangible and practical recommendations for organizations in terms of what types of language to use, and learning more about the modifications that would be useful to neurodivergent candidates.
Kristin Allen (19m 6s):
Additionally, we’ll be looking to continue to collect more data in the types of studies that we’ve completed so far to increase those sample sizes and just bolster the confidence in the findings that we’ve got so far through replicating them in new studies.
Matt Alder (19m 21s):
And final question, can people get involved with this? Is this something people can help you with?
Kristin Allen (19m 26s):
Absolutely. So we are looking for research partners to help us to progress this important research. So we’ve got opportunities for clients and academic partners to work together with us. If you are interested, please do reach out to us. We would love to talk to you and work together in a way that can help to progress this research. We will be publishing a white paper on November 1st that will be summarizing the findings from these first five studies in more detail than what I was able to share today and also some actionable takeaways for organizations. So keep an eye out for that. You can also look at our blog for regular updates on our research program and we’ll continue to publish our findings on our blogs through white papers.
Kristin Allen (20m 7s):
We’ve got some conference presentations coming up and some research publications that we’ll be submitting. So we really wanna work together, partner together, and share our ideas with the talent assessment industry so that we can break down barriers to employment for the neurodivergent community.
Matt Alder (20m 23s):
Kristin, thank you very much for talking to me.
Kristin Allen (20m 25s):
You’re welcome. It was great. Thank you so much for having me today.
Matt Alder (20m 30s):
My thanks to Kristin. You can subscribe to this podcast in Apple Podcasts, on Spotify, or via your podcasting app of choice. Please also follow the show on Instagram. You can find us by searching for Recruiting Future. You can search all the past episodes at recruitingfuture.com. On that site, You could also subscribe to our new monthly newsletter to get the inside track about everything that’s coming up on the show. Thanks very much for listening. I’ll be back next time and I hope you’ll join me.
