Recruiting Future is a podcast designed to help Talent Acquisition teams drive measurable impact by developing strategic capability in Foresight, Influence, Talent, and Technology.
This episode is about talent and technology.
Candidates’ use of AI is an increasing problem for many employers. Along with dramatically increasing application numbers, AI tools are, in some cases, allowing potentially unsuitable candidates to jump to the front of the queue, endangering the integrity of the recruiting process. One potential answer to this challenge is to deploy science-based psychometric testing much earlier in the process than has traditionally been the case
So how should assessment tools and technologies evolve to combine scientific robustness, candidate engagement, and resilience against AI manipulation
My guest this week is Alan Bourne, Partner at Omatti and an expert in assessment innovation. Alan highlights the issues with legacy assessment tools and sets out a blueprint for transformation that embraces AI to enhance both the candidate experience and the predictive accuracy of the tools. We also discuss the role of assessment in the move to skills organisations.
In the interview, we discuss
• The real impact of candidates using AI
• Why resumes & interviews are the least effective and least fair way of assessment
• Defensive proctoring versus proactive innovation
• Using AI to develop immersive, simulation-based assessments
• Getting the right balance between science and engagement
• The progress of skills-based hiring
• The danger of just replacing experience-based CVs with experience-based skills
• What will the future look like?
Follow this podcast on Apple Podcasts.
Follow this podcast on Spotify.
00:00
Matt Alder
AI is breaking the traditional recruiting process by overwhelming recruiters with hundreds of perfect CVs and providing candidates with a cheat code for getting hired. Could long established science be the solution that we’re looking for to defend the integrity of the hiring process? Keep listening to find out.
00:22
Matt Alder
Support for this podcast comes from Greenhouse. Greenhouse is the only hiring software you’ll ever need from Outreach to offer. Greenhouse helps companies get measurably better at hiring with smarter, more efficient solutions powered by built in AI. With Greenhouse AI you can generate stronger candidate pools faster and source high quality talent with more precision, streamline the interview process with automation tools and make faster, more confident hiring decisions with AI powered reporting. Greenhouse has helped over 7,500 customers across diverse industry verticals from early stage to enterprise become great at hiring, including companies like Airbnb, HubSpot, Lyft, SeatGeek, HelloFresh and DoorDash. If you’re ready to put the power of AI into the hands of your hiring team, you can visit greenhouse.com to learn more.
01:44
Matt Alder
Hi there. Welcome to episode 705 of Recruiting Future with me, Matt Alder. Recruiting Future is a podcast designed to help talent acquisition teams drive measurable impact by developing strategic capability in foresight, influence talent and technology. This episode is about talent and technology candidates. Use of AI is an increasing problem for many employers. Along with dramatically increasing application numbers. AI tools are in some cases allowing potentially unsuitable candidates to jump to the front of the queue, endangering the integrity of the recruiting process. One potential answer to this challenge is to deploy science based psychometric testing much earlier in the process than has traditionally been the case. So how should assessment tools and technologies evolve to combine scientific robustness, candidate engagement and resilience against AI manipulation? My guest this week is Alan Bourne, partner at Omatti and an expert in assessment innovation.
02:54
Matt Alder
Alan highlights the issues with legacy assessment tools and sets out a blueprint for transformation that embraces AI to enhance both the candidate experience and the predictive accuracy of the tools. We also discuss the role of assessment in the move towards skills based organizations.
03:13
Matt Alder
Hi Alan and welcome to the podcast.
03:16
Alan Bourne
Great to be here. Thank you.
03:18
Matt Alder
An absolute pleasure to have you on the show. Please could you introduce yourself and tell everyone what you do?
03:24
Alan Bourne
Yeah, sure Matt. Thank you. Yes, so I’m Alan Bourne. I’m an organization psychologist, was a co founder at TalentQ and a founder at Sova. So I’ve been very involved in the Product side of assessment over the years. Currently I’m partner at Omatti where I help organizations review and develop their assessment capability.
03:43
Matt Alder
Fantastic. So you’ve been working in the assessment space for a very long time. How have you seen things develop over the. I suppose, particularly over the last few years. And what do you think the key issues are in assessment at the moment?
03:54
Alan Bourne
Yeah, sure. I mean, I think probably over the last decade the industry has developed incrementally at being quite a lot better at tailoring to each client’s needs. So it used to be pretty very off the shelf prior to that. And as things have become more digitized, it become better at mapping to sort of different customers, kind of needs, behaviors, competencies, etc. Which is great. But meanwhile it all still continues to be very questionnaire based or sort of test or form based. There’s been a bit of experimentation with things like gamification, but I think assessment still struggles to be engaging and relevant to the job at the same time. And that’s been the sort of eternal challenge that hasn’t quite been solved. So whilst it’s improved, it’s still not overcome that.
04:42
Alan Bourne
And we either have tools that are very sort of scientifically driven, but perhaps a bit boring or whizzy, but it’s unclear how well they work in. The big challenge now really is we’ve got to solve that. And now we’ve got candidates using AI and they’re arguably winning the arms race in terms of overcoming assessment. So we’ve got quite a few significant issues to deal with, I think.
05:07
Matt Alder
Yeah, absolutely. I want to talk in depth about candidates and AI in a second, but I just suppose something kind of struck me as you were talking there. The assessment as a science has been around for a really long time and I’m kind of always amazed that it’s kind of ignored in a lot of recruiting. You know, people are either ignorant that there’s science out there that can predict how well people will be will doing jobs and things like that, or choose not to use it. And I think you touched on some of the reasons there. But what’s your kind of view on that? Why is this just not an absolute sort of used every single time?
05:43
Alan Bourne
That’s a very good question. I mean, I think there’s probably a couple of things there. One of them is the extent to which this sort of assessment industry within the wider talent field has succeeded in getting its message across. And I think it’s probably, you know, there’d be parts of the industry that have been in their own ivory tower, let’s say, and other things like that. So that there’s certainly some of that’s gone on, but also just making it clear what the business benefits are and really articulating it. So there’s that side of it. Meanwhile, actually, 90% of the time people are still using a CV in an interview, which is the least predictive and most unfair way of doing things. So whilst assessment has expanded, it’s still got a huge distance to go, I think.
06:25
Matt Alder
Yeah, no, 100%. I mean, it’s kind of crazy that CVs and interviews are so still just the norm, really. And I suppose the big thing here is candidates now using AI to apply and kind of effectively breaking the CV and interview process that exists. What are you seeing happening? What do you think the impact is? And do you think companies are really appreciating the significance of what’s going on?
06:49
Alan Bourne
I think it’s increasingly getting some daylight. I think there’s a lot of. It might be a bit unfair, but I think there’s a bit of willful sort of ostrich syndrome because it’s just too, it feels to many like too difficult a problem to solve. But yeah, you’ve got candidates who can, you know, use ChatGPT or another language model, whatever they choose to firstly script what they’re going to say, how to answer interview questions, and there are people doing that real time interviews or video interviews, and then also for certain types of test question, whether it’s, you know, cognitive or situational questions, those kind of things. Anything that’s text based or that can be quickly heard by a language model, you can get a piece of AI to help you say what a good answer is. And so there’s a lot of it going on.
07:36
Alan Bourne
And the problem you’ve kind of got is it’s a minority of people doing it. There’s estimates between maybe 8 and 30%. So let’s say if you took 15% as a sort of illustrative number that I’ve heard from a few places, that minority are doing quite a good job, unfortunately, of leapfrogging past a lot of the other candidates, particularly when it’s high volume, like early careers and those kind of things. And the downside of that is if you kind of leapfrog to the front, then it might be 15 of people at the, you know, at the beginning, but that might be, you know, 30 or more actually get hired because they’ve all jumped to the front of the queue. So it raises a lot of questions. About the integrity of the process of actually the candidates as well.
08:20
Alan Bourne
And there’s a risk of making poor hires. So, you know, it’s been in, you know, Bloomberg and BBC and etc have covered the story, but it hasn’t quite, I don’t think really penetrated practice yet.
08:32
Matt Alder
Yeah, and I think people tend to look at kind of overall numbers, you know, have their application numbers gone up rather than sort of digging into, well, what’s actually happening within those numbers as well.
08:42
Alan Bourne
Yes, exactly. And you know, to your point, I think there was a institute for student employers, stat, from looking at the last year that said applications had gone up 59%. Now some of that is believed to be due to people using AI to do their applications, which you can understand. The issue that follows is if you’ve then, when you assess them, have asked them not to use AI and some of the people are, that’s when it starts to get pretty messy. And I don’t, you know, people do things like look at what the average scores are, but that can hide quite a lot of sins. So we need more precise ways of understanding what’s going on and ways of mitigating it, I think.
09:19
Matt Alder
And how can assessments solve that problem? What do you think the way forward with it is?
09:24
Alan Bourne
I think the challenge fundamentally is actually about the format of questions and therefore how sort of vulnerable they are to this process. There’s nothing wildly advanced about how people are using AI here. You basically copy and pasting or taking an image. So it’s very straightforward. There’s a definitive way of handling that, which is which people call proctoring, where you’re videoed or monitored, what you’re doing on your computer while you’re doing an assessment. In that there’s obviously some mileage in that to try and defend the process when it’s happening. And there’s also the idea you might use AI to detect forensically when people are doing this, but I think it’s not clear that’s super effective yet. So there’s sort of the defensive bit that’s fine. But that still leaves us with the issue I mentioned earlier of assessment still not being particularly exciting.
10:15
Alan Bourne
So the opportunity you’ve got a bit more proactively is can we use AI to look at redoing how we do assessment in the first place, you know, can we make it much more immersive and engaging, maybe a lot more simulation based so that the actual experience is a kind of real time experience of doing the task or speaking to a customer or, you know, communicating at a Border and whatever it might be and where it’s much more immersive and you can’t really in real time get an AI to do that for you. You just have to be in the moment and do it. And so I think there’s a real win there if we, instead of being defensive, treat it a bit more positively.
10:53
Matt Alder
And I suppose that comes back to your earlier point about science and technology and everything like that, because we’re seeing a lot of AI recruiting solutions coming into the market that potentially also offer sort of degrees of assessment and those kind of things. And then as you say, some of the more traditional assessments are sort of very question based and sort of very dull. What’s the right balance between science and technology in this kind of just incredibly disruptive time?
11:22
Alan Bourne
Well, I think considered a very good question. I think that the fundamental bit there is it’s not an either or. Which actually then makes it, you know, an interesting challenge to solve. So yes, you want the science piece to be clear, particularly around precision and predicting who’s a good hire. And secondly also importantly, fairness in terms of how that’s done. So that’s the measurable part of is it working? That’s absolutely essential and that should be table stake. So anything that doesn’t demonstrate that, don’t touch it. But then alongside that, clearly having a, a seamless in a engaging candidate experience is really important. I think there’s lots that have been talked about candidate experience for the last 10, 20 years. Maybe not quite so much has actually happened.
12:03
Alan Bourne
So the sweet spot is how do we solve for getting both of those things right at the same time. And if you just have wizard candidate experience but make terrible hires, clearly that’s not a great idea. Equally, we don’t want to sort of be stuck in the past in how we’re doing it. So. So that’s the challenge really. You need to have both.
12:21
Matt Alder
Yeah, no, 100%. And I think that, you know, you’re kind of warning there about some of the technology that’s out there, that’s reinventing recruitment, but not any attention to the actual science of this is kind of really important because I think that’s a trap that lots of people could potentially fall into.
12:37
Alan Bourne
Yeah, no, exactly right. And I think it’s this is the case in all areas where our AI is, you know, appearing. But if you think of many other areas, like healthcare is a great example, clearly you want to know that any AI that’s being used for diagnostics is really good at doing the diagnostics and doesn’t get it wrong. Right. You know, hiring is not as life and death than that but. But you still need to get it right and there’s legal reasons you need to get it right as well. So it’s incumbent on employers to sort of buy solutions or build solutions that meet that criteria. I think.
13:15
Matt Alder
One of the other big areas that I’m seeing assessment talked about is within the whole kind of the move towards skill based thinking, skill based organizations, skills based hiring, all those kind of things. Do you think employers have got the right approach to that? Are they getting it right? How do you kind of see that kind of part market?
13:36
Alan Bourne
But some of the conversations that I’ve had with employers around this I think show that they’re very willing to want to make a go of skills based hiring, skills based organizations and they want to do that. Sometimes they feel like they’ve been told to do it from higher up in the organization, which isn’t always the best place to start. But I think most people buy into the underlying philosophy of let’s look objectively at people’s skills and then use that to work out what roles they can do and make the best of their potential and all of that. I think the concept bit feels like there’s quite a lot of engagement, but it’s one of those pieces where we sort of sold the concept too early without actually the industry having got enough solutions to deliver it.
14:17
Alan Bourne
It’s my sense and there’s still a lot of people going well, we want to be skills based. How do we actually do it? And you end up with a lot of firms have competency frameworks of behavior, frameworks of one kind or another which are very broad. And skills needs to be much more discrete so that all needs a rework. And also they’ve got knowledge based, you know, hard skills and then behavioral skills. And you kind of need to bring all of this together coherently if you’re then going to kind of create the talent intelligence approach that I think most people are aspiring to. So that, yeah, I sense there’s quite a big gap, execution gap between buying into the concept but not quite knowing how to do it.
14:54
Alan Bourne
And I’m not sure there’s what, you know, there’s not yet a sort of silver bullet solution to that normal may there ever be. So we’ve got to do quite a lot more work to help.
15:03
Matt Alder
I think and I think part of that as well is people kind of understanding perhaps what is even meant by skills in the first place because I’ve seen a lot of dialogue around skills based hiring that kind of reduces it to those kind of hard skills. It’s like, you know, we’re going to ignore where someone went to university and where they’ve worked before. We’re going to look at these very kind of short term hard skills they’ve got. That’s not really the right way to think about it, is it?
15:28
Alan Bourne
No, I think there’s a real risk there that you kind of replace experience based CVs with experience based ways of looking at skills, which doesn’t really solve the problem. I think you want to measure what people can actually do, not what they say they can do or where they’ve been before. All those kind of things. Right. So. And I think that’s possibly the sort of gap, there’s a bit of a trap there that happens where you end up just sort of defaulting to what feels easy. And that isn’t going to make life wildly better than it is at the moment. Yeah, there’s lots of issues with bias and poor definition of requirements now, but you’re not going to improve it if you know, just go down the route you’re describing.
16:05
Alan Bourne
I think we’ve got to stand back a wee bit and go, right, what’s a more holistic view of the behaviors, skills, kind of, you know, power skills as they’re called, underlying enablers that make people perform well in the end have frameworks of all of those things in. I think that’s essential to make it work. And then if you don’t have technology to then sort of see that data and have assessment data that proves that people have the skills rather than they just say they’ve got them. You know, you’ve got to solve all of those problems to make this work, I think, unfortunately.
16:38
Matt Alder
Yeah, no, I think that’s a big point there because I’ve seen a lot of kind of activity in this area that’s using kind of self reporting to try and assess the skills in the business. And I remember talking to one organizer, a number of organizations actually when I was doing some research around this last year, you know, and they were saying people were just sort of writing in people skills generically as the most popular skill that they had without, without kind of breaking that down any further in.
17:03
Alan Bourne
There is, you know, there are sort of fairly predictable demographic differences, you know, from a sort of gender perspective and in, you know, socioeconomic background around the extent to which people articulate what they can do and either inflate it or not. If you rely on self report on skills, it’s not any better than self report on anything else. Right. You know, you’ll end up with the same people saying they’re brilliant and that doesn’t really give you objective data.
17:25
Matt Alder
So tie this all together for us. What does the future look like? What do you think the world of assessment and recruiting and TA might be like in a few years time when the technology and we can see it coming has kind of sort of finally caught up with where we’d like it to be?
17:42
Alan Bourne
Yes. Well, I think my take would be that if you think about the different technologies that would be relevant to really improving in this area. When people talk about AI, they often just thinking about language models and actually AI is quite a lot broader. So you’ve got, you know, live Personas that, you know, bring characters to life and you’ve got obviously all the sort of voice in and out capabilities and all of the different things here, as well as analytical usage of AI. So there’s a few different strands to it. But if you, when you bring that together, I think the actual technology is just about mature now to really change how this, how assessment looks and works. So I don’t think the problem now is the technology.
18:21
Alan Bourne
I think the problem is our, you know, kind of capacity to grab hold of it and use it and start innovating a bit more boldly than we have done. So, you know, there’s lots of work to be done. On the science side, of course, you know, as I was inferring earlier, but I think the tech isn’t now the barrier, it’s now with us to grab hold of it and move it forward. And I think that in terms of what that looks like, I really see creating more task based simulations of in some of that might be sort of practical tasks, but also interactive tasks with different Personas of, you know, dealing with a customer or, you know, managing a project team or whatever, the particular scenarios are relevant to the job.
19:02
Alan Bourne
I think the capacity to do that, you know, is going to be the next opportunity for the industry and it’s just a case of getting on with it now 100%.
19:10
Matt Alder
Alan, thank you very much for talking to me.
19:13
Alan Bourne
Pleasure. Thank you very much.
19:16
Matt Alder
My thanks to Alan. You can follow this podcast on Apple Podcasts on Spotify or wherever you listen to your podcasts. You can search all the past episodes at recruitingfuture.com on that site. You can also subscribe to our weekly newsletter, Recruiting Future Feast and get the inside track on everything that’s coming up on the show. Thanks very much for listening. I’ll be back next time, and I hope you’ll join me.
