AI is a game changer for talent acquisition, and as we start to move from the hype phase to the doing phase, its potential capabilities and current limitations are becoming more apparent.
So what can and can’t AI do for talent acquisition right now, and how will things develop in the future?
My guest this week is Mona Khalil, Senior Manager Data Science at Greenhouse. As well as building AI into their products, Greenhouse has done extensive research with both TA leaders and job seekers to understand the potential positive and negative impacts of generative AI on recruiting.
In the interview, we discuss:
• Current talent market challenges
• Employer uptake of generative AI
• Reducing bias versus codifying bias
• Levels of nuance in decision-making
• Parsing and normalizing candidate data
• Are employers monitoring AI effectively?
• Asking vendors the right questions about their AI integrations
• How do candidates feel about AI?
• What does the future look like
Listen to this podcast on Apple Podcasts.
Transcript:
Matt: Support for this podcast comes from Greenhouse. In today’s competitive hiring landscape, taking a people first approach is business first. That’s why Greenhouse helps companies adopt a flexible, fair, and efficient approach to hiring. Greenhouse empowers everyone from recruiters to hiring managers to make confident decisions that help strengthen your business so you can get measurably better at hiring. Discover how Greenhouse can help you hire for the kind of business you want to build. Learn more at greenhouse.com/hire. That’s greenhouse.com/hire.
[Recruiting Future Podcast theme]
There’s been more of scientific discovery, more of technical advancement, and material progress in your lifetime and mine than in all the ages of history.
Matt: Hi, there. This is Matt Alder. Welcome to Episode 572 of the Recruiting Future podcast. AI is a game changer for talent acquisition and as we start to move from the hype phase to the doing phase, its potential capabilities and current limitations are becoming more apparent. So, what can and can’t AI do for talent acquisition right now? And how will things develop in the future? My guest this week is Mona Khalil, Senior Manager, Data Science at Greenhouse. As well as building AI into their products, Greenhouse has done extensive research with both TA leaders and job seekers to understand the potential positive and negative impacts of generative AI on recruiting.
Matt: Hi, Mona and welcome to the podcast.
Mona: Hi, thanks for having me.
Matt: An absolute pleasure to have you on the show. Please, could you introduce yourself and tell us what you do?
Mona: Yeah, absolutely. My name is Mona Khalil. I’m the senior manager of data science here at Greenhouse Software. So, at our core, we’re an applicant tracking system, but we offer a full suite of tools to enable you to recruit effectively. Our mission as a company is to make you great at hiring.
Matt: Fantastic stuff. So obviously you work with lots of customers around the world. It’s been an interesting year [chuckles] in many perspectives when it comes to talent and hiring. What are the main challenges you’re seeing your customers having with talent markets at the moment?
Mona: Yeah. So, I think one of the things that we’ve been or I think the first thing that comes to mind is definitely just reduced capacity to be able to do the work that’s necessary to recruit effectively. I know a lot of teams are stretched thin, a lot of recruiting, and really many other teams are just based on the economy, looking at ways to become more efficient, to do more with less. We’re also seeing much higher volumes of hiring. So just needing to be able to sort through, evaluate and communicate with much larger volumes of candidates as well as figuring out, I think, which is the big topic here we’re going to talk about is how to use new and emerging technology like AI to make your recruiting processes more efficient.
Matt: Yeah, absolutely. And one of the things I think we’ll all remember about 2023 is just the amount of time we spent talking about generative AI. It really has been the big narrative. What kind of uptake is there with generative AI for the employers that you’re seeing? I know everyone’s talking about it, but how much are actually people using it as part of their strategies?
Mona: I don’t think I can give a specific number on who’s using it, but I don’t know anyone who isn’t interested in figuring out how to effectively incorporate it into their process. Actually, let me think, we actually performed a survey, like a candidate experience survey, and we found something like a third of HR leaders are already using it. Another half of them are considering using it. And I think even the remaining percentage might be thinking about how to incorporate it into other processes. There’s just so much that can be done with generative AI.
Matt: Absolutely. And its probably important to kind of zero in on some specific use cases so we can kind of talk about them. So, one of the things there’s been a lot of conversation about is how AI could perhaps help to improve DE&I, is that something that you’re seeing at the moment?
Mona: I’ve been seeing that as a very big question. I think so far, we haven’t seen evidence that it can improve the DE&I directly. So, I think, for example, if you’re using some type of AI to evaluate a candidate, to write a message to a candidate, those are things– I definitely wouldn’t recommend the former. There’s a lot of research coming out showing that it’s quite easy to make any type of, like a generative AI or a large language model produce pretty biased results. I don’t think we’ll know for sure exactly how the dust settles on that for some time. I think it can be a great tool if you’re prompting, for example, ChatGPT to evaluate a job description, see if there are ways in which you can maybe make it more inclusive. You can have it review messages you’re sending to candidates. There are a lot of ways in which you can prompt it to actually to create messaging, language, questions, and content to be more inclusive. But I wouldn’t consider it inherently a solution to trying to create a more inclusive hiring process.
Matt: And I suppose to dig into some of the ways that it could be used potentially in conjunction with humans to reduce bias. You kind of mentioned sort of text analysis there and things like that. What other things can be done or what do you see happening with AI to kind of help reduce that the very obvious bias that we see in hiring processes.
Mona: I think AI is very good at reporting and synthesizing data as well as large amounts of text. So, I think anybody in this field, like we know on our end the data science team, that recruiting involves so much text and information synthesis about an individual person and then doing that at scale to hire people for dozens upon dozens of roles. You’re evaluating every scorecard, the response to every interview question, every interviewer’s evaluation of a candidate. And we know that generative AI is great for synthesizing that information, maybe into a more brief summary. We also know it’s great at synthesizing data. So, for example, like helping you run reports, recommend chart types, recommend interpretations of your data, basically helping you synthesize the information that you need to make those human decisions more effective.
Matt: And I suppose picking up on that human decision bit, there’s a lot of debate about the future role of recruiters in this. In terms of whether they’ll be replaced by AI. What do you think is kind of happening there in terms of humans still making decisions or AI making decisions?
Mona: That’s a great question and I’ve been hearing that across many, many fields. I don’t see recruiters being replaced by AI anytime soon. Most of the models that I’ve seen, there have been a lot of test models over the years around whether or not some kind of AI can effectively rank candidates. None of them have been promising and none of them have been shown to make fairer decisions on aggregate than humans. And I think the other thing to remember is that in an AI model, once it’s developed and trained based on previous human decisions, its potentially codifying whatever bias existed in those decisions. And then you can’t put the AI through DE&I training to actually make its decisions better. It’s actually very hard at that point to improve upon it if you don’t have new and better human decision data to train it on.
And then finally, I would say every company is different. The culture, the needs of your individual company differ from the next. And that’s not something that an AI model can very easily capture. So, the decisions of a recruiter, I think, are always going to be important to understand the many, many levels of nuance that are necessary to make those effective decisions.
Matt: You talked about all of the amount of text and data that’s around the recruiting process. When it comes to resumes and resume parsing and those kinds of things, I mean, how is AI helping with that process? Because it’s always seemed to me that we’re still using a very old-fashioned text-based process of people sort of talking about what they do and what they can do. How is AI helping there? I suppose particularly also from a bias perspective.
Mona: That’s also a great question. So, I’m going to start with the parsing part of things. I think anybody in recruiting [unintelligible 00:09:16] that resume parsers tend to be, at best, not very accurate. Actually, one of the things that we’ve done is we have a resume parser that we’ve integrated into Greenhouse that’s actually probably one of the best out there as far as accuracy by a large margin. And it uses a lot of this new technology like GPT-3. So, the first step of just accurately getting the information where you need it in a structured manner is something that we’ve prioritized as far as leveraging this new tech. And then the other thing is going to be actually standardizing and potentially normalizing that information.
There are so many different ways in which a person can convey that they have a skill. And I know that in recruiting, for example, your job is to know recruiting as well as a lot about the individual roles that you’re hiring for. So potentially what an AI model or what AI can actually do is synthesize the similarity of the different languages, potentially surface skills or find skills that are described in different types of languages and identify them as something similar or the same thing. So that you can accurately and reliably see that if a candidate describes a technical skill using language that maybe you’re not familiar with, that they can still be identified as having that skill.
Matt: You talked about the issues with AI effectively making selection and hiring decisions. There’s a huge amount going on in terms of regulations. There’re regulations in the US. There was a presidential executive order a couple of days ago. The EU is very close to implementing its kind of AI act. There’re lots of things going on and lots of things that people need to look out for. Do you think that employers are effectively monitoring the AI they’re using and understand what’s happening in their processes?
Mona: I would say not everybody because I think it’s often hard to tell exactly what technology is being used under the hood by the software that you purchase to help you work, manage and evaluate your candidates in the process. I think from personal experience, I’ve seen some folks who have a good understanding as far as monitoring the tools, but on some level it’s fair that you would expect the tools to monitor themselves. So, I know that with the New York City law, for example, that went into effect this year, it’s been quite challenging for a lot of employers that I’ve seen to figure out how they’re going to get the third-party audits, who they’re going to do the audits from. I could see a lot of folks being quite surprised, I think, by the end of the year as to what the outcomes are as far as the impact of different tech that’s being used.
Matt: Is that down to because as you say, a lot of the AI that’s being used in the process is coming from the vendors that they’re using. Is that down to asking the right questions of the vendors?
Mona: A lot of times it is. I would say that’s a very big part of it, asking whether or not they do bias audits of the tools that they create, what guardrails and practices they’ve put in place to make sure that they’re not producing disparate outcomes. So, for example, maybe over selecting from candidates from one demographic group and under selecting from another, maybe at the intersection of these groups. It’s important to ask, what did they do to– what guardrails did they put in place when they developed their tool in the first place? And then what do they have in place to monitor it? Because those decisions can drift over time. A model can do what’s called drifting, like whatever tool that you’re using will need to be continually monitored. So, I would say ask, what did they put in place in the first place, what tests that they do, and how do they monitor it to make sure that decisions remain fair and equitable?
Matt: We’ve talked a lot about employers. We’ve talked about the process, we’ve talked about legislation and monitoring. What about the candidates? I mean, what do the candidates feel about AI?
Mona: From our survey, about a third of candidates are actually worried about being rejected by AI. I would say that that’s a pretty big bias and fairness concern among folks. And I think as a candidate, if you don’t have the insight into the tools that the companies are using, it’s often hard to tell. But that is definitely a prevalent concern among the candidates that we spoke to in our survey.
Matt: What would your advice be to the TA leaders listening in terms of their overall strategy around AI or how they fit AI into their talent acquisition strategy? What should people be aware of, be watching out for and taking action on?
Mona: I would say be very cautious of the tools that you use that impact whether or not you believe a candidate’s qualified. Make sure that you ask those questions of the vendors that you use to see whether like what due diligence they’re doing to make sure that they’re putting the best possible fair decisions in front of you. And overall, just really be cautious and think twice about whether these tools will enhance your recruiting processes or whether they could have any downstream impacts, especially with, like you said a lot of legislation that’s starting to come out. The executive order was definitely a surprise to us. And if you’re in a part of the world, like if you’re in the EU, you’re in the US. You could potentially have to produce additional audits, which could be a cost to your company if you have tools that directly impact the decisions you make about who’s qualified for a job or not.
Matt: Final question. Where’s all this taking us? What do you think the future of talent acquisition is going to look like? Where might we be in three to five years’ time when it comes to AI?
Mona: That’s a great question. I think with any hope and I think this is something that actually my team is very much looking into. We want to reduce the mundane work associated with recruiting and really empower recruiters to be able to make those decisions effectively. So, to spend less time doing repetitive tasks, tasks that you know with the assistance of AI could actually enable you to make thoughtful and intentional decisions about who to hire as much as possible, giving you the space to audit, say, the questions that you ask of candidates continually improve those processes. Really, my hope is that AI will be an assistant in the process, so integrated into tools in very intentional ways that are designed to not have a negative impact on people’s employment, but ultimately to aid them to be more effective in their jobs.
Matt: Mona, thank you very much for talking to me.
Mona: It’s been a pleasure.
Matt: My thanks to Mona. If you’re a fan of the Recruiting Future podcast, then you will absolutely love our monthly newsletter, Recruiting Future Feast. Not only does it give you the inside track on what’s coming up on the show, you can also find everything from book recommendations to insightful episodes from the archives, and get first access to new content that helps you to understand where our industry is heading. For a limited time, subscribe to the Recruiting Future Feast newsletter and get instant access to the video recording of the recent remixed webinar on AI and talent acquisition featuring some of the smartest thinkers in the industry. Just go to mattalder.me/webinar to sign up. That’s mattalder.me/webinar.
You can subscribe to this podcast on Apple Podcasts, on Spotify, or via your podcasting app of choice. You can find and search all the past episodes at recruitingfuture.com, and don’t forget to sign up for the newsletter, Recruiting Future Feast. Thanks very much for listening. I’ll be back next time, and I hope you’ll join me.
