Some of the most interesting and effective technological innovation I have seen this year has been in the area of assessment. Whilst many of the new approaches aren’t yet widespread, I think they point towards a coming fundamental shift in the way we think about this part of the recruiting process.
My guest this week is Dr Nathan Mondragon, Chief IO Psychologist at Hirevue. Hirevue have been doing some fascinating work within the science of assessment and its ability, via data and technology, to accurately predict performance.
In the interview we discuss:
• How technology and science are challenging the established norms of assessment
• The hidden data in words, inflection, pauses and facial micro expressions
• How video analytics can be combined with subsequent job performance data to develop predictive hiring models.
• Using science to understand the art of recruiting and the changing role of humans within the recruitment process
Nathan talks about the types of tangible ROI that employers in a number of different industries have gained from using this approach and he also shares his thoughts on what we can expect to happen in the future.
Subscribe to this podcast in iTunes
Transcript:
Matt Alder [00:00:00]:
Support for this podcast comes from HireVue. HireVue’s team acceleration software combines digital video with deep learning analytics to help companies build and coach the world’s best teams. Team Acceleration Software is a modern digital answer to antiquated recruiting and training software that has placed barriers and bias in the way of finding, selecting and coaching a company’s most important asset, its people. Visit hirevue.com that’s spelt H I R E V U E to learn how organizations like Vodafone, Unilever, Nike, Red Bull, IBM and JP Morgan Chase are modernizing.
Matt Alder [00:00:47]:
The way they work.
Matt Alder [00:01:05]:
Hi everyone, this is Matt Alder. Welcome to episode 74 of the Recruiting Future podcast. Some of the most interesting and effective technological innovation I’ve seen in our space this year has been in the area of assessment. Whilst many of the new approaches aren’t widespread yet, they point towards a future fundamental shift in the way we think about this part of the recruiting process. My guest this week is Dr. Nathan Mondragon, Chief IO Psychologist at HireVue. HireVue have been doing some fascinating things within the science of assessment and its ability via data and technology to accurately predict performance. This is an interview that everyone involved in recruiting should most definitely listen to. Hi Nathan and welcome to the podcast.
Nathan Mondragon [00:01:59]:
Thank you, Matt. Good morning. Morning here and afternoon for you.
Matt Alder [00:02:03]:
Absolutely. Yeah, almost, I was going to say almost evening, it’s mid afternoon. Could you introduce yourself and tell everyone basically who you are and what you do?
Nathan Mondragon [00:02:15]:
Sure. So my name is Nathan Mondragon, I’m by title, I’m the chief industrial and organizational psychologist at HireVue, what folks refer to in EMEA as an occupational psychologist. And at Hirevue, I’m responsible for working with the team of biopsychologists and a team of data scientists and our sales and consulting folks to package and sell and deliver what’s called our digital assessment solution, which we can talk about a little bit later.
Matt Alder [00:02:49]:
Cool. And could you sort of give us a little bit of. Fill us in a little bit more on your background and what you’ve done sort of before you came to Highview?
Nathan Mondragon [00:02:59]:
Yeah, so my background, the majority of my work experience has been in combining assessments in a technology delivery. So all the way back to the mid-1990s, working with, starting to work with different consulting firms of industrial organizational psychology consulting firms, I was always blending technology deliveries of assessment solutions. So in 96 I actually part of a team that delivered the very first online test for hiring purposes ever implemented, that was in conjunction with Coopers and Lyran, and ever since then have been involved in that sort of background. Spent about five years with different consulting firms like shl, ddi, aon, et cetera, in doing those or helping to webify their delivery tools. And then kind of realized in the early 2000s that the IO psychology firms were excellent at the science and the research side of things, but they weren’t so good at the technology side. So I wanted to go to the technology type companies and sort of apply my trade within an enterprise software company essentially. So I joined RecruitSoft, became Teleo and then acquired by Oracle 3, four years ago. And I built an assessment practice directly within the Taleo recruiting applicant tracking system. And that had great success for about 13, 14 years before. Before jumping ship and journey and joining HireVue.
Matt Alder [00:04:34]:
I think that’s interesting because what I’m certainly noticing at the moment is technology really seems to be driving assessment into some, some, some different and interesting places. And I presume that’s sort of what, what you guys are doing, what you guys are doing at HireVue. How is assessment evolving or how is technology evolving assessment? And you know, what are you kind of working on in that sort of cross section at the moment?
Nathan Mondragon [00:05:06]:
Yeah, it absolutely is. And that’s really the reason I joined HireVue a little over a year ago is because what they were doing from an advanced technology and an advanced science perspective was second to none. Nobody is. People are trying it, but nobody’s really being overly successful with it. And HireVue has figured it out with their video data. So what they were doing really kind of captured my interest beyond just doing interview content within a platform kind of thing. So what HireVue at its core is a video interview or a video type platform that’s used for collecting applicants, video interview responses or doing live interviews, kind of like we’re doing over the Skype here kind of thing. But they also have an add on platform that does coaching type applications. So once somebody gets hired, then they can continue to use that video platform to go through training and development and continue to coach themselves. So that basis of a platform of capturing video data essentially from candidates or employees is the core product. And about two years ago, Mark Newman, the founder of HireVue, brought in some what they call quants, you know, the data science folks, and they were doing quantitative type analysis with mutual funds and physics background, etc. They brought a couple of those folks in and they Started to look at the video data and said, you know what, we can actually apply what’s called machine learning or deep learning to this video data to look at kind of what do people say, the words they use, what their voice sounds like, what their facial and micro expressions look like as they speak, etc. And start to extract really good data out of those video captures. So they started to look at that and research it and started to find that the data they were extracting from those video files actually had a relationship to people who ultimately got hired or people who performed better when they got started the job, that sort of stuff. And so that background is what gained my interest because that’s some pretty leading edge science, that background of that, those video analytics that they were doing gained my interest in the main reason I joined, I joined HireVue, you know, 14 months ago kind of thing.
Matt Alder [00:07:35]:
So to dig into what you sort of just described there a little bit more detail before we sort of talk about how it works in practice. You sort of mentioned the words that people use, but you also mentioned the tone that they use and micro expressions. Could you, could you sort of, could you explain that a little bit more in terms of what that might tell us about someone, what the sort of data points are there?
Nathan Mondragon [00:08:04]:
Yeah, sure. So there’s different indicators in the data set. I mean, it’s what they call features. So when you think of all the words people use, the types of words, positive wording, negative wording, nouns, verbs, pronoun, you know, adjectives, adverbs, etc. Those are all different elements. And then the audio file, the voice inflection, the monotone sound of the voice, the inflections in the voice, the amounts of white space in the audio file, or the use of the ums and the ahs and the ok’s and the pauses in the speech kind of thing, have certain data elements linked to them. And then the facial movement has indications as well. So smiling, frowning, eyebrow movement, eye contact, up and down with the camera has different emotional type categories attached to it. And then when you put those three together, that can be pretty telling as well. So, you know, Matt, if I asked you, do you like your boss? And your response was, I love my boss, on the wording, on the text alone, that’s pretty positive wording. So it sounds pretty good. But if your answer was I love my boss and you roll your eyes and look away from the camera and your voice drops at the same time, that has a different meaning to the answer than the, you know, than the positive text Alone kind of thing. So it’s, it’s really those three sets of data laid on top of each other that can be very telling in, you know, in an interview analysis, essentially. So it’s almost like the way I kind of parallel it is if you were doing a live interview with somebody and they were sitting across a table from you, you would be reading, engaging their body style, their language, their reaction to your question, they’d be doing the same to you kind of thing. And it’s been able to capture a lot of that analysis that you run through in your head that never gets down to the paper, essentially, that doesn’t get into your notes and doesn’t get into your ratings of how good or bad you think this person is in the interview and subsequently would be to work for you. That’s the type of stuff that we’re capturing that we’re identifying in these data elements with the video analytics, plus other competency and personality type things as well.
Matt Alder [00:10:25]:
Now I suppose to dig into a bit more about how you’re sort of using this in practice, obviously this data, as you say, has a meaning in its own right. Are you then cross referencing that with other data, with performance data to actually bring like a proper contextual meaning to it? How does the, how does the sort of product develop from that point?
Nathan Mondragon [00:10:51]:
Yeah, and that’s, and that’s kind of what I was brought in to help do. So that’s exactly what we’re doing is that’s really where the golden nugget really kind of comes in here. But let me cut it back up and sort of say most of the customers when they start working with us, they have about a 4, 5, 8 week hiring process from the time an applicant applies for a job to the time they get that job offer, let’s call it six weeks. And the reason it’s six weeks is because there’s multiple steps and multiple time and scheduling things that have to take place between each of those steps. What we’re doing and what we’re able to do now with this video analytics is consolidate a lot of those steps. So a lot of our customers are getting like a five day, six day, seven day, eight day from the time an applicant applies to the time they get the higher offer. Now because of the consolidation with the video analytics it allows us. So what we’re doing is we’re taking, let’s say on average about six questions are in a customer’s video interview for pick your job group. High volume entry level hourly jobs like cashiers or college grad hiring or salespeople, whatever it might be. There’s about six, seven, eight questions in their video interview. It takes two to three minutes of response time per interview question. So we’re getting let’s say 15 minutes of recorded video data. We’ve already talked about kind of how we analyze that video data that gives us what we call our predictor set or our input set, you know, kind of thing. Then we want to link that up to which is what you were leading to the outcome data. So who performs better, who gets the job, who turns over more frequently, who sells more, who gets promoted, you know, whatever the business metric might be that’s of interest to that customer for that job group, that’s what will that business data metric data will collect, link it up to the video data and then run a whole bunch of analysis with the video data and the outcome data to identify which of those data video features actually link to that job performance data. And we’ll build an algorithm scoring engine on the back end basically to generate a result for that customer. For any candidate that then comes through and takes that video interview, that score gets generated and that score is statistically linked to the business metric data that it was built upon like job performance or turnover or something like that. And that helps guide them in then who they should bring in for the live interview or the next step of the process, whatever that might be.
Matt Alder [00:13:34]:
I mean, it’s absolutely fascinating stuff, you know, just unbelievable in some ways from, you know, what was, you know, what technology’s made possible. I mean, do you have, are you seeing, you know, tangible results from using this kind of process? Are clients getting the kind of outcomes that they expect?
Nathan Mondragon [00:14:00]:
Oh yeah, yeah. Huge ROI so far. So, you know, anywhere from turnover reduction to sales improvement, job performance rating improvements, longer 10 year safety violation reductions, you name it. You know, we’re doing some studies now with a couple of other retailers around loss prevention or what they call shrinkage and theft, you know, kind of to identify the features in the videos that would identify people that would have a tendency to steal essentially. So we’ll probably have some loss prevention reduction type results here in the next three, four months and others. We have a study underway with healthcare companies right now for nurses. So looking at how do we identify the nurses that are going to be. It’s a unique combination for healthcare, for nurses where they need to have that technical knowledge of medical ability and that kind of stuff combined with the patient care of servicing and taking care of patients and the empathy and all that kind of stuff along with Nurses are being asked to manage teams. They’re managing the nursing assistants and the other nurse crews and that kind of stuff. So it’s a unique combination that is needed in a pretty unique professional job. So we’re doing a bunch of studies with some healthcare companies right now to identify a nursing model essentially as well. So that’s probably about three or four months away as well. So we’re finding a lot of ROIs. A lot of companies are putting dollar values to these returns. One attached a $2.8 million savings in deferred cost essentially. So they had a. We found 420 days longer tenure for the people that scored well on the digital assessment versus those that scored poorly. That 420 day longer tenure equated out to a number of hires that they didn’t have to make, basically. And those hires would have cost 28,000 per hire and times that times 100 hires for the year, which would have been average that they would have had to have made. It’s a $2.8 million savings right there off the bat. So. So results like that we’re finding quite frequently with, with our model builds.
Matt Alder [00:16:23]:
Now what’s fascinating about this for me is it’s very often people will talk about the, the art of recruiting and the science of recruiting. And you know, what I’m seeing from the kind of march of technology in the assessment area is that art of recruiting is sort of being deconstructed into science and, and automated. That’s an aspect of what you guys are doing very, very successfully. Where is the place for the human recruiter in all of this? Are they being replaced by the algorithm or is their role changing?
Nathan Mondragon [00:17:03]:
Yeah, it’s a little bit of both, to be honest with you. There is, I think the common term that people are throwing around today is called recruiterless recruiting. Maybe we’ll see that someday. I don’t. But what we’re finding, our customers and some of our very large customers, some very large banks in the world they know, et cetera, they’re repurposing and reallocating their recruiter base. So it’s not necessarily getting rid of individuals or the FTE count, but it’s really the recruiters typically are responsible for finding people, pushing them into the funnel or the process, working them through each step of the process and then trying to close them at the end and get the hire out of it, and then kicking them into the employee onboarding piece. That’s a lot of tasking and administrative type work that recruiters are responsible for. What we’re finding our customers are doing is reallocating and repurposing those recruiters to the top of the funnel and the bottom of the funnel. So at the top they’re saying go out and source and look for talent and find places to get talent and get them interested and get them into the pool, essentially kind of thing. So there’s good recruiters that are good at that sort of task and that sort, sort of a strategic move, if you will, of getting recruiting and filling the funnel. Then let the sort of algorithms, if you will, run through and look at the resume and generate scores to that look at the video interview and do the digital assessment approach and generate scores on that and let the computer essentially kick out to the recruiter the top individuals that link to future job performance or future success and then let that recruiter now spend their valuable time on the high touch cost component. So let the computer do the technology part where that menial administrative task and stuff that recruiters don’t like doing anyway and really shouldn’t be doing. Let the recruiters now deal with the high touch and actually closing and selling those highly qualified candidates to convince them to join, you know, company X, Y or Z kind of thing. So work at the top of the funnel and then work at the bottom of the funnel in selling and closing those highly screened and highly qualified candidates. That’s what a lot of our customers are doing, is repurposing of the recruiter.
Matt Alder [00:19:26]:
Yeah, and I can, I can completely see the logic in, in doing that. That, that makes a lot of, a lot of sense. Now, sort of final, sort of final question or area for discussion. Now obviously what, what you guys are doing, you know, is arguably kind of on the cutting edge of, you know, of assessment and recruiting. Obviously, you know, you’re getting the roi, the clients are, the clients are doing it. So it’s, it’s establishing itself. What’s next? What’s on your radar? Where might technology take us next? What’s kind of on the horizon that you can see that might change things even further?
Nathan Mondragon [00:20:08]:
Yeah, I think one of the main things is we kind of touched on this a little bit around expanding the algorithms into the other pre screening areas like the resume. Right. So we’ve been researching recently, we pulled a database of 10,500 candidates that had the resume, had the video interview and we knew their job performance once they got hired. And we looked at through our machine learning and deep learning that, you know, that precursor to the artificial intelligence type work, we’re able to identify area features on the resume and generate a resume score, generate the digital assessment of the video interview score, linked those up to job performance and found that there was what we call incremental validity. There’s each of them added to the prediction of the job performance, but each of them together resume plus the digital assessment of the video interview was more powerful together than separate. So that’s going to be one proof point, I think. And other companies are kind of researching that resume prediction now. We’ve already kind of got it and now we’ll sort of productize it a little bit. That’s one area, I think, to continue that middle funnel, middle pre screening ground that we just talked about and sort of automating that with the machine learning. The second is once these people become employees, what are we doing with them? And how can we take the rich data that we gathered on them from maybe their resume score and their digital assessment score and maybe their other traditional assessment scores that were in the evaluation process? How do we take that rich information and then apply it to developmental and coaching activities once they get hired and apply that within a video platform again? And that’s, again, this is, you know, in Hirevue’s core is the, what we call our coaching video, our coaching video platform, which is. Does just that. But we’ll take the data that the candidate generated and the scores we generated on them when they were an applicant and apply that to their coaching profile so that they can then kick off some developmental activities and that sort of stuff once they become an employee, all within a video interview platform. So I think that the short answer is video will be the centerpiece of it and it’ll be taking the applicant video analytics and applying that to video developmental planning, if you will, for the person that wants to become an employee.
Matt Alder [00:22:37]:
Nathan, thank you very much for talking to me, man.
Nathan Mondragon [00:22:40]:
I appreciate it. I appreciate the time. And this has been, this has been a joy.
Matt Alder [00:22:45]:
My thanks to Nathan Mondragon. You can subscribe to this podcast in itunes, Stitcher, or via your podcasting app of choice, just search for recruiting future. You can find all the past episodes@www.rfpodcast.com on that site. You can also subscribe to the mailing list and find out more about working with me. Thanks very much for listening. I’ll be back next week and I hope you’ll join me.
