With all the talk of AI and Programmatic Advertising currently going on in the Recruitment Marketing space, we can sometimes lose sight of the raw power of the data that sits behind these innovations in technology.
My guest this week is Matt Plummer, VP of Product at ZipRecruiter. It is likely that many of you, especially those outside of the US, may not have heard of ZipRecruiter but they are the fastest growing Jobs Marketplace in America. They have 70 thousand active clients and handle upwards of 20 million applications through their system every month.
That is a vast amount of data and, as you will hear in our conversation, the insights and innovations this data is driving are absolutely fascinating.
In the interview we discuss:
• Programmatic Advertising
• Building a Netflix style recommendation based matching engine and some of the data points used to deliver it
• Creating a “magical” experience for candidates and employers
• Unique metrics produced from a subscription model
• How machine learning can out-think humans when it comes to spotting job seeking patterns
Matt also shares his vision for the future and talks about the forthcoming TA Tech Europe conference in Dublin
Subscribe to this podcast in iTunes
Transcript:
Matt Alder [00:00:00]:
Just before we start the show this week, I wanted to ask a quick favour. I’m currently doing some research for a white paper that I’m writing on the future of career sites. So if you’re currently working in a recruiting or recruitment, marketing or employer branding capacity for an employer, I’d be really grateful if you could fill in my survey. There are only four questions and it shouldn’t take longer than a couple of minutes. I’ll also make sure you get an advanced copy of the white paper when it’s ready. The link for the survey is www.bit.ly CareerSiteSurvey. That’s www.bit.ly CareerSiteTurvey and CareerSite Survey is all in lowercase.
Matt Alder [00:01:06]:
Hi, this is Matt Alder. Welcome to episode 121 of the Recruiting Future podcast. With all the talk about AI and programmatic advertising in the recruitment marketing space, we can sometimes lose sight of the raw power of the data that sits behind these innovations in technology. My guest this week is Matt Plummer, VP of Product at ZipRecruiter. Now, many of you, particularly those outside the US, may not have heard of ZipRecruiter, but they’re America’s fastest growing job marketplace with 70,000 active clients and they handle upwards of 20 million applications in their system every single month. That’s a huge amount of data. And as you’ll hear in the interview, the insights and innovations this data is driving are absolutely fascinating. Hi, Matt, and welcome to the podcast.
Matt Plummer [00:02:03]:
Thank you, Matt, for having me. I’m pleased to be here.
Matt Alder [00:02:05]:
My absolute pleasure to have you on the show. Could you just introduce yourself and tell everyone what you do?
Matt Plummer [00:02:11]:
Absolutely. I am the VP of Product at ZipRecruiter. I’ve been with the company for a little over four years now. Prior to that, I was an advertising technology, so consumer advertising tech, building ad exchanges and ad servers and bringing a little bit of that back over to this industry.
Matt Alder [00:02:24]:
So ZipRecruiter is very well known in the US, but perhaps less well known outside. For the benefit of people who’ve not heard of you or people who don’t really have a clear idea about what you do, could you Talk us through ZipRecruiter?
Matt Plummer [00:02:36]:
Yeah, ZipRecruiter is the fastest growing online employment marketplace in the US. We are essentially leveraging technology, artificial intelligence, to connect millions of job seekers and employers to the right jobs and the right job candidates. So we’re trying to make those matches, get people to work and get businesses back to their business. Really.
Matt Alder [00:02:54]:
I think it’s a really interesting marketplace at the moment in terms of how technology is driving everything. I’m presuming that you guys are doing a lot in Programmatic advertising and artificial intelligence. Which technologies are you using to drive the growth of the company?
Matt Plummer [00:03:10]:
Yeah, actually all of the above. So Programmatic is kind of interesting from the buy side in particular. And I think we’ve seen the most Programmatic adoption by recruitment marketing agencies who are looking to better optimize how they spend their client budget. What’s interesting about Programmatic though is Programmatic ultimately is just decision making on where and sort of how to spend a budget. It really doesn’t actually affect the matching itself. So if you look at display advertising, Programmatic is kind of merged with what we would call real time bidding. And in that case the buyer is actually able to make a decision on who to show ads to. In our industry, everything is mostly search based or potentially sort of match based. In those cases, companies like ZipRecruiter, we’re doing the matching. So when an agency is trying to leverage Programmatic to manage a buy, there’s kind of a fallacy in that, in that we’re still doing all of the matching. So we’ve been working with agencies in the US in particular, some in the uk, to make sure that we’re taking Programmatic in the right direction and not kind of getting held up by some hyperbole and some exciting buzzwords. So that’s on programmatic on the AI front, ZipRecruiter in particular. We actually launched an R and D center in Tel Aviv, Israel in 2015. And we’ve got about 40 engineers there and they’re really heads down focused on our AI powered candidate matching technology. The path that we looked at was taking a look at the consumer space where you’ve got this transition from a lot of keyword based searching to AI driven recommendations or matching. And I’ll use Netflix as a pretty universal example probably. Well, 10 years ago we were probably still ordering DVDs off of Netflix, but then as they went online with streaming, there’s still a lot of searching. I’d have to go search or browse through content categories in a directory structure. Now for the most part, when you flip on Netflix on your Apple TV or whatever device you’re using, a lot of your content consumption is driven by recommendations. And I forget the exact stat, but it’s somewhere in the neighborhood of 3/4 of the of the content consumed on Netflix is driven by that matching. And so we took a cue from that and we looked at job search and a lot of the top searches in the job industry are sort of a blank search, which I would call like a shoulder shrug type of query. Somebody who’s basically looking for a job or feature based searches like part time work from home benefits. And so those aren’t really indicative of what the person’s skilled for or suited for. So we took the Netflix sort of approach and we spun that around and said, well, let’s move from keyword based searching to recommendations where we can actually take jobs as they’re posted and then recommend those to candidates in real time based on their actual likelihood of being rated positively by the employer.
Matt Alder [00:05:56]:
I think they’re both really interesting areas. Just a big. Just to dig a bit deeper, backing up to talk about programmatic again. There’s probably quite a few people listening to the show who are not fully up to speed with it. Could you just talk through how it works and why your approach is different?
Matt Plummer [00:06:14]:
Yeah, and I’ll use the agency example. I think it’s probably the most universal and probably the most common right now. So when an agency is trying to manage recruitment, marketing, budget for a client, they’re really trying to deliver the best value and ultimately the best roi. We’re a little behind in this industry and how we look at roi, but we’re getting there. So what it started out as was really rules based management. So once a job, for example, reached a certain application threshold, like 100 applications, stop sponsoring the job. So that allowed the budget to kind of be more allocated to the jobs that really needed it and didn’t sort of flood one job with a bunch of candidates and starve out the others. So that was really phase one of what people were referring to as programmatic. And then the next phase, which is kind of coming about now is more like budget allocation. So the agency wants to make sure that, let’s just use a $10,000 budget as an example. They want to make sure that $10,000 budget is spent with the publishers or the search marketplaces like us, where the efficiency is there so they can get volume, they can get quality, they can get a good, let’s call it a CPH or maybe even a cost per hire. And today or yesterday maybe the way that they do that is manual budget allocation. So like 20,000 or sorry, 2,000 of that, 10 would go to one publisher, another thousand would go to another, and then the agency Rep Would have to sit there and ensure that the spend was being the dollars were actually being spent because the agencies are incentivized to ensure that the dollars do get spent, but they’re getting spent efficiently and effectively. That’s a lot of sort of mid month or intra month management of those budgets. And so the modern or the current sort of step evolution in Programmatic is to automate more of that. So tracking all of the CPAs and ideally tracking down to a higher sort of quality candidate level and making budget allocations that allow the dollars to be spent where the publisher is delivering the most value to the client. So that’s kind of the current evolution of Programmatic and that’s where there’s some issues where the agency and their Programmatic tech can’t ultimately affect matching, but they can make some budget allocation recommendations. So there’s value in it. We just have to make sure that the Programmatic platforms and the publishers are all working together to make sure that it’s doing the best thing possible for the client.
Matt Alder [00:08:32]:
And in terms of using artificial intelligence for job matching, I think this gets talked about a lot, but it’s always interesting to see how it might be working in practice. From what you’re doing in your development center, how is it affecting the way that people search for jobs? What’s the candidate experience like and what are employers getting from it?
Matt Plummer [00:08:53]:
Yeah, we try to make the experience on both the employer side and the candidate side feel magical. The internal sort of code name for what we started doing, we actually referred to as Magic 5. It’s not a sort of product name, it’s just an internal code name. And the idea was we wanted to deliver five qualified candidates to the employer within the first 24 hours of a job being posted. So imagine this experience where, especially for a small business owner, you’re in the back office for your business, you post a job and almost before you turn out, turn out of your chair and get back to work, you’ve got your first couple of qualified candidates coming in. We wanted that real magical experience. We didn’t want to have some special process that the employer had to go through or that the candidate had to go through. So it’s all really behind the scenes, same thing on the candidate side. When the candidate is getting these job notifications, all they know is I’m getting these great recommendations. It’s really easy to apply to the job. They don’t understand that it’s coming from anything that looks at their aggregate behavior and recommends jobs based on that. Using machine learning they just know that we recommended something that seemed like a good fit to them.
Matt Alder [00:09:58]:
You mentioned the aggregate learning and the machine learning that make the match work. What kind of behaviors are you looking at to give that high quality match?
Matt Plummer [00:10:08]:
I can’t go into all the details, but some of the ones that would probably be relatively obvious if you just kind of sort of tear it apart a little bit. Jobs that you apply to, how those applications are rated by the employers. Right. That’s a big one. We want to make sure that our recommendations are, can be tested against something in real life. And that real life test is typically an employer evaluation of the actual candidate, job clicks, job searches. So anything that the user is doing, including whatever’s on their resume, if they’re using different resumes for different types of jobs, all of those things can be inputs into, into our system.
Matt Alder [00:10:44]:
Interesting. I can see how those data points would make the match work. What about on the client side? Is it a case of them just getting magical better candidates or do you think there’s an appreciation for what goes into the technology of matching?
Matt Plummer [00:10:59]:
Yeah, very much so. A couple of interesting data points that we’ve been able to now collect over the last couple of years. So a candidate who applies to a job through a recommendation that we make with our machine learning, they’re about twice as likely to move through the post application process, so sort of the initial screening and review process than a candidate who finds a job just through keyword based searching. So right there that tells us that we’re recommending much better jobs than they’re finding sort of organically and that the employers are appreciating that because they’re moving them through. So a downstream metric that we often look at because we are a subscription business primarily are subscription metrics. So you’ve got things like retention and reactivation. And when somebody does cancel their account, we survey them and to understand is it just a, they don’t have a hiring need anymore, is it that they’ve and they’ve successfully filled the position or were they unsatisfied with the service? And so that survey has been really useful and we can directly correlate some of our advances in our matching technology with employer satisfaction. And then that also then kind of backs out into retention and reactivation. So we’re seeing our own business metrics improve because of the improvements in our matching that’s led to our customers hiring people faster and putting butts in seats faster so they can kind of get back to their running their own business.
Matt Alder [00:12:25]:
I think what’s Very interesting and significant. There would be the volume that you’re dealing with, the volume of data that you’re dealing with. To get that level of intelligence for people who are not overly familiar with ZipRecruiter. Could you just give us a sense of the scale in terms of the number of clients you have, the number of jobs and the number of candidates you’re dealing with?
Matt Plummer [00:12:46]:
Yeah, we have about 70,000 active customers today, and on an average month we see anywhere between 15 and 20 million applications. And those are applications that are actually being captured in our system. There’s probably another 10 to 15 million that we’re responsible for that end up in third party ATSs. But out of those 15, 20 million applications, we collect a lot of data points. So it’s all, again, all the search, search inputs that we’ve got from people, all the clicks, all the applications, and then all the ratings of those applications. So really, to properly use machine learning and AI, you’ve got to have a lot of data to learn from or systems to learn from. So we’ve identified some interesting patterns that we would have never done, either with a smaller volume of data or even as humans ourselves.
Matt Alder [00:13:32]:
What’s that told you about job seeking in general? Is there anything that surprised you about the way that people are looking for jobs?
Matt Plummer [00:13:39]:
Yeah, so we actually identified some interesting patterns in the data. And this is machine learning identifying them, not us as humans. One example was this habit of people who are looking for product management jobs in New York were also looking for those same types of jobs in the San Francisco Bay area. And when we talk about that, it sort of intuitively could make sense, but, you know, would you have come up with that on your own? Unclear. But our data sort of identified these sort of connections, if you will. And one way to think about that and how these connections are identified is imagine a recruiter who has seen hundreds of millions of applications in their lifetime, has placed a bunch of people, has seen the employer or the recruiter reviewing signals that come back from that, those quality signals. And so they’ve seen so much, they’ve got so much experience that when a new job is posted, they just kind of go back into their unlimited brain power and identify the things that they’ve seen in the past and how that couldn’t relate to the job that’s now being posted. So going back to the product management position, if a new product management job is opened up in San Francisco, we may recommend that job to people who are actually in New York because we know that there’s A strong correlation for people who again are in New York looking for product work, who are willing to relocate to the Bay Area.
Matt Alder [00:14:58]:
We’re obviously still seeing a lot of change in this sector at the moment. What’s next? What’s on your radar for the next 18 to 24 months? What are the logical next steps for this?
Matt Plummer [00:15:09]:
I think for the next year, year and a half, we’re going to continue to focus on what we do best, which is, which is matching employers with candidates. So if we can continue to do that, we still think there’s a lot of improvements to make. There’s still a lot of data to learn from improvements in our classifiers and our machine learning models. So I would say we’re just going to keep, continue to double down on what we already do well so that we can be a clear industry leader and that we can be delivering employers with a good volume of candidates with good velocity and amazing quality.
Matt Alder [00:15:45]:
Great stuff. Now, my very good friends at TA Tech helped set up this interview and I know you guys are going to be at their event in Dublin later in the month. What are you most looking forward to about the TA Tech Europe conference?
Matt Plummer [00:15:59]:
A couple of things actually. I love the TA Tech circuit. It’s really come a long way since when I joined this industry and now they’ve got a handful of conferences focusing on different topics. So it’s been really fun to see that grow and evolve. Then the European conferences I find to be really interesting because you get such a variety of different opinions and positions from the different markets within Europe. So somebody coming from Germany is going to have a different perspective and different sort of problems that they’re dealing with than somebody from the uk. So I find that that diversity of opinions and challenges people are facing is pretty interesting. I find also personality wise, there’s a lot of more sort of stronger personalities and more strong opinions. So that’s always kind of fun. And then for this conference in particular, I’m going to be co presenting with our R and D lead from Tel Aviv and we work really well together and I’m really excited to get on stage with him and talk about some of the things that we’ve been doing, both from the business impact, as we’ve kind of discussed today, as well as some of the technology. So letting Avi, our R& D lead there, dig into some of the tech and kind of share that with the audience, that’s going to be really exciting.
Matt Alder [00:17:13]:
Matt, thank you very much for talking to me.
Matt Plummer [00:17:15]:
Yeah, Matt, it was great to catch up on this I think it’s a fascinating topic.
Matt Alder [00:17:19]:
My thanks to Matt Plummer and my thanks also to the guys at TA Tech who helped organize this interview. You can subscribe to this podcast in itunes or via your podcasting app of choice. The show also has its own dedicated app, which you can find by searching for Recruiting Future in your App Store. If you’re a Spotify user, you can also find the show there. You can find all the past episodes@www.rfpodcast.com on that site. You can also subscribe to the mailing list and find out more about Working with me. Thanks very much for listening. I’ll be back next week and I hope you’ll join me.
