Hiring

  • Olivia Wold
  • No Comments

We met 50 jobseekers in Nairobi — here’s what we learned
2000 1389 Olivia Wold

In late March, we had the chance to get to know an impressive group of professionals at our Nairobi office.

We wanted to meet some of our all-star applicants face-to-face and pick their brains about job seeking in Kenya, and also give them the chance to ask us questions and learn more about the Shortlist process.

The candidates — all experienced professionals in marketing, operations, finance, supply chain, sales, HR, IT and more — blew us away with their enthusiasm and commitment to unlocking their professional potential. Here are four of our team’s takeaways from these conversations:

1. Candidates are hungry to find their dream jobs

Each of the candidate attendees brought so much enthusiasm to the space. In fact, they were so excited to talk about their careers and job searches that they even came to hang out with us on a Friday night!

Candidates who are currently job-seeking are really the experts on what’s working and what’s not, and we appreciated their eagerness to give feedback. For example, in a discussion about job boards, we learned that some that we thought were popular weren’t so user-friendly, and we got tips on new ones (like Jobs in Kenya, UN Jobs Lists, Career Solutions and Impact Pool) to check out. Do you have other job boards to recommend? Let us know in the comments!

2. Candidates know what they want — but not where to get it

In small group sessions, we got to hear about the job search resources that candidates would love to have (and were willing to pay for), but don’t know how to find. Some of their ideas included:

  • Live chat with a career coach
  • Have an expert review and edit their CV (especially to cut it down from four or five pages to one!)
  • Get help with interview preparation, for example, receiving a list of common interview questions for a specific sector or job type
  • Take a test that tells jobseekers what positions they’re best qualified or suited for
  • Learn to develop an “elevator pitch” or cover letter that makes a powerful, positive impression on employers

This was definitely food for thought for us, as we strive to support both the candidate and employer throughout the sourcing and screening stages and beyond. We’re still in the process of developing our offerings for candidates, but if you know of any similar services or resources, let us know in the comments!

3. Recruiters should remember the Golden Rule

The job search can be a time consuming, disappointing and seemingly never-ending experience. Taking the extra effort to treat others the way you would want to be treated (with respect and fairness) would make everyone’s lives better during this process.

Many candidates expressed annoyance with getting generic rejection notices at the end of a lengthy job application. Understanding that it’s not possible for companies to write a personalized note to every applicant, candidates pointed out that getting a rejection addressed to “Dear Applicant” did not make them feel like their effort and interest was recognized.

People noted other ways for companies and recruiters to respect their time and effort, like making sure to take down expired job openings from their websites, or not accepting applicants when they already have someone else in mind for the job. These comments were a valuable reminder that behind every CV and application is a real person, who needs to be treated as such.

Lastly — authenticity matters! Jobseekers are very attuned to whether or not the companies they are applying to feel honest and real – in everything from the job description, to their e-mails, and especially the interviews. Companies should remember that the candidate is evaluating a fit with you just as much as the company is evaluating the candidate — so be real, tell it like it is, and be authentic!

4. You can’t beat in-person connections

In today’s world, a huge amount of interaction happens face-to-screen instead of face-to-face. Certainly, technology has enabled greater access to opportunity and incredible advances both in recruiting and professional development. But, we heard (and saw!) from candidates how excited they were to get in a room with their peers and talk about these career-critical topics. Webinars, Twitter chats and blog posts are great resources, but there’s nothing like being in a room together to make connections and form relationships.

When asked about professional networking groups, most people weren’t aware of any standout professional associations or networking groups in their fields. And of those who named some, they said they might help connect you with other professionals in your field, but don’t offer robust services for jobseekers. Let’s crowdsource some ideas — if you have more ideas for great meet-ups for jobseekers and professionals in Nairobi or Mumbai, let us know in a comment below.

See you next time!

To everyone who joined us at our happy hour — thank you, and we hope to see you again soon! If you’d be interested in attending events like this in the future, get in touch with me at Olivia@shortlist.net.

  • Paul Breloff
  • No Comments

The Limits of Machine Learning in Recruiting: Garbage In, Garbage Out?
1200 900 Paul Breloff
Hardly a day goes by without someone asking: “So Shortlist is basically using artificial intelligence to screen CVs faster?” My response: “Well, not really…”

There’s no doubt that AI, neural nets, natural language processing, and machine learning are having a moment. They’re the shiny new toys in Silicon Valley right now, and with good reason — the capabilities are powerful, and if the vision is realized, the world will change, big-time. And no doubt there are a number of companies looking at ways to apply AI, machine learning and related concepts to recruiting.

It’s true that we incorporate sophisticated data analysis into our screening. We collect hundreds of data points from each candidate, and we’re starting to engineer even more features or variables that can predict who gets hired, who performs well, and who sticks around. These data feedback loops can get even more powerful as we do more and more hiring with a single employer, learning what they like and who does well at their company.

That said, we see glaring limitations to applying machine learning and AI to screen candidates based on their CVs and social media profiles alone — summed up in the old computer science adage of “garbage in garbage out.” GIGO is the concept that any output can only be as good as the inputs, and ultimately, any predictive algorithm is only as good as the data going in and the outcomes you’re predicting. In this case, most fancy new machine learning-based solutions in the recruiting space are using CV data as the primary input, and “similarity to other candidates/employees” or to a job description as the outcome to be predicted.

But this is myopic: CVs are shown to have only modest predictive relevance to performance in a job when considered on their own. Of course prior experience can matter, but CVs alone fail to capture individual performance and contribution, raw talent, actual competence, motivation, and a host of other factors that are important in assessing quality.

There’s also the pesky reality that they’re often embellished and written to be picked up by keyword-driven screening engines, which distorts analysis and results. Think back to your own hiring experiences; how many times has that candidate who looked so great on paper disappointed in reality? And have you ever had great colleagues who didn’t go to great schools or work at fancy corporates but who have shined in the real world? So don’t ignore a CV, but don’t rely on it exclusively!

And there are limits on the availability of “outcome data” — i.e., how do people actually perform once on the job? Ultimately, you need good “training data” and time to build a good algorithm, which means you need a data set with outcomes you care about — i.e., what was the person’s productivity on the job, did they stick around, were they a great culture fit? Unfortunately, that data is rarely available when training an algorithm in recruiting contexts.

Unless you’re running predictions about actual performance, actual retention, and actual outcomes that matter — then you run a great risk that you’re just pattern matching the status quo, which may entrench the same hiring mistakes, the same biases, the same lack of diversity that we already see.

But if CV data is all we’ve got, what’s a recruiter to do? Well, we can start to generate new data and new signals. Our technology automates the collection of dozens of new, user-generated data points — raw data about experience and salary expectation, performance data drawn from cognitive and competency tests, and the meta-data about how a candidate goes through the process which can be mined for motivation, speed, curiosity. As we use this treasure trove of new data to supplement traditional CV data, we become even more excited about the promise of machine learning approaches to make sense of it, yielding better matches for companies and candidates alike.

So, it’s an exciting time for AI (particularly as my brilliant brother joins one the coolest AI companies out there — congrats Tom!), but I don’t think it will be the standalone silver bullet in recruiting for some time to come. Humans are just too darn complicated.

  • Paul Breloff
  • No Comments

Can we shift the recruiting paradigm from pedigree to potential?
1200 900 Paul Breloff
All around the world, companies big and small are facing a similar problem: Hiring is so much harder than it should be.

While India adds a million people to its job market every month and Africa is set to add more people to its workforce by 2020 than the rest of the world combined, over half of emerging market companies still can’t fill the roles they have open. Startups consistently rank talent acquisition as a top barrier to growth. What gives?

I saw this dilemma firsthand while investing in financial technology startups around the world for the last five years, as the founder of seed venture fund Accion Venture Lab. Once an investment was closed and cash was in the bank, the company’s problem shifted from not having financial capital to not having the human capital they needed to be successful.

The picture is even bleaker on the jobseeker’s side. Even skilled professionals often can’t get hired because they didn’t go to the “right” school, didn’t work at the “right” company, don’t know the “right” people, or fall victim to unfair biases during the application process. They are left lobbing their CV into job board black holes, never able to show potential employers what they can do.

This needs to change. We believe that talent is equally distributed, but opportunity is not. What often appears to be a lack of talent supply in markets is more often a failure of not knowing where to look or what to look for. At Shortlist, we want to level the job search playing field, shifting the recruiting paradigm from one based on pedigree and prejudice to a new version grounded in competency and potential.

How are we doing it?

1. Bringing intelligence to technology

Technology has burst on the scene to flatten access to job opportunities and broaden candidate pools (thank you LinkedIn and Monster). But without intelligent intermediation, more tech creates more noise, more decision fatigue, more work, and more despair for companies and jobseekers — not better outcomes. Just ask any of our employers who have received 2,000+ applications to a single job posting.

We combine a chatbot questionnaire with online assessments and phone screens to help us decide who is most likely to be great in a job. This filtering layer combines technology, data, and a human touch to ensure that talented candidates don’t slip through the cracks, particularly those who risk being overlooked based on CV alone.

2. Creating signals beyond the CV

Most companies have been hiring the same way for centuries (seriously): source and skim a lot of CVs, speak with some of the candidates, then make a decision — and regret those decisions more often than they would like. Not only is it hard to discern genuine ability and fit through a CV and unstructured interview alone, but this mode of decision-making is also often riddled with bias and prejudice.

Companies often do this not because they think it’s best, but because, frankly, there’s nothing else to go on. It’s like the joke about the economist looking for his keys under a streetlamp, not because that’s where he lost his keys, but because that’s where the light is better. At Shortlist, we engage candidates digitally to user-generate more accurate signals. We screen not only for basic experience fit but layer on additional data points for cognitive ability, competencies, and motivation. To be Shortlisted for a job, it’s more important to show us what you can do, not just tell us what you’ve done.

3. Refocusing on what matters

Let’s be clear: many people who went to great schools and worked at impressive companies are great and impressive. But for the vast majority of job-seekers, particularly in emerging markets like India and Kenya (where we work), prior experience paints an incomplete and often misleading picture of a candidate’s capabilities.

Schooling and subsequent corporate experience is — in all countries — more often determined by “birth lottery” than by merit. And we all hold biases, positive or negative, about certain schools or corporate brands. Looking past pedigree and refocusing on potential is the first step towards a world where everyone gets a shot at fulfilling professional experiences. Further, reconceiving the nature and focus of talent screening matters not only for hiring fairness, but also for hiring effectiveness. Building a team based on merit and performance instead of connections and pedigree is not only the right thing to do — it’s good for the bottom line.

The Shortlist mission

At Shortlist, we are on a mission to unlock professional potential and help great companies succeed in building great teams. We’re starting with a new way to match talent with opportunity, but we’re just getting started.

We want to level the talent playing field, but we can’t do it alone! We want to learn from each of you about what you think works to find and understand great talent, and what makes a great team. Visit our website, email us, or tweet at us — we’d love to talk with you about how we can help you hire. We’ll be using this blog as one of the ways we share the ideas behind what we do and how we do it, so stay tuned…

Structured interviews
  • Simon Desjardins
  • No Comments

Everyone should be using structured interviews — here’s why
1080 565 Simon Desjardins

Interviews are often less predictive of on-the-job performance than we imagine, a problem we wrote about here. Despite their ubiquity, unstructured interviews — where we ask candidates different questions in different sequences that may or may not be tied to job requirements — have been debunked as an effective predictive technique.  The interview process is vital to the hiring process and using structured interviews may be a better approach.

Structured interview 101

Decades of research tells us that using a “structured interview” is more than twice as effective than its unstructured counterpart in predicting on-the-job performance, and even more so when combined with competency-based assessments. The evidence is clear, and yet it’s extremely rare to see structured interviews employed in practice.

The idea behind a structured interview is underpinned by the objective of keeping the interviewer focused on questions that can predict performance and reducing the variance of scoring that exists when different interviewers prioritise different attributes in a candidate. They are best suited to interviews for junior and mid-level roles involving multiple interviewers.

Anatomy of a structured interview

  • Candidates are asked the same questions, in the same order, by every interviewer.
  • Questions are explicitly linked to key competencies required to do the job.
  • A standard rating scale is used by interviewers to grade candidate answers.
  • Interviewers agree in advance what they are looking for in a good answer.

When used correctly, the structured interview reduces the risk of bias affecting the interview outcome, increases consistency in ranking candidates, and minimises the interview time.

The key to using this concept effectively is crafting predictive questions and understanding that you’ll only have time to ask a few. Forcing ourselves to prioritise which two or three competencies actually drive 80 percent of the performance in a given role is a good place to start.

Reducing the time we spend on the interview process

We often hear hesitancy to adopt structured interviews because they can be perceived to take more time. “We have to hire 50 people this month. We don’t have time to implement a new structure,” is a common justification. Some may be surprised to learn that adopting a structured interview process actually reduces average interviewing time, particularly when more than one interviewer is involved. This is achieved both by reducing the pre-interview preparation period (because the questions are already prepared and optimized for an efficient interview) and by reducing the time to make a final hiring decision (because interviewers are clear about what they’re looking for).

Pathway to building a predictive question bank

Ultimately, building discipline around data collection throughout the interview process will help move beyond improving just our “hire rate” to something far more valuable: improving the rate at which high performers are selected. Hasty bullets written in our notebooks from 6 months ago are all but impossible to link to the ultimate performance of a candidate down the line (“What did we ask her again?”). If we have consistent interview data — meaning which questions were asked and what the responses were — we can baseline those questions with their teams to identify which interview questions and corresponding answers are most predictive of identifying high performers. We’ll also be able to pinpoint which questions are ultimately uncorrelated and can be dropped in the future.

Why are structured interviews so rarely used?

The ultimate benefits of a structured interview process can take months to materialise. Responsibilities are spread across multiple people on the team. Half a year may have passed by the time the candidate has been onboarded and we’ve had a chance to evaluate performance. The original interview seems like a distant memory at this point. Taking a decision to adopt a structured interview process will probably require a push from senior management along with commitment to enforce the practice.

We often have a related challenge of convincing hiring manager colleagues to change their interviewing practices, particularly when our colleagues don’t necessarily perceive interviewing to be a process in need of fixing in the first place.

At Shortlist, we have worked with clients ranging from start-ups to large multinationals. We also design competency-based assessments to further enhance screening outcomes before a candidate even reaches the interview stage.

Not all organisations will transition off of the unstructured interview as a screening tool, but the evidence to do so is clear. It’s time to implement a better way to interview.

About Shortlist Insights

Shortlist Insights helps companies build capacity to improve how they recruit and manage talent. We combine best practices from industry experts, research, and our experience to deliver practical and tested solutions and thought leadership. Ultimately, we help our clients build a competitive people advantage.

Related: Unstructured Interviews: Less Predictive Than We Think

Need help hiring, we can help
unstructured interviews
  • Simon Desjardins
  • No Comments

Unstructured Interviews: Less Predictive Than We Think
1080 651 Simon Desjardins

Interviewing effectively is surprisingly tough. It’s a process that sometimes seems straightforward and yet often leaves us feeling like we haven’t quite gotten the clarity we were hoping for out of an hour-long session, or that we’ve simply made a gut decision. The main reason for this feeling lies in fact that many people use unstructured interviews, which makes it much harder to reach a conclusion about whether the candidate would actually perform well or not.

In this series, we will explore some central challenges we face in the interview process, and highlight best practices and tools to make immediate improvements.

The statistics about interviews are both counterintuitive and somewhat alarming: the chances that unstructured interviews will accurately predict a candidate’s performance is less than 25 percent. You’d actually be better off flipping a coin, and would have saved several hours in the process!

Anatomy of unstructured interviews

  • Candidates are asked different questions by each interviewer.
  • Questions are not necessarily linked to key competencies required to do the job.
  • Candidates are not assessed using a standardized rating scale.
  • Interviewers haven’t aligned on acceptable answers beforehand.

With dozens of studies across multiple geographies and timelines showing them to be one of the worst ways to predict on-the-job performance, the evidence against using unstructured interviews is overwhelming. Yet we continue to rely on them almost exclusively to make hiring decisions. What’s going on?

Limiting the effects of bias

Bias plays a huge role in how we rate candidates in an interview setting. Humans are naturally and strongly predispositioned to favour people who are like them, which is a hazard when our objective is to build diversity on our teams.

Compounding this problem is our susceptibility to “first impression” bias, where we consistently end up asking easier questions of people who form a strong first impression, and harder questions for those who don’t. In their now-famous study, Tricia Pricket and Neha Gada-Jain showed how snap judgements made in the first 10 seconds of an interview could predict the outcome of that interview.

Mere awareness of these biases does little to counteract them, even for experienced interviewers. Asking candidates to complete a competency-based assessment process before the candidate reaches the interview stage adds a layer of objectivity to the screening process and can help put interview results in context.

Linking interview questions to competency requirements

Unlike unstructured interviews, great interviews start with the interviewers aligning on what competencies and other requirements will actually drive performance in a given role. It may sound obvious, yet we often see a majority of an interview being spent asking questions without a clear competency in mind.

Brain teasers, for example, are a perennial favourite, though they have been shown (most conclusively by Google’s HR team) to have no correlation to performance. By a similar token, academic scores (and by extension — questions about them) have equally little bearing as predictors of performance, unless we’re hiring people immediately after they graduate.

Asking questions that actually force a candidate to reveal a key trait can be risky without preparation and thought. For instance, if we wanted to understand a candidate’s “motivation to join,” we might be tempted to ask the basic question, “Why do you want to join our company?” This question, however, is easily answered by a clever candidate who has done their research. A less obvious but more revealing question, such as “What preparation did you do in the time between when this interview was scheduled and today?” might give us a much more meaningful data point on the same subject.

Interviews and the Stanford prison experiment

Back in 1971, Stanford University psychologist Philip Zimbardo led a set of experiments that changed what we know about human behaviour. Zimbardo arbitrarily assigned participants to play the role of “prisoner” and “guard” in a role play exercise, and the inhumane behavior of the “guards” revealed the extreme psychological effects of perceived power. The results were replicated many times across multiple countries and cultures.

Interview rooms aren’t prison cells, of course, though much of what was learned in those studies applies to this context. Given that employers have a job to offer, we sometimes assume a position of authority in relation to the candidate, and act accordingly, without realising it. Imagine, however, that we’re interviewing a top performer who has multiple employment options. In this situation, we’re being interviewed as much as we’re interviewing. Leaving five minutes at the end to answer a candidate’s questions won’t be enough to properly address concerns and communicate why the candidate should join us.

Interviews driving improved performance

Admittedly, interviews tend to be high stakes environments for both parties. We’re under pressure to make the perfect hire. The candidate is nervous. Skepticism, hope, and bias are at risk of permeating every exchange. If you leave time to “sell” the candidate at the end, we’re down to 40 minutes at most to ask five to eight questions. Given the exponential impact that high performers have on organisations, these 40 minutes are crucial.

Putting more thought and structure to that time will separate you from the vast majority of other hiring companies, including your competitors how are using unstructured interviews.

If unstructured interviews don’t work, then what’s the answer? In the next article in this series, we outline the basics of the “structured interview” and when combined with competency-based assessments, will save time and significantly improve the outcomes of the interview process.

About Shortlist Advisory

Shortlist Insights helps companies build capacity to improve how they recruit and manage talent. We combine best practices from industry experts, research, and our experience to deliver practical and tested solutions and thought leadership. Ultimately, we help our clients build a competitive people advantage.

Related: Everyone should be using structured interviews — here’s why

Start Typing