Recruiting in Tech: We’re doing it wrong

The software development industry is highly competitive and ever-changing. With a constant stream of new technologies, tools and development approaches surfacing each day, keeping up to date is a mission in itself.

We invest in our people to propel us forward but without highly talented and productive designers, developers and testers, technology companies simply wouldn't survive. These days “average” performance just doesn't cut it. To deliver bleeding-edge solutions that differentiate us from our competitors we need bleeding-edge people.

Alan Eustace, SVP of knowledge at Google was quoted stating that “A top notch engineer is worth three hundred times or more than an average engineer”. Yet when it comes to expansion and hiring new staff, we use age old techniques that just don’t work.

We’re not as good as we think we are…

We all think we are great at hiring but in truth the majority of us rely on a subjective mix of cliché interview questions, obscure hypothetical “what if” situations, and gut feelings when making our final decisions.

What are your greatest strengths?

If you woke up and had 2,000 unread emails and could only answer 300 of them how would you choose which ones to answer?

What song best describes your work ethic?

The correct responses are easily rehearsed, and completely unquantifiable. Many studies have also shown that even managers with the best intentions are incredibly prone to confirmation bias and that we have usually formed an opinion of a candidate within the first 3 minutes of an interview. Sometimes this even happens as early as 10 seconds in! The remaining interview time is then spent (usually subconsciously) trying to confirm that bias.

To further worsen the situation very few managers track their success rates so there is little data to help improve their processes over time.

So do these unstructured interview questions really offer insights in to the candidate’s potential on-job abilities, or is it just guess work?

The science doesn’t lie

A study published in 1998 by Frank Schmidt and John Hunter analysed 85 years of data to see how well different assessment techniques were able to predict future performance. They found that unstructured questions, like the ones above, were only able to predict on average 14% of a hire’s on-job performance. Reference checks explained a miserable 7% and the number of years of work experience only added another 3%!

So even if we use all of these techniques we still only have a 24% likelihood of accurately identifying a candidates actual capabilities.

A smarter alternative

So what can we do differently? Which techniques will tell us if the candidate has what it takes to excel? The study showed that the following three approaches were the most effective:

1. Sample work test results were found to be the single most effective predictor of on job performance at 29%. For development roles this might involve solving a real world technical problem.
2. General cognitive ability tests came in second place at 26% and aim to assess a combination of raw intelligence and learning ability (much like IQ tests).
3. Structured interview questions came in tied second with 26%. These types of questions have right and wrong answers that are easily scored against predefined criteria (when asked consistently across all candidates) and therefore less likely to be affected by interviewer bias.

Nail all three and we’re looking at an 81% chance of identifying a candidates on job abilities!

Lessons learned at Google

Laszlo Bock, Senior VP of People at Google, adds his experience on the subject in his book “Work Rules – Insights from inside Google that will transform how you live and lead”.

Google receives over 2 million job applications per year and hire only several thousand. This makes them more selective than all of the top universities in the US. Though the global giant has a little more power in the recruitment space than most companies, Laszlo's recommendations for longer term success can be applied anywhere:

1. Assessing candidates for conscientiousness and leadership has large impact on success rates. “[Candidates who score high in these tests] are more likely to feel responsibility for the teams and the environment around them. In other words they are more likely to act like owners rather than employees”. Leadership skills should be team focused. The use of “we” rather than “I” is a key indicator here. Remember we’re trying to create a collaborative culture of trust and ownership.
2. A concise and enforced hiring rubric is essential as it removes ambiguity and bias. All candidates should be assessed against the same criteria to ensure that the resulting quantifiable data can be used to compare each applicant, both against fellow candidates and against existing high performing staff.
3. “Only hire people better than you”. This is the only way to grow your organisations talent pool. It may take longer but it’s worth the extra time effort.
4. “You also need managers to give up power when it comes to hiring [for their own teams]”. Laszlo admits that this is a difficult change to make but highlights one key scenario that hiring by committee aims to avoid: Managers urgently seeking staff inevitably lower their standards as the search drags on.
5. Looking at the data, Google has also found that academic performance alone is a poor indicator of performance, particularly beyond the first two years out of University.
6. Laszlo strongly recommends using multiple interviewers (including the candidates potential subordinates) to assess the applicant against the identical criteria independently and comparing the results. But be careful, he also notes that more than 4 interviews has been shown to add little value (1% each) and quickly frustrates the candidate.

So next time you’re looking to hire new staff take a step back, plan your approach and never settle. Hiring is incredibly important investment in your company’s future. Accept that finding the right person will take time and that making a rush “gut feeling” decision will only cost more in the long run.