The human impact of live technical interviews

Read the eBook

Perspectives from engineering, talent acquisition, the Interview Engineer, and the candidate

Starting the hiring process with a live technical interview might seem like a luxury. Software engineering time is so constrained that most engineering leaders (87%) agree that “it would be easier to reach hiring targets if they had more time to interview candidates” and 61% agree that “interviews are a financial drain on my company.” In response, many companies place a take-home or online code test at the top of the hiring funnel in an effort to limit engineering time spent interviewing. However, due to poor candidate experience and poor signal compared to live interviews, code tests often end up taking more engineering time to vet candidates and decrease hiring yield from an already limited talent pipeline.

Compromising signal and candidate experience impacts many stakeholders. Here, we’ll explore the value of the signal and positive experience delivered by a fair, predictive, and enjoyable technical interview through the eyes of engineering, talent acquisition, the Interview Engineer, and the candidate.

Capture signal on critical skills

In May of 2020, 64% of engineering leaders reported that interviewing is a drain on their team’s morale. Time spent interviewing the right candidates can be energizing, but time spent interviewing the wrong candidates can be draining.

Code tests often lack the opportunities for the candidate to demonstrate a competency and lack of live interviewer participation can also allow for cheating. As a result, the hiring funnel retains the wrong candidates and dedicates valuable engineering time to interviewing them.

Engineering productivity and morale can be improved by reducing false positives with a live technical interview aligned to your hiring bar.

They can also unintentionally filter candidates out due to a binary pass/fail recommendation that requires absolute completeness and correctness.

Live first-round technical interviews showed “...a dramatic improvement over [code tests] and our own emailed questions. Without a live technical interview, too many false positives get through, causing bad onsites and impacting team morale.”

Claus Moberg VP Engineering, Roblox

Capture signal on developer seniority

Senior developer candidates are often reluctant to take coding tests and prefer a live interview in which they can have a conversation about an approach or solution. At the top of the hiring funnel, this can decrease sign up rates. By their very nature as a solo-effort, code tests will miss a core competency of any senior role—communication.

Senior software engineers will be needed to work across teams, align on approaches, and contribute to the culture of the team. Leveraging a live technical interview at the top of the hiring funnel will filter in those candidates with the communication skills characteristic of a senior software engineer.

“...The test might be ‘write an app.’ Ok, we could verify that it worked, but it made it hard to say that the candidate is more experienced, which we really look for in their thought process and communication skills. A live interview helps us see seniority in both. Like how a candidate takes feedback and how they would work cross-functionally to approach a methodology. Everyone has opinions on how to approach an issue, but the most important thing is how they work together. So a live interview can make up for the missing piece of a take-home test.”

Tina Tian VP Talent Acquisition, Ethos Life

Completeness and correctness are costly

A code test requires completeness by its nature. Yet, in a live interview, an experienced interviewer will be able to get sufficient signal to provide an accurate recommendation regardless of completeness. Reliance on completeness is in fact so unimportant that Karat data shows 55% of offers go to candidates with “incomplete” solutions.

Absolute correctness is another characteristic required by code tests that is of limited value. Karat data shows that 12% of offers go to candidates who received guidance from an Interview Engineer.

Giving guidance doesn’t mean giving away the interview. In fact, it is much more representative of a real working environment in which software engineers work together to do code review and quality assurance to ensure their code will work as intended.

"...If you’ve got an Interview Engineer there, someone who understands the code...the approach...the problem, they can see you were just a couple of minutes away from finishing, or you were just a semicolon off. I can give you credit for being really close. That might be the difference between getting to the next stage of interviews or not.”

Adam Dangoor Interview Engineer, Karat

Connect talent to opportunity

The internet is filled with candidate commentary on code tests from all perspectives. To understand why much of it is negative, consider that what may be a basic business process to a company is something much more meaningful to a candidate. An interview can be exciting, terrifying, or somewhere in between. It can mean moving to a new city, buying a house, or the opportunity to provide for a loved one.

The interview connects talent to opportunity. The human guidance behind a live technical interview means it can be predictive, fair, and enjoyable for the candidate all at the same time.

“Comparing this [code tests], I prefer this. I never felt rushed because the interview engineer was there to reassure you that while you may not finish it, this is only part of the hiring process and you will get partial credit. That calmed me down a good deal and I really enjoyed myself.”

Lead Front-end candidate, InVision