How Karat supports leading technical recruiting processes.
Technical interviewing and the technology to make it predictive, fair, and enjoyable.
Our mission is to make interviews fair, predictive, and enjoyable.
What developer candidates need to know about the Karat interview.
Tips for bringing down onsite-to-offer ratios and cutting your interview tax
It’s not uncommon for us to start working with a new client and discover that onsite-to-offer ratios are in the 5:1, 10:1, or even 15:1 range.
For engineering teams, high onsite-to-offer ratios represent a massive tax. Having a 10:1 ratio means that for every new hire, you’re conducting 10 onsite interview loops, each with somewhere between 3-8 people.
That doesn’t even factor in the time spent on technical interviews, phone screens, and candidate sourcing. At that rate, it’s easy to imagine spending well over 100 hours of engineering time per engineering hire, which is an unfair request for software engineers who are already overworked to the point of needing to hire more colleagues.
But what should an onsite-to-offer ratio be?
The exact number depends on the particulars of your market, geography, recruiting and role requirements, but anything over 15:1 sounds unnecessarily high. To start with, if your ratio is over 10:1, try cutting it in half, or even aim for 5:1.
Performance reviews, improvement plans, and promotion requirements all tell you what actually gets assessed for a particular role. What are the common reasons employees succeed or struggle?
Identify the skills and experiences that are not only most relevant to the role but also important when making a hiring decision. Cross off knowledge or skills that can be learned on the job. Prioritize the competencies that remain, and then build interviews and a decision-making process around the top items. This is your signal.
Before changing your process too much, assess the consistency and predictability of your current process.
Characterize the reasons candidates are rejected. Evaluate the relevance and importance of the skill/knowledge/experience behind each reason. In other words, is each competency contributing signal or noise to the hiring decision?
On interview question guides and assessment rubrics, explicitly clarify what is considered noise and what is considered signal. Update how questions are asked and how observations are recorded to make it easier for interviewers to succeed in this task.
Improve your current onsite process by making each step in the interviewing process more intentional.
Once you understand why candidates are being rejected, assess that earlier in the process. For example, if the System Design interview is a common cause of onsite rejection, move some system design questions into an earlier gate.
While you could move the System Design interview into a new gate wholesale, often you’ll want to create a smaller, more efficient module to get at the core competencies, either as part of a phone screen or as part of a 1-hr pre-onsite gate that contains a few different modules.
You can even create smaller interview modules by being specific about the competencies you are assessing. Being specific will allow you to cull extraneous context and tasks to turn a 45-minute interview into a 15-minute foray.
“System Design” is a topic, not a competency. Consider Bloom’s Taxonomy and be specific about the verb and objective.
Do you need your engineering managers to create an architecture from scratch? Or to demonstrate that they can analyze two solutions proposed by a senior architect? Do they need to evaluate the judgement of their architects, or exercise good judgement themselves? Do they need to remember the CAP theorem or do they need to generate insightful conclusions based on known constraints around system consistency and availability?
It may be that asking an engineering manager to write code is a good proxy for the problem-solving skills, strategic thinking, and general learning and mastery skills that you are looking for, not to mention exposing experience likely to elicit empathy and understanding for and from their reports.
Similarly, while it may be that someone could learn a particular skill or piece of knowledge on the job, say the CAP theorem, it’s also a proxy for real experience and may suggest a candidate hasn’t yet grappled with the challenges of distributed systems, and may not be ready to set up their team for success on an ambitious redesign.
The problem is that proxies are a double-edged sword. They hide what you’re getting at for both interviewers and candidates, which means things tend to go wrong in the assessment happening on both sides of the table. Without turning this essay into a book, my main advice is this: whenever there are directly relevant and important competencies that you can assess, assess those first.
If you still need a hiring decision to rest atop a proxy, make it as explicit as possible for everyone. “This isn’t a coding interview, this is a strategic problem-solving interview. We’ll first ask behavioral questions to explore how you’ve used strategic thinking to solve problems in the past. Then we’ll ask some greatly simplified toy problems to see how you solve problems and utilize strategy in real-time.”
For the behavioral part, you might try to dig into a technical challenge, or you might be ok with investigating their strategic skills when handling a people management quandary or a surprise challenge impacting a business deadline.
For the skill assessment, maybe it’s more about problem-solving skills. If you’re looking for candidates’ ability to come up with approaches and analyze trade-offs, you may realize the coding is not even important. That is signal that you can uncover in a subsequent code review interview, or in a different module where they explain code they’ve written previously (ideally, this would be a patch you can also look at, too, or else they’ll have to paint you a detailed picture using words).
Or perhaps, you do want to see them spend 5 minutes writing code, but you also tell interviewers not to waste time while the candidate nervously overlooks a missed semicolon or forgets a function name. Just tell them and move on; it’s noise to this particular signal.
Try different approaches and measure your onsite-to-offer ratios until you start seeing the improvement you’re looking for.
Senior candidates, especially the ones you want to hire, are likely happily employed, or else considering multiple exciting opportunities. The start of your pipeline could be the start of a long-term relationship; the foundation matters A LOT to candidates. That conversation, ideally with a hiring manager, is a chance for the candidate to:
Regarding the first point, you can still hire for humility, ability to compromise, respect for the less empowered, etc. The point is that the leverage equation is different than when hiring college graduates. In early-career hiring, it’s more common to ask candidates to prove they can code, often with very little veneer.
A better trade-off for senior candidates is not a code test, but rather a written application. Give the candidate has a chance to answer a few questions about their past experience, strengths, and future interests. Replacing the conversation with a coding exam is not a decision about saving time, but about what competencies matter.
For senior candidates, you want to maintain the initial human connection as much as possible. Reduce the onsite-to-offer ratio by making the initial phone screen assess more signal.
You may need a 1-hr gate between a non-technical recruiter phone screen and the 4-5 hr onsite. However, before introducing new competencies into that gate, consider what your current interviews are doing right and what you may want to change. Move the common rejection reasons earlier.
Work backward from job success. Getting the right answers to the wrong questions creates chaotic results, which is near impossible to improve iteratively. Once you know specifically what competencies are relevant and important to a hiring decision, the interview questions (and guides and structured interview rubrics) practically write themselves.
I hope that helps!
Your email address will not be published. Required fields are marked *
Inclusive hiring is essential for improving the diversity of your software engineering team. No longer is inclusive hiring the sole responsibility of the talent acquisition...
Hiring software engineers is notoriously challenging. Whether it’s a lack of time to conduct technical assessments, inability to find the right candidates in the first...
Although more womxn are entering the tech field, there still lies a huge gender gap when it comes to the ratio of men vs. women...