In today’s job market, the hiring landscape varies greatly across industries. While some are capitalizing on the surplus of software engineering talent from recent tech layoffs and hiring top talent at an accelerated pace, others are proceeding with caution. For those in the latter group who are taking a more measured approach, this article provides insights on auditing and optimizing your hiring process so that you’ll be prepared to quickly and effectively fill positions when the market picks up again.

We have developed a comprehensive rubric for auditing a technical hiring process, along with some best practices for each stage, based on 300k+ interviews across 400+ companies in virtually every industry hiring software engineers.

Sourcing audit and benchmarking

What optimal sourcing looks like: a high volume of technically proficient, diverse candidates are completing interviews for roles at your company. In order to understand the strength of your talent sources, it’s essential to have a reliable technical interview that you can trust as part of your hiring signal. Working with an interviewing partner like Karat allows organizations to benchmark sourcing and hiring pipeline quality externally against competitors based on interview passthrough rates or the total percentage of candidates you’re interviewing that score in the top quartile globally.

Best practices to improve sourcing quality:

• Consider remote work as a benefit and a strategy: Remote work is the single most effective way to expand a company’s candidate pool and help overcome the challenge of finding and closing good engineering talent. By offering remote work options, companies can access top candidates that fit their needs and budget. While some companies are subject to regulations that require in-person work, those that can provide remote work options have a competitive advantage. Additionally, the adoption of remote work has opened up new opportunities for companies to tap into international and global markets where average technical proficiency is on par with many top tech regions in North America. Embracing remote work not only increases a company’s talent pool but also helps them stay competitive in the ever-evolving job market.
• Remove pedigree bias: A key strategy for improving candidate sourcing is to break away from the traditional notion that top talent only comes from the top universities or from Big Tech. By focusing on these factors, companies risk missing out on a wealth of talented individuals who may have a different background or experience but are equally capable of making valuable contributions. Evaluate candidates based on applied skills, experience, and potential rather than their pedigree; doing so can help companies access a much larger pool of engineering talent while increasing the diversity and creativity of their workforce.
• Scrutinize your sourcing channels: Analyzing your sourcing channels is a crucial aspect of auditing the overall hiring process. The effectiveness of different sourcing channels can vary greatly between companies, making it important to assess the return on investment for each channel. A thorough understanding of sourcing channels helps companies to maximize their resources and improve efficiency over time.

The company in the example above discovered the most efficient sourcing channels for hiring candidates in 2022, allowing them to make informed decisions on future strategic investment.

Technical assessment

What an optimal technical assessment looks like: for candidates, the tech stage of the interview process is fast, fair, and flexible; for companies, it provides clear and easily interpreted results that are a strong indicator of candidate success while not being a drain on internal resources. Best practices:

• Go resume blind: To remove pedigree bias at this stage, interview engineers should not know the candidates’ education and work background.
• Offer guidance: Code tests usually demand exact accuracy, which the candidate must achieve independently. This differs from the collaborative nature of a real software engineering team, where engineers work together to tackle complex issues. Interviews should reflect the collaborative nature of software engineering work by allowing for open communication between the candidate and interviewer, ultimately resulting in a stronger signal of critical competencies and even a higher likelihood of offer acceptance.
• Use structured scoring: To further minimize implicit bias, the interviewer should use a structured scoring rubric rather than a simple pass/fail recommendation. Ideally, the interviewer should not be aware of the passing score threshold, and the decision to move the candidate to the next round should be made objectively using an automated scoring system or by someone not involved in the interview.
• Offer a redo opportunity: A redo interview can benefit both the sourcing process and closing rates. It offers a cost-effective way to re-source candidates, as they can try again instead of requiring a new candidate to be sourced, thus improving recruiter efficiency. In terms of closing, taking a redo indicates a candidate’s level of intent – these candidates are less likely to accept a competing offer. From a diversity, equity, and inclusion (DEI) perspective, redo opportunities have been shown to impact candidates from underrepresented groups. Research from Karat’s Access Gap report suggests that underrepresented candidates disproportionately experience interview anxiety and impostor syndrome, leading to false negatives on coding tests (performing below their actual abilities). A redo opportunity helps to mitigate these false negatives and provides a more accurate assessment of a candidate’s abilities. If DEI is a priority, redo interview opportunities help level the playing field and provide more equitable outcomes.

Onsite interviews

What an optimal onsite process looks like: qualified candidates are invited to the onsite stage efficiently and fairly, resulting in efficient offers and hires. Best practices:

• Scrutinize pass-through rates: Check for pass-through bias to ensure candidates who scored similarly from different backgrounds weren’t being advanced at different rates. When recruiters manually decide who to move forward in the hiring process, we often see pedigree bias and implicit bias resulting in majority demographics being favored – even when performance is equal. This is a critical point in the hiring process where bias is frequently observed.

The company in the example above found that male candidates from the “Requires further review” recommendation tier were being moved forward to the onsite stage more often than female candidates. This was quickly resolved with additional recruiter training and the continued tracking of this metric.

• Evaluate your tech assessment: Use pass-thru rates along with onsite-to-offer rates to evaluate whether your tech stage is too selective or too lenient.

The chart above is a general rubric for how to interpret pass-thru and onsite-to-offer rates. “High” and “low” values for each metric will vary depending on company, industry, and market conditions.

• Track close rates against the market: Offer acceptance rates vary widely across industries and also over time as market conditions change. In 2021 amidst a very strong candidates’ market (most candidates had multiple offers to choose from), top companies had difficulty achieving 60% close rate goals; by late 2022, after the economic downturn and resulting layoffs, even non-tech companies were closing candidates at 70% or higher. Offer acceptance is influenced by a range of factors, such as – compensation, reputation, candidate experience – but this metric is more nuanced than simply “improving or declining”. Market context is vital to understand how your company is faring compared to your hiring competitors.

In Conclusion

Shifting job market conditions present different challenges and opportunities for companies seeking to hire top software engineering talent. Regardless of current approach, all companies can benefit from auditing and optimizing their hiring process to ensure that it is fair, efficient, and effective. By following the best practices outlined in this article, companies can increase their talent pool, reduce bias, and make data-driven decisions to understand and improve their hiring process at every stage.

Looking for help in auditing your hiring process? Karat can help. Request a consultation.