Case Study

How Wayfair improved hiring efficiency with Karat’s Interviewing Cloud

A customer story with Wayfair

Challenge

Wayfair was up-leveling their engineering organization while building new product infrastructure, which required more interviewing capacity and predictiveness than they had on their own.

Solution

Karat conducts technical interviews on behalf of Wayfair and provides analytics, insights, and detailed recommendations to increase hiring yield across all roles.

Results

12,000+ interviews conducted, 24,000+ engineering hours reclaimed
284 total hires
23% YoY improvement in onsite-to-offer ratio
19% improvement in technical interview passthrough rate

Challenge

In 2018, Wayfair was experiencing tremendous growth as they expanded their market footprint, broadened their product offerings, and had an increasingly loyal customer base driving a higher rate of repeat purchases. As the company eyed their next growth targets, the head of engineering saw one thing clearly — they had to up-level their entire organization to build out the technology needed to power this growth.

While that might sound straightforward — hiring a large number of engineers at scale — this presented two challenges. First, Wayfair needed their current engineering team to focus as much time as possible and shipping more code to keep up with customer demand and growth targets, so taking time away from that work to conduct technical interviews would cause significant delays. And because they were up-leveling the entire organization, they didn’t have enough engineers with the level of skills needed to assess candidates for these higher-level roles.

Jonathan Kehoe had recently joined Wayfair as the Head of Technical Recruiting, with the objective to build a recruiting engine that would help the engineering team meet their hiring targets while building higher-level, higher-quality teams. “This was a daunting challenge to walk into”, said Kehoe. “We were already seeing market shortages of engineers across the board, so having enough candidates in the pipeline to begin with was getting really tough. And with not enough engineers to assess most candidates, our modeling told us it would take far too long to hit our hiring targets. That was time we just didn’t have.”

Kehoe came across indeed’s story about how they scaled their engineering organization. He saw two key themes in common — the need for as much interviewing capacity as possible, coupled with the ability to accurately and consistently assess higher-level technical skills. When he learned that Indeed had worked with Karat to achieve their goals, leveraging the capacity in their Interviewing Cloud and the strength of their hiring signal and recommendations, he reached out immediately.

We were already seeing market shortages of engineers across the board, so having enough candidates in the pipeline to begin with was getting really tough. And with not enough engineers to assess most candidates, our modeling told us it would take far too long to hit our hiring targets. That was time we just didn’t have.

Jonathan Kehoe
Head of Technical Recruiting

Solution

Before Karat could begin conducting interviews for Wayfair, their teams worked closely together to identify the competencies and skill levels that Wayfair’s engineering organization would require, along with Wayfair’s technical hiring bar for each role. This process, called Alignment Engineering, ensures the best possible onsite-to-offer ratios because it calibrates Karat’s recommendations to Wayfair’s specific criteria. Doing this up front gives clients like Wayfair confidence they are advancing the right candidates to the next round.

The initial scope of the work was focused on Wayfair’s storefront engineering team — 350 interviews for front-end engineers. Karat’s recommendations were well-calibrated quickly, and based on data from Wayfair’s on-site interview rounds that followed Karat’s technical interviews, the recommendations were highly predictive of who would ultimately receive offers. Wayfair saved considerable engineering time and they were making quality hires, albeit not at a fast-enough pace.

By combining Karat’s data with their own ATS data, Wayfair learned that a key reason for this was the quality of Karat’s screening process, which was sending through a lower percentage of candidates to the on-site round than when Wayfair was doing their own technical interviews or using an online code test. Kehoe and his team were able to use this data to overhaul their sourcing strategy. “That was such a big learning,” said Kehoe. “The power of Wayfair’s and Karat’s data together really was powerful. We saw our sourcing strategy in a new light, and we could see more clearly what was working and what wasn’t. Once we made adjustments based on which candidates were ultimately hired, our whole hiring process really began to take off.”

Importantly, Wayfair’s engineering team was seeing significant benefit from working with Karat. They were getting significant time back to build products. With some really big goals to hit, their confidence in achieving them increased every month they worked with Karat. Wayfair’s team was more focused on core work than ever, and each time they brought on a new group of higher-level engineers everyone was doing better work.

Karat’s Interviewing Cloud has become a significant differentiator for how we hire engineers. Our engineering team loves the time they get back -- tens of thousands of hours over just two years. Karat’s 24/7 interview coverage is great for candidates, and their around-the-clock scheduling support has helped us to streamline the transition to remote hiring.

Jonathan Kehoe
Head of Technical Recruiting

Results

Karat has conducted over 12,000 interviews on Wayfair’s behalf as of August 2021, which has helped their engineering team reclaim over 24,000 software engineering hours back to product work. Yet while Kehoe initially reached out to leverage Karat’s interviewing capacity and meet hiring targets, there were so many more benefits they received.

Building on the insights Kehoe and his team gained into their sourcing strategy, they began to apply the Wayfair and Karat data to other parts of their technical hiring process. Karat’s insights have helped Wayfair make critical talent strategy decisions, including whether to centralize hiring for some or all roles, increase the consistency of their leveling, and whether and where to open new offices. “The perspective we get from Karat is unmatched,” said Kehoe. “We can never get enough information about how our results compare to others, are there sources of talent we’re missing, are we saying ‘yes’ or ‘no’ to candidates we should be saying ‘yes’ or ‘no’ to? I have significantly more confidence now than I did three years ago that we’re making effective decisions throughout our hiring process. And my team is more confident that in such a dynamic talent market, the things we’re testing have a higher probability of paying off.”

From a starting point of interviewing front-end engineering candidates for Wayfair, Karat is now supporting Wayfair’s centralized hiring model with a much broader scope and for more senior roles. Kehoe and his team are continuing to work closely with Karat as Wayfair keeps growing and innovating, and he cites three reasons for this — the time savings, the quality of the recommendations, and their increased yield of quality hires in a competitive talent market.

Karat saves Wayfair time in three ways:

    • First, because Karat conducts technical interviews 24/7 with a global network of Interview Engineers, candidates can choose the day and time that’s best wherever they are. This is more important than ever as companies welcome more remote employees and broaden their sourcing footprints. Wayfair can now engage a more diverse set of candidates without a dependency on the availability of their engineers, so hiring is always moving forward. Karat also supports candidates around the clock, which reduces operational overhead for recruiting teams and increases responsiveness for candidates.
    • Second, because Karat’s technical interview is highly-calibrated, professional, and consistent, Wayfair saves time they would otherwise spend continuously training their own team to conduct better technical interviews.
    • Finally, because Karat’s recommendation system has been so closely calibrated to Wayfair’s specific technical hiring bar, their engineering team has the confidence that every candidate they ultimately interview will meet their minimum requirements.

In addition to the time savings, the depth of the Karat recommendations lead to a stronger signal, so Wayfair makes more informed choices about who to bring onsite. In addition to the top-level recommendation, Karat provides insights into the components of the interview from technical knowledge to coding ability. “Karat’s recommendations are so multi-faceted, they help us understand candidates more thoroughly than we would otherwise. Just having 4 tiers of recommendations helps us prioritize who to move to the next round, and seeing the components of the assessed skills broken down gives the team focus on where to further evaluate during that final round.”

Karat’s Redo interview has also been a game-changer for Wayfair. Because Karat offers Wayfair’s candidates the ability to redo an interview, Kehoe and his team are able to increase their hiring yield without doing additional sourcing. Redos have been very efficient for Wayfair, with 31% of their candidates completing redo interviews, leading to 86 hires from this group of candidates alone. As one of their candidates said, “I had a redo interview, because I felt I could do better than I did in the first interview. That was exactly what happened: my second interview was much better, and I felt I was able to demonstrate my skills properly.”

Wayfair is continuing to grow at a rapid pace and is always focused on up-leveling the skill level of their engineering organization, so their imperative to hire quickly without sacrificing quality remains. Wayfair’s pass-through rate of candidates from the Karat interview to the on-site round has increased 19% over the past two years, and Karat continues to publish interview results for Wayfair in about a week after the candidate is invited to interview.

Kehoe sees the partnership between Wayfair and Karat continuing to grow. As he said, “Karat’s Interviewing Cloud has become a significant differentiator for how we hire engineers. Our engineering team loves the time they get back — tens of thousands of hours over just two years. Karat’s 24/7 interview coverage is great for candidates, and their around-the-clock scheduling support has helped us to streamline the transition to remote hiring. Karat is also incredibly consistent and has data nobody else has, it’s like holding a mirror up to your full hiring process and seeing what’s working well and where you need to improve. There is no way we could do what Karat does as well as they do it.”

 

Karat and Wayfair partnership by the numbers

12,000+ interviews conducted, 24,000+ engineering hours reclaimed
284 total hires; 86 from Redo interviews
23% YoY improvement in onsite-to-offer ratio
19% improvement in technical interview pass-through rate to on-site round