How Ethos Life improves software engineering candidate quality

The Challenge

In March 2019, Ethos Life was preparing to raise its next funding round. The company needed to simultaneously grow its software engineering team and accelerate an ambitious product development roadmap. The goal was to rapidly deploy new technologies to transform the customer experience in a very old industry—life insurance.

VP of Engineering, Vipul Sharma, leads the team responsible for that innovation. “Ethos is a modern solution for life insurance and we’re leveraging advanced technologies such as machine learning and predictive analytics on top of a complex platform built by our team. This requires a strong engineering team at the heart of the operation, so being able to properly identify and bring in engineering talent is definitely important in positioning Ethos to succeed.”

To successfully identify and hire the right engineering talent, Ethos developed interviewer training, but the volume of interviews hampered the engineering team’s ability to do their jobs simultaneously. The talent acquisition team surveyed Ethos’ engineers about their interview load and found that the majority of the team felt overburdened by interviews.

Interviewing is a productive use of time as it allows us to build our team. However, we had to consider that when engineers are interviewing, this is time they’re not able to spend on high priority projects.”

Prior to partnering with Karat, Ethos evaluated code testing tools and a homegrown test to identify the right candidates with the goal of maximizing hiring yield. However, Head of Talent Acquisition, Tina Tian, quickly realized that those tools failed to evaluate the seniority of a candidate.

“For example, the test might be ‘Write an app.’ Ok, we could verify that it worked,” said Tian. “But it made it hard to say that the seniority is there, which we really look for in communication style. A live interview helps us see seniority in communication style. Like how a candidate takes feedback and how they work with other teams to approach a methodology. Everyone has opinions on how to approach an issue, but the most important thing is how they work together. Take home tests can’t identify that.”

At the same time, Sharma saw that the test would not scale. “Creating and reviewing the take home tests took engineering time. Additionally, we missed the opportunity to have candidates walk us through their thought process or show us their communication and working style.

Our technical calls solved this, but again, it was taking engineering hours to build the test, administer or attend the interview, and review afterward. Our team understands how important it is to build the best team and was willing to put in the time and effort. However, this was not the ideal setup. We were, at the time, and are still, building rapidly and constantly require all hands on deck.”

Then, a member of the engineering team suggested Karat. “We had to quantify the value engineering time, once we did, it was significant. We realized that using an external partner would have a high ROI. Internal bandwidth was important, that’s why we went with Karat,” said Tian.

“We realized that using an external partner would have a high ROI. Internal bandwidth was important, that’s why we went with Karat.”

The Solution

Karat conducts first-round technical interviews on behalf of Ethos for Back End, Front End, Full Stack, Data Science, and SWE 1 roles in Austin, Singapore, and San Francisco. To get started, Karat’s Alignment Engineers worked with the Ethos team to identify competencies that accurately represented the Ethos hiring bar. After two calibrations, the competencies for each role properly represented the seniority and coding style needed to be successful on the Ethos Life engineering team.

“Defining our bar was a step forward for us. It allowed our team and the Karat team to be on the same page with the candidates we wanted to attract and hire,” noted Sharma. “Long-term, this saved time and created more trust in the process. As a result, we were able to gradually have more candidates go through the Karat process, and more candidates aligned with what we were looking for on the engineering team.” Tian added, “we ended up with a framework that quantifiably tells us what each approach to solving the question really means. If they’re an 8 out of 10 in algorithms, then the candidate is a ‘yes’, but if they’re an 8 out of 10 in systems design, then it’s a ‘no.’” She advises future Karat partners, “it’s really valuable for talent acquisition and engineering to spend time to build that framework with Karat.”

As Compass’s growth continued to accelerate, so did their partnership with Karat as more of their engineering teams adopted the solution. When it came time for Sirosh and his team to open a development center in India, he and Smith were focused on keeping the hiring bar there consistent with their existing teams. “This added another dimension to the value of Karat’s data”, said Smith. “Because their process is so consistent, their data gave us tremendous confidence that our hiring efforts in all locations were meeting the bar.” There was another unexpected benefit of Karat, as well. Because Karat conducts technical interviews 24/7, they were able to support Compass’s candidates and recruiting teams with around-the-clock support while keeping hiring moving forward.

“[Karat] allowed us to reflect on what kind of team we wanted to build, and the caliber of engineers needed to get us there.”

Ethos

Today, Sharma says, “the end result is our stellar team and growth trajectory. The other thing is that [Karat] allowed us to reflect on what kind of team we wanted to build, and the caliber of engineers needed to get us there.”

Ethos has not only reclaimed engineering time previously dedicated to interviewing and evaluating code tests, but realized efficiencies on the recruiting team as well.

“We realized that using an external partner would have a high ROI. Internal bandwidth was important, that’s why we went with Karat.”

“The cost is a no-brainer, we can turn any recruiter into a tech recruiter overnight,” said Tian. She added, “engineering trusts the [Karat] recommendations and lets talent acquisition run with the process, they say ‘I trust your judgment.’”

The engineering team is also more open when talent acquisition asks them to do a final round interview, because they’re more confident the candidate will be someone they want to hire.

“We’ve definitely seen an improvement in morale. Our engineers are some of the most talented individuals you’ll encounter, and it’s a clear value add for them and the broader team when they can dedicate more time to product launches, code reviews, and generally building the Ethos products,” said Sharma.

“The advantage we saw in Karat was our ability to look at the candidate’s code and evaluate technical skills, but also see the interview recordings, which give us a preview into the candidate’s communication skills and working style. This gives a very balanced view of the candidate, and it allows us to be as involved or hands-off as we’d like. This also helps with the quality checks as we can always see how candidates are being evaluated.”