Industry Trends & Research
Webinar insights: harnessing the power of genAI in tech hiring
The Karat Team
The recent webinar, “Harnessing AI and the future of tech hiring,” featured an enlightening discussion about the future of generative AI, or genAI, in software development and engineering hiring. Moderated by Scott Bonneau, EVP of Product and Operations at Karat, the webinar brought together industry leaders Sagnik Nandy, CTO at Okta, and Eddie Zhou, Founding Engineer at Glean, to share their insights on the applications, challenges, and implications of incorporating genAI into engineering hiring and software development processes. This article recaps the highlights from the webinar, shedding light on how generative AI is poised to reshape the landscape of engineering and beyond.
Understanding genAI’s role in software development
The conversation kicked off with a succinct introduction to generative AI, highlighting its ability to process vast amounts of information and simulate human-level reasoning.
Both panelists delved into their organizations’ experiences with integrating generative AI into practical workloads. Sagnik Nandy discussed pilot programs leveraging tools like Copilot and context-driven information to streamline onboarding and operational efficiency. Eddie Zhou shared how his team used their own product, Glean, to accelerate tasks, particularly in templated or batch projects, highlighting AI’s role as a significant accelerant.
Nandy also noted that today’s generative AI systems can serve as the backbone for agents that mimic human reasoning, making them more advanced than language models of the past. This fundamental difference is paving the way for innovative applications across various domains, including software development.
Balancing AI-specific roles with foundational skills
A noteworthy poll question presented to the 300-plus webinar attendees revealed a balanced split regarding hiring for AI-specific roles. This result underscored the ongoing evolution of AI in the engineering sector and raised questions about how AI’s role intersects with traditional skill sets.
In fact, Eddie Zhou emphasized his team’s focus on hiring strong software engineers and thinking about machine learning and AI engineering as supplemental skills they need on top of strong fundamentals — which he assesses without the use of AI coding tools.
“All of our ML engineers are all software engineers in title. Everyone is held to the same software engineering bar. We also have additional AI-specific interviews including ML systems design or ML foundations,” shared Zhou. “For us, those are additive skill sets, and that’s because we’re focused on hiring horizontal ML engineers who are capable of building both the model and systems that incorporate those models. This horizontal persona can be a lot more flexible in terms of finding people that can integrate intelligence into a product. So from a hiring space, AI and ML is very similar to traditional hiring with an additional set of evaluation that involves a little more statistical and mathematical foundations in addition to the problem solving and algorithm side.”
Nandy highlighted the need to evaluate candidates’ depth of AI knowledge and problem-solving skills, especially in scenarios that demand thoughtful design and creativity.
“We have subject matter experts who go a lot deeper, but also AI-aware generalists. People who are injecting intelligence and machine learning into the software development process,” Nandy said. “For that group, we’re making sure people can deliver what they claim to. Everyone claims to have some form of AI experience today, and filtering out how relevant and real it is. AI can be more of an art. It’s easy to get programming feedback in terms of will the code work versus how do you solve a feature problem where the outcome wasn’t what you expected. Teasing that out to understand the level of depth is something we are injecting into the process.”
Human interviewers can assess problem-solving beyond genAI
While acknowledging the utility of AI tools like Copilot for day-to-day tasks, Zhou shared that Glean discourages the use of ChatGPT during interviews. Instead, the company focuses on assessing candidates’ core skill sets without relying on AI. This is done through human interactions, rather than any kind of experimental AI-detection software. This also echoes what Karat researchers have found in their test interviews with genAI tools like ChatGPT.
Rather than analyzing static code samples, Zhou stressed a human-plus-technology approach that involves coaching interviewers to increase the frequency of their check-ins with candidates to probe their approach and problem-solving abilities, ensuring a more accurate evaluation of candidates’ true capabilities.
“Ultimately that process isn’t as difficult as it sounds,” noted Zhou. “The reality is having a human probing how candidates are solving problems at any kind of depth will be very apparent. It’s just like knowing if a candidate has seen a problem before and is just regurgitating an answer. We’ve been able to acknowledge it and continue hiring at a healthy pace.”
The future of genAI and software development
The webinar concluded with a forward-looking perspective on the future of genAI in software development. The panelists acknowledged the ongoing evolution of AI technology and the potential for disruption and innovation. They emphasized the need to stay adaptable and reevaluate strategies regularly, given the rapid pace of change in this dynamic field.
As AI continues to advance, it’s clear that embracing its potential while maintaining a balanced approach will be key to realizing its benefits in the engineering landscape and beyond. The full webinar is accessible on demand.
The conversation continues
Join us for Part Two of our series on genAI and the future of tech hiring, featuring Jocelyn Lai, Global Head of Talent Acquisition at Duolingo; Chad MacRae, VP of Talent Acquisition at Tinder; and Elena Stefanopol, Director of Recruiting at Labelbox on September 13. Sign up for the September 13 event today.
Last week the Karat team had the privilege of meeting with the finalist for the In-House Recruitment’s High-Growth Company Award. We sat down with some of the best leaders in the business and discussed how they’re tackling the opportunities and challenges of hiring in 2023. Hiring collaborative problem solvers One finalist who has seen quite […]
Industry Trends & Research
It may be the spookiest day of the year for those celebrating Halloween, but something else has been haunting the tech industry for quite some time: the technical hiring process. Job seekers looking for new opportunities in 2023 have had some pretty eerie experiences — experiences rife with scary strategies, terrifying tactics, and ancient rituals […]
Industry Trends & Research
The tech hiring landscape continues to shift as the world’s workforce navigates global economic uncertainty. Fortunately, these shifts create opportunities for software developers and engineers. The Bureau of Labor Statistics predicts the job market for these professionals remains resilient with an expected 26% increase in employment over the next decade. Furthermore, Karat’s 2023 Hiring Trends […]