How to Hire an AI Engineer

Artificial intelligence (AI) is a rapidly growing area of technology, and the job market for AI engineers reflects this. Despite the overall market for tech talent trending downward, the number of AI job listings is up. Companies are ramping up hiring for AI engineers, paying a premium for talent, and shifting their resources to AI development. 

At the same time, companies are interested in adopting AI tools to increase operational efficiencies and gain a competitive advantage. There are several ways that AI can be used in the hiring process by recruiters, hiring teams, and even candidates, but companies have been cautious to incorporate AI into their hiring and interviews. 

This guide helps talent acquisition and engineering leaders understand AI engineering roles, identify and hire skilled AI engineers, combat bias that AI can introduce, and think through whether and how to use AI when hiring.

What Is an AI Engineer?

The most common job postings for AI engineers are seeking full-stack engineers to develop “tools, systems, and processes that enable artificial intelligence to be applied in the real world.” Full-stack software engineers are skilled in both front-end and back-end development. The front-end of an application refers to everything a user sees and interacts with, while the back-end consists of everything that users can’t see, like databases and servers. In order to handle both front-end and back-end development, full-stack engineers know front-end technologies such as HTML, CSS, JavaScript, as well as back-end programming languages like Python, Ruby, and PHP.

Since Python is the most popular AI programming language, full-stack engineers are well-equipped to build and train AI models. This also mirrors what we see in terms of candidate language preferences in Karat interviews. Other back-end programming languages that are often used for AI include Java, C++, and R. Full-stack engineers generally know how to work with big data and have strong problem-solving skills — two important qualities that are needed when working with AI.

In addition to developing and deploying AI models, AI engineers may also turn machine learning models into APIs, create and manage AI development and product infrastructure, and automate AI infrastructure used by data scientists. AI engineers are also responsible for prompt engineering, which is an area of AI that’s gotten a lot of buzz. Prompt engineering is the process of developing and optimizing prompts “to efficiently use language models (LMs) for a wide variety of applications and research topics.”

This describes the traditional AI software engineer role — one that’s focused on developing AI. However, we’re seeing a surge for generalist engineering roles that can work with AI and require strong fundamental engineering skills. This is because companies want engineers who can shift with the needs of the business, upskilling and adopting new technologies as needed. Limiting the definition of “AI engineer” to specialized AI jobs overlooks the fact that AI has an increasing role in the day-to-day jobs of all engineers. We break down AI engineers into four common generalist and specialist roles below.

4 Types of AI Engineers

Engineers Who Can Leverage AI as Part of Their Jobs

AI is unlikely to entirely replace software engineers, but it will change the nature of their jobs. Gartner estimates that 75% of enterprise software engineers will use AI code assistants by 2028, and 63% of organizations are currently piloting, deploying, or have already deployed AI code assistants. It’s clear that engineers are increasingly being expected to leverage AI to work more efficiently, helping companies reduce costs, improve employee retention, and go to market faster. 

With the increased use of AI, companies are also concerned about overreliance on it. To prevent this, companies also need to make sure that engineers understand the code being produced. When interviewing, this means incorporating AI tools into the technical interview or using code reviews to assess how the candidate analyzes AI-generated code. 

Engineers Who Train New AI Models

Developing new AI models is the most cutting-edge AI work being done today. There are only about 200 people in the world with real expertise in this, and they’re working for companies like OpenAI to do deep machine learning with massive datasets. 

Training AI models is “the process of feeding curated data to selected algorithms to help the system refine itself to produce accurate responses to queries.” This iterative process depends on both the quality of the input data and the ability of trainers. Organizations that are developing and updating their own proprietary models need people who understand the theory behind this work and have hands-on expertise. 

Machine Learning and Data Engineers

Outside of software engineers who can leverage AI tools, machine learning and data engineering are the biggest areas of growth we’re seeing in AI. While the two work together, machine learning uses data to develop AI models, and data engineering refers to the collection, storage, and preparation of data. 

Making sure the data that’s going into your models is clean and your outputs are tracking with your intended outcomes is critical for any organization’s AI efforts, so you’ll need both machine learning and data engineers to know if your AI is working.

Machine Learning Operations Engineers

Machine learning operations (MLOps) combines operations practices with machine learning in order to streamline deploying, maintaining, and monitoring machine learning models. MLOps engineers are responsible for building and maintaining the infrastructure that machine learning engineers use to develop and scale models, increasing reliability and reducing risk. To do this, MLOps engineers need to be knowledgeable about machine learning frameworks and models and have programming skills. They also need to be strong communicators and collaborative, as they work with data scientists, machine learning engineers, and other team members. 

Where to Recruit AI Engineers

Indeed reported that AI jobs make up 22% of job postings in the U.S. software development sector and that percentage is rapidly rising. As of March 2024, the share of AI jobs had almost reached its March 2022 peak. Demand is outstripping supply though — there are more than 10 job ads per AI professional in some states. This has led to candidates with little to no experience being hired for senior roles. 

Due to a skills shortage and high demand, companies that need AI engineering talent can’t simply put up a job listing and wait for candidates to come to them. Instead, they need to take a more proactive approach and even look internally for full-stack engineers who can learn how to become an AI engineer. Aside from searching for individuals on LinkedIn, there are several places where recruiters and hiring managers can look to find talented candidates: universities that specialize in AI, cities that are establishing themselves as AI hubs, and AI bootcamps. 

Universities Specializing in AI

Universities are keeping up with the boom in AI, making it an obvious place to recruit AI engineers. While some offer AI courses under their computer science department, there’s a growing number that now have four-year AI degrees. Others limit AI to graduate programs or minors. However, as this is a fast-developing area, how universities think about their AI programs will likely change in the coming years. 

To narrow down which universities you should look at, U.S. News & World Report ranks the best undergraduate and graduate AI programs in the United States. Across both lists, the following universities rank in the top 10:

• Carnegie Mellon University
• Stanford University
• Massachusetts Institute of Technology
• University of California, Berkeley
• Georgia Institute of Technology
• University of Illinois Urbana-Champaign
• University of Washington
• Cornell University

All of these universities don’t just excel in AI, but they’re also among our top 50 schools to hire all types of software engineers. 

While 32 out of the top 50 universities are located in the U.S., there’s plenty of talented AI students and graduates around the world. If your company is able to hire internationally, this expands your talent pool and potentially decreases the amount of competition you face from other companies. Consider these universities in Canada, Asia, Europe, and India that specialize in AI:

• University of Tokyo (Japan)
• Peking University (China)
• Shanghai Jiao Tong University (China)
• Technical University of Munich (Germany)
• University of Toronto (Canada)
• Indian Institute of Technology Madras (India)
• University of British Columbia (Canada)
• Nanyang Technological University (Singapore)

Cities That Have Emerged as AI Hubs

As universities are adapting to the rise of AI, cities across the world are racing to cement themselves as hubs for AI talent and innovation. There are some obvious locations such as San Francisco, New York, and Seattle. These cities are prominent tech hubs in the U.S. and some of the best cities in the world to hire software engineers, so they would also naturally become centers for AI. San Francisco employs 27% of AI professionals, New York employs 13%, and Seattle employs 9%. 

The U.S. continues to be the top destination for AI talent, but other countries like the UK, China, and India are actively working to cement their spot. DeepMind, the AI company owned by Google, has been located in London since 2019, and OpenAI established its first international office there too. In June 2023, the British Prime Minister pitched the UK “as a global center for artificial intelligence and regulation of the technology.” Prior to that, the government published a white paper on its plan for AI regulation. 

China is catching up to the U.S. in terms of AI talent. In 2019, Chinese AI researchers made up 10% of the most elite AI researchers. In 2022, this grew to 26%, while American researchers accounted for 28%. This is partly due to the expansion of AI programs across Chinese universities over the last three years, as well as the development of an AI industry that’s able to absorb all of that talent. The country has also set ambitious goals, seeking to become the world’s “primary” AI innovation center with a core AI industry gross output of $150.8 billion by 2030. 

India has emerged as a global tech hub and it’s now positioning itself to be a leader in AI too. Harvard Business Review ranked Bangalore, India as fifth for diversity among AI workers and the second-largest AI talent pool. Other cities like Hyderabad benefit from government support that can help foster innovation, drive AI job opportunities, and establish a thriving ecosystem for AI companies and startups.

Harvard Business Review’s top 50 global cities for AI talent include:

• San Francisco
• New York
• Boston
• Seattle
• Bangalore
• Los Angeles
• London
• Beijing
• Chicago
• Washington, DC

AI Bootcamps

Graduating from undergraduate or graduate AI programs isn’t the only way for people to gain AI engineering skills and knowledge. Several universities offer bootcamps, including Columbia Engineering, California Institute of Technology, and University of Houston. Some tech bootcamp companies also have AI and machine learning tracks now.

Coding bootcamps commonly partner with organizations to help their graduates get placed in a job, and this may extend to AI bootcamps as well. By partnering with bootcamps, you may be able to establish a steady candidate pipeline and be one of the first companies to access up-and-coming AI engineers.

How to Interview AI Engineers

Interviewing AI engineers is generally similar to interviewing any other type of software engineer. Conducting technical interviews involves establishing a standard interview process, designing interview questions and a scoring rubric, identifying and training interviewers, conducting the interview, and then evaluating the candidate to reach a hiring decision. This process is applicable to AI engineering roles, but some of the specific components need to be tailored to the specific job description. 

Technical Assessment

Technical assessments typically take the form of code tests, take-home assessments, and live technical interviews. These formats can be used for AI engineers, but the content of the assessment and what it tests for needs to be adjusted to evaluate the candidate’s knowledge and skills in AI and any subfields that are relevant to the job. This can include testing for coding proficiency or the ability to understand machine learning concepts.

Interview Questions and Scoring Rubric

Interview questions can introduce bias, so it’s extremely important to get them right. Good interview questions have a scoring rubric, test for ability rather than knowledge, and focus on dealbreakers. 

Common example interview questions ask candidates to explain a concept, technology, or method. For example, “Explain the difference between supervised and unsupervised learning” or “How can overfitting be prevented in an AI model?” However, these questions can be easily answered if the candidate has memorized information without truly understanding it.

You can turn knowledge questions into ability questions though by introducing a situation that the candidate has to analyze. Here are two examples to inspire your own interview questions:

• Come up with a relevant task where the candidate needs to train a model on data. Then, ask them whether they would use supervised or unsupervised learning to do so and why they selected that method. This requires the candidate to understand the right type of data needed and the relationship between the input data and the desired output, as well as to be familiar with what each learning technique is and when each should be applied.  
• Describe a scenario where the AI has been overfitted and ask the candidate to explain what they would do to correct this. By painting a picture of what has happened without explicitly saying that the model has been overfitted, this question helps identify which candidates can recognize the signs of overfitting. Additionally, asking for a solution tests for problem-solving skills and whether the candidate knows which technique to prevent overfitting is applicable in this scenario.

Don’t forget that it’s important for AI engineers to also have soft skills such as communication and teamwork. Aside from asking technical questions, see how the candidate has demonstrated soft skills that are important for the job. These example behavioral questions from Insight Global ask candidates to show that they’re team players, adaptable, and clear communicators:

• Give an example of a time you collaborated with an interdisciplinary team on an AI project.
• Describe a challenging project — AI or otherwise — you’ve worked on. What challenges did you run into, and how did you overcome them?
• Describe a time you had to describe complex AI concepts to a non-technical coworker or client. How did you approach it? 

Scoring rubrics ensure a fair and inclusive interview process by removing bias. With a rubric or scorecard, interviewers will ask similar questions to all candidates and compare candidates on the same criteria. Every rubric needs to be unique to the job’s role and responsibilities, so start by identifying the competencies that are relevant to the AI engineer role. Then, turn the competencies into a scale of observable behaviors that interviewers can look for and check off on the rubric.

Let’s go back to our example interview question above that asked the candidate to identify whether supervised or unsupervised learning would be the best approach to training a model for a specific task. Creating a great scoring rubric first requires you to figure out what matters when answering the interview question. In this case, we want:

• Candidates who can understand what is needed to train the model for the task described.
• Candidates who can accurately understand the difference between unsupervised and supervised learning.
• Candidates who can apply their understanding of the differences to identify the correct technique to use in the scenario.

From there, we can craft a rubric that describes what this specifically looks like:

• Award 1 point if the candidate correctly describes the difference between unsupervised and supervised learning.
• Award 1 point if the candidate identifies the correct method to use. 
• Award 1 point if the candidate clearly explains why the method is most appropriate for the scenario.

AI and Bias in Hiring

AI has great potential to make hiring more efficient and fair. In theory, it can be impartial and treat all candidates equally, but the downsides are that it can embed biases into algorithms and then scale those biases. Candidates are also wary of companies’ use of AI in hiring. The Pew Research Center found that a majority of Americans (71%) oppose AI making the final hiring decision. 

What’s particularly scary about this is that AI can embed bias in a way that’s not easy to catch. It can create a black box that makes hiring recommendations impossible to review or audit, allowing bias to embed itself into human decision-makers even after they stop using AI. Companies that want to use AI for hiring need to be very careful by being aware of these problems and knowing how to combat them. 

Focus on Competencies and Consistent Measurement

To address combat bias, companies need to make all components of their hiring process competency-based and create consistency in how they measure candidates. By focusing on skills-based hiring, you’ll ignore other identifiers, such as where the candidate went to school or the companies they’ve worked at, that can introduce bias and generate inaccurate hiring signals. 

Because AI scales biases, it really matters what data you put into your AI solution. For example, you can eliminate race, gender, and name from your model’s analysis, but if those factors have been historically correlated with success, those biases will show up in the hiring decision. Even if you train the AI model on previous top performers, any biases that led to hiring those candidates will show up in correlated variables. 

At Karat, one of the many things we do to reduce unconscious bias is not showing candidate resumes to Interview Engineers. The resume has no bearing on the type of interview that will be done, so there’s no reason to have Interview Engineers review them before interviewing the candidate. Showing the resume to them would only introduce pedigree bias. 

Ensure There’s a Human in the Loop

When there’s a human in the loop at the right time, this ensures that hiring decisions are made with both technology and humans, rather than solely with technology. For example, our Interview Engineers score interviews based on competencies and rubrics. They do not make the final recommendation about who passes or not because we don’t want to introduce likeability bias.

Keep in mind that recommendations are just recommendations. After getting the recommendation, humans still have to make the final decision. AI can actually help mitigate bias here because when one candidate scores higher than another, a human has to explain why they want to override the AI, and oftentimes, that’s because of bias. 

Similarly, if you use a human for a key part of the hiring process, you can have the AI check the human decision by scanning for discrepancies and initiate a human review to see if something happened that may not have been fair. 

Review Training Data and Test the Impact

The last way to prevent bias is to review the data that’s used to train your AI and test the impact of your hiring approach. You need to have training data that’s clean, employ diverse data sets that represent all candidate groups, and then constantly monitor the results. By consistently assessing the results, you can even detect biases outside of the interview. As an example, let’s say that equal numbers of men and women meet the hiring bar, but men advance to the next step at a higher rate. You can see this in your data, easily pinpoint the decision that caused this, and reverse it.

The History of AI Engineering

AI has come a long way since the idea of intelligent computers was first considered in 1950, and its development has gone through several ups and downs. Here’s a look at how AI has progressed to get to where we are today and notable milestones along the way.

The Birth of AI

The birth of AI happened in the early 1950s. Most famously, Alan Turing published his paper “Computing Machinery and Intelligence” in 1950, which considers the question, “Can machines think?” In 1952, a coder named Arthur Samuel created a program that could play checkers against a human. This was the first AI program written and run in the U.S. It wasn’t until 1956 that the term “artificial intelligence” was coined by John McCarthy when he hosted the first academic conference on it. He later went on to create LISP, the first AI programming language, in the 1960s. 

The AI Boom

The period from 1980 to 1987 marked the AI boom — a time where there was increased funding and rapid development and research. Deep learning techniques became popular; expert systems “which mimicked the decision making process of a human expert” were introduced; AI research and development projects received over $1 billion in funding; and many AI companies were started and funded by venture capital.

AI Winter

An AI winter refers to a period of decreased funding and interest. The AI winter between 1987 to 1993 was caused by setbacks in the machine market and expert systems. Expert systems require a lot of data. While computer storage increased, it was still extremely limited. The cost to develop AI systems was difficult to justify and few companies could afford to do so. The cost of developing AI systems also outweighed the business returns, as companies realized they could use less intelligent systems to achieve the same outcomes. Because expert systems couldn’t deliver on their promise, this led to a collapse in funding. 

AI Makes Significant Strides in the 1990s and 2000s

Despite the lack of funding and public interest from the AI winter, AI began to thrive in 1993 and many landmark goals were achieved over the next two decades. The chatbot A.L.I.C.E. was developed and had enough basic knowledge to hold a conversation with a human. IBM’s Deep Blue, a chess-playing computer program, beat the world chess champion in a highly publicized match. Speech recognition software, developed by Dragon Systems, was implemented on Windows, and Cynthia Breazeal created Kismet, the first robot that was able to recognize and display emotions.

AI in the Enterprise Today

Over the last 20 years, the language and image recognition capabilities of AI have advanced rapidly, and they’re now comparable to the capabilities of humans. AI can now generate content, including text, images, audio, and video. While AI is more powerful than ever and can be used for a variety of purposes, companies, especially enterprise companies, are being fairly cautious with deploying AI. 

Although a lot of companies are experimenting with AI, they’re not deploying it in consumer-facing products. It’s clear that AI is a strategic priority, with 89% of executives ranking AI and generative AI (GenAI) as a top-three tech priority for 2024 and 85% saying they’ll increase spending on AI and generative AI in 2024. However, “90% are either waiting for GenAI to move beyond the hype or experimenting in small ways.” 

We see clients are starting to leverage GenAI in order to drive efficiencies in software development and internal processes, but they’re all pretty cautious when it comes to customer-facing products — especially in regulated industries such as pharmaceuticals, healthcare, and financial services.

When it comes to applying AI to hiring and talent strategies, companies have also been cautious. There’s been some recalibration, but incorporating AI hasn’t immediately happened at most organizations. We’ve received many questions about using GenAI in interviews, but almost none have embedded it in their processes yet.

At Karat, we think that using AI in hiring and interviewing will happen eventually and the future of software engineering involves using AI on the job. Developers in 37% of organizations are already using generative AI to help with code generation or completion. We recommend mirroring real-life working conditions in your interviews when possible. If your engineers use AI in their jobs, this means allowing candidates to consult AI tools when interviewing. Regardless of whether your organization’s engineers currently use AI in their job, it’s important to build AI-enabled engineering interviews that account for new developments in GenAI and give you a confident signal on the candidate’s actual capabilities.

Skills Needed for  AI Engineering

When hiring for AI engineers, these are some of the hard and soft skills to look for in candidates: 

• Programming: Common programming languages used in AI development are Python, Java, R, and C++.  
• Big data: AI engineers not only work with large volumes of data, but they need to also be proficient in using big data tools like Hadoop, Apache Spark, and MongoDB.
• Mathematics and statistics: Developing AI involves calculating algorithms and an understanding of probability. Also, AI engineers use statistics calculus, linear algebra, and numerical analysis to predict how AI programs will run. 
• Machine learning: AI engineers need to be knowledgeable about machine learning models and algorithms, machine learning techniques, and how to evaluate the performance of these models. 
• Critical thinking and problem solving: Working with AI involves a lot of trial and error. AI engineers build prototypes, run and test them, make tweaks, and troubleshoot errors. 
• Communication: Because AI engineers often work cross-functionally, they need strong communication skills that enable them to communicate AI concepts to non-technical team members. 
• Teamwork: Many AI projects are team efforts that require AI engineers to work with other engineers as well as other teams. 
• Adaptability: Since the field of AI is constantly evolving and rapidly advancing, it’s important for AI engineers to be able to adapt as new techniques and technologies are developed. 

Don’t forget that instead of hiring AI engineers, you can also upskill existing engineers in your organization. There might be a surprisingly large number of current employees who could fill this role, as “more than 60% of a company’s future roles can be filled by current employees, assuming that adequate programs are in place.” 

Considering that 44% of employers’ top challenge to finding talent has been a lack of well-qualified applicants, upskilling might actually be a better option than hiring new talent. Upskilling has been proven to be easier and more cost-effective than hiring a new employee. Many of the skills listed above overlap with skills that software engineers have. Also, existing engineers are already familiar with your technical practices, processes, and infrastructure, and you know how they work within your team and company culture. 

One of the most convenient ways that software engineers can learn how to become an AI engineer is by gaining skills and experience on the job. This benefits both you and your employee, since the engineer can start contributing while they learn. Companies can also support employees in upskilling by offering a learning and development stipend that engineers can use on self-paced courses and bootcamps, or companies can even offer tuition assistance programs to help engineers pursue a degree in AI.

Examples of AI Engineers

As you build out your organization’s AI engineering team, you can look to some of the top AI leaders for inspiration and to see where AI is headed.

• Andrew Ng: Andrew Ng is one of the leading researchers in machine and deep learning. He co-founded DeepLearning.ai, an education technology company that’s “empowering the global workforce to build an AI-powered future through world-class education, hands-on training, and a collaborative community.” He was also the founding lead of Google Brain, a deep learning research team, and led Baidu’s AI Group, where he was responsible for driving the company’s global AI strategy and infrastructure.
• Geoffrey Hinton: Often called the godfather of AI, Geoffrey Hinton is best known for his work in artificial neural networks. He has been teaching at the University of Toronto for over three decades, and he worked at Google for over a decade as part of the Google Brain team before resigning in 2023 to be able to freely speak about the risks of AI.
• Fei-Fei Li: Fei-Fei Li is the inaugural Sequoia Professor in the Computer Science Department at Stanford University and Co-Director of Stanford’s Human-Centered AI Institute. She’s referred to as the godmother of AI and is best known for inventing ImageNet, a groundbreaking online image database that sparked the deep learning revolution. Most recently, it was announced that she’s building a spatial intelligence startup.
• Demis Hassabis: Demis Hassabis is the co-founder and CEO of DeepMind, one of the world’s leading AI labs that’s building general AI systems and was acquired by Google in 2014. DeepMind’s many inventions include AlphaGo, the first computer program to defeat a Go world champion. This achievement was considered to be a decade ahead of its time.
• Yann LeCunn: Currently the Chief AI Scientist at Meta and Silver Professor of Data Science, Computer Science, Neural Science, and Electrical Engineering at New York University, Yann LeCunn received the 2018 A.M. Turing Award for his work developing the convolutional neural network. This new approach recognized printed and handwritten text and was widely used to read numbers written on checks.

Resources for Hiring AI Engineers

While the specific skill sets, responsibilities, and interview questions for hiring AI engineers are different from hiring other types of software engineers, the process and best practices for interviewing and evaluating candidates are the same. In addition to this guide, Karat has many other resources to help you succeed in finding and identifying AI engineering candidates who are most likely to succeed in your organization: 

• Learn the ins and outs of technical recruiting and get plenty of tips in The Ultimate Guide to Technical Recruiting.
• Sharpen your hiring team’s interviewing skills with The Interviewer’s Playbook.
• Discover the best places to hire software engineers with our 2024 list of the top 30 cities around the world. 
• Learn about how AI is impacting technical interviews and what you should do about it in our guide on Interviewing in the Age of AI.

Conclusion

A shortage of skilled AI engineering candidates has created a highly competitive market that can make it difficult for companies to hire. This not only requires companies to take a more proactive approach in sourcing candidates, but it also means that companies should look internally at their existing software engineers to identify those who are strong at programming and problem-solving and can be upskilled. When hiring new talent, applying the fundamentals of technical recruiting and interviewing will lead to success: have a clear job description, establish a fair interview process that includes consistent interview questions and a scoring rubric, and remove as much bias as possible. 

Whether you’re hiring AI software engineers or other types of software engineers, it’s also clear that AI is having a growing role in hiring and interviewing processes. Companies need to decide whether candidates can use AI tools when interviewing, and they need to be aware of how AI can potentially introduce bias if they choose to incorporate it into their hiring strategy. Stay ahead of how AI is changing hiring in our Harnessing AI: The Future of Tech Hiring webinar with leaders from Okta and Glean.