How to interview an AI engineer in 2026
Model fluency is table stakes. Here is how we assess judgement, evaluation rigour and systems thinking in modern AI hires.
Two years ago, an AI engineering interview was mostly a machine learning theory quiz. Today the strongest candidates rarely differentiate themselves on theory at all — they differentiate on judgement. They know when a retrieval pipeline is the right answer and when a fine-tune is a very expensive way to avoid writing a prompt.
Start with an evaluation-first exercise. Give the candidate a vague product goal, such as 'summarise support tickets so agents can triage faster' and ask them to define what good looks like before they touch a model. Great engineers immediately reach for a labelled sample, a baseline and a metric. Weaker ones start naming models.
Probe failure modes next. Ask how they would detect silent regressions after a model provider ships an update, how they would handle prompt injection in a tool-using agent and what they would log to make an incident debuggable at 3am. The specificity of these answers is the single best signal we have found.
Then test systems thinking. AI features live inside real products: caching, latency budgets, cost ceilings, streaming responses, graceful degradation when the provider is down. A candidate who has genuinely shipped will describe these constraints unprompted.
Finally, keep a short, humane coding segment. Not a puzzle — a small, realistic task in their own editor with their own tools, including AI assistance. Watching how someone directs a model is now part of watching how they work.
We run this loop in three conversations across five days. It consistently surfaces the people who make products better, not just the people who can recite an architecture diagram.