Project Deep-Dives & Behavioural Rounds for ML Engineers
The round that decides your level. How to tell a project story that survives ten follow-up questions, and how to answer 'tell me about a failure' without hurting yourself.
Covers: STAR method, project storytelling, failure stories, disagreement, quantifying impact, ambiguous problems
The behavioural round is usually where levelling is decided. Two candidates can have identical technical scores and land at different levels because one described doing tasks and the other described owning outcomes.
30-second answer
Structure it as: business context and why it mattered, your specific role and what you decided, the two or three hardest technical decisions with the alternatives you rejected, the measured outcome in business terms, and what you would do differently. Two minutes for the narrative, then let their questions steer the depth.
1 — Context and stakes (20 seconds)
What was the business problem and what did it cost? Quantify it. “Support agents were manually triaging 40,000 tickets a month; median first-response time was 14 hours and CSAT was falling.”2 — Your role, precisely (10 seconds)
Be exact about scope and honest about the team. “I owned the modelling and the serving path; a data engineer built the ingestion, and I worked with a PM on the routing policy.” Overclaiming is the fastest way to fail a deep-dive.3 — The hard decisions (60 seconds)
Two or three, each with the alternative you rejected and why. This is the substance of the answer. “I chose a two-stage funnel over a single model because the review team could only handle 200 cases a day, so precision@200 mattered more than aggregate AUC.”4 — Outcome in business terms (20 seconds)
Numbers, and the metric your company actually cared about. “Auto-resolution went from 0% to 34%, first-response time dropped to 2 hours, and we avoided hiring six additional agents — roughly $600k a year.”5 — What you would change (15 seconds)
Genuine reflection, not false modesty. “I'd build the evaluation set before the model next time. I spent three weeks tuning before I had a trustworthy way to measure, and two of those weeks were wasted.”
| Instead of | Say |
|---|---|
| “I built a churn model” | “I reduced voluntary churn by 8% by identifying at-risk accounts 30 days earlier” |
| “I used XGBoost” | “I chose gradient boosting over a neural net because we needed SHAP explanations for the compliance team” |
| “It had 0.87 AUC” | “At the threshold we shipped, we caught 71% of churners while flagging only 4% of accounts — matching the retention team's capacity” |
| “The team decided” | “I proposed X, the staff engineer pushed back on Y, and we settled on Z because…” |
| “It worked well” | “It ran for 14 months; we retrained monthly and it degraded about 2 points of AUC before each retrain” |
Say these points
- Lead with the problem and the decisions, not the chronology.
- Be precise about your scope and honest about the team's contributions.
- Two or three hard decisions, each with the rejected alternative.
- Close with business-metric impact and a genuine reflection.
- Only tell stories you can defend three follow-ups deep.
Avoid these mistakes
- A tools-and-steps narrative with no decisions in it.
- Overclaiming ownership — deep-dive questions expose it immediately.
- Quoting only offline metrics with no business outcome.
Expect these follow-ups
- What was your baseline, and how much of the lift came from the model versus better features?
- How did you know the improvement was causal?
- What would you have done with twice the time? With half?
30-second answer
Pick a real failure with real consequences that you were genuinely responsible for, explain the root cause honestly, describe what you did to detect and fix it, and — most importantly — what you changed *systemically* so it could not recur. The systemic change is what converts a failure story into evidence of seniority.
The failure, plainly
State what went wrong and what it cost. “Our demand forecasting model went live in March and by May the stockout rate had gone up, not down. It cost roughly two months of lost margin in the affected categories.”The root cause, honestly
Say what actually happened, including your part in it. “The store-level average sales feature was computed over the whole dataset including the target week. It was a leak I introduced. Offline MAPE was 4%; live it was 22%.”How it was caught
Be honest about whether you caught it or someone else did. “The ops team flagged it, not our monitoring — which was itself part of the problem.” Interviewers respect this; pretending you caught everything yourself is transparent.The immediate fix
Rollback, mitigation, communication. “We rolled back to the previous heuristic within a day, and I wrote up what happened for the ops team so they knew what to expect.”The systemic change — the point of the story
“I added a point-in-time correctness test to the feature pipeline that fails CI if any aggregate window extends past the prediction timestamp, and we started running a forward-only holdout on every model. That class of bug cannot ship silently now.”
| Good failure themes | Why they work |
|---|---|
| Data leakage discovered in production | Common, technical, and has a clean systemic fix |
| Model degraded silently due to upstream change | Shows you understand monitoring gaps |
| Optimised the wrong metric | Shows product judgement developed the hard way |
| Shipped without a rollback path | Very relatable; the fix is concrete |
| Over-engineered a solution nobody needed | Shows learned pragmatism |
| Miscommunicated model limitations to stakeholders | Shows growth in cross-functional work |
Say these points
- Pick a real failure with real consequences that you owned.
- State the root cause honestly, including your part in it.
- Be truthful about whether you or someone else caught it.
- The systemic fix is the point — fix the class of bug, not the instance.
- Prepare two: one technical, one judgement-based.
Avoid these mistakes
- A trivial failure that signals you have never owned anything.
- Blaming another team, even accurately.
- No systemic change — just 'I was more careful afterwards'.
Expect these follow-ups
- How would you have caught it earlier?
- What did you tell stakeholders while it was happening?
- Has that class of bug recurred since?
30-second answer
Pick a substantive disagreement, explain both positions fairly, describe how you tried to resolve it with evidence rather than opinion, and state the outcome honestly — including cases where you were wrong or where you disagreed and committed anyway. The quality of the resolution process matters far more than who turned out to be right.
Describe both positions fairly
Steelman theirs. “The staff engineer wanted to ship the simpler heuristic first. His argument was that we had no production feedback loop yet and a model would be unmeasurable — which was a good argument.” If you cannot articulate their reasoning convincingly, you did not really engage with it.Explain what was actually at stake
Not preference — consequence. “The difference was about six weeks of work and whether we'd be able to iterate after launch.”Describe how you tried to resolve it
Evidence beats assertion. “I proposed we spend three days building the offline evaluation harness first, because that would tell us whether the model could plausibly beat the heuristic before committing six weeks.” Converting a disagreement into a cheap experiment is the single most valuable move here.State the outcome honestly
Including if you lost. “The harness showed the heuristic was within 3% of what the model could achieve, so we shipped the heuristic. He was right, and I'd have burned six weeks.”Say what you learned
“Now I default to building the measurement before the solution, and I'm much quicker to ask 'what's the cheapest experiment that resolves this disagreement?'”
| Situation | The strong move |
|---|---|
| Disagreement on approach | Propose a cheap experiment that resolves it with data in days, not weeks |
| You lack the context they have | Ask what they know that you do not — often the disagreement dissolves |
| It's genuinely a judgement call with no data | Make the trade-off explicit, let the owner decide, and commit fully |
| You are overruled | Disagree and commit; write down your concern and the trigger that would change the decision |
| You were wrong | Say so explicitly and describe what updated your view |
Say these points
- Steelman the other position before explaining yours.
- Convert the disagreement into the cheapest experiment that resolves it.
- Stories where you were wrong or committed after losing are stronger than vindication stories.
- Pre-register what would change your mind, then commit fully.
- Never disparage the other person.
Avoid these mistakes
- Only telling stories where you were proven right.
- Escalating to a manager as the first step.
- Framing the other person as not understanding the problem.
Expect these follow-ups
- Have you ever changed your mind on something significant? What changed it?
- How do you handle it when you are overruled on something you believe is a real risk?
- How do you disagree with someone much more senior?
30-second answer
Work backwards from the decision the output will drive. Ask what action changes based on the model's answer, who takes it, and what happens today without one. That usually converts a vague request into a concrete prediction problem with a clear metric. Then propose the smallest thing that tests the hypothesis, rather than trying to specify the full system up front.
1 — Find the decision
“We want to use AI on our support tickets” becomes answerable once you ask what happens to a ticket after the model scores it. Auto-reply? Route to a team? Flag for a manager? Each implies a completely different system.2 — Find the current baseline
How is this decision made today? By whom, how fast, how accurately, at what cost? This gives you both the bar to beat and the metric everyone already understands.3 — Find the cost of being wrong, in each direction
This determines your metric and threshold, and it is a question stakeholders can always answer even when they cannot specify requirements.4 — Find the constraints
Latency, volume, explainability, privacy, budget, and who maintains it. Constraints eliminate most of the solution space quickly, which is a good thing.5 — Propose the smallest testable version
“Let me spend a week labelling 300 tickets by hand and running a simple baseline. That will tell us whether this is a 60%-accuracy problem or a 95% one, and we can decide whether to invest properly.” A cheap, time-boxed reality check beats a six-week spec.
SCOPING_QUESTIONS = {
"decision": [
"What action changes based on this output?",
"Who takes that action, and how quickly do they need it?",
"What happens today when there is no model?",
],
"value": [
"How often is this decision made?",
"What does a wrong decision cost, in each direction?",
"What would a 10% improvement be worth?",
],
"data": [
"What data exists today, and who owns it?",
"Do we have labels, or would we need to create them?",
"How long until we learn whether a decision was correct?",
],
"constraints": [
"Latency budget? Volume? Peak?",
"Does anyone need to understand WHY it decided that?",
"Privacy, regulatory or contractual limits?",
"Who maintains this in six months?",
],
"success": [
"What number, measured how, would make this a success?",
"What would make us shut it down?",
],
}
# The last question is the most useful and the least asked. If nobody can
# name a shutdown condition, the project has no accountability and will
# consume effort indefinitely.Say these points
- Ask what decision changes based on the output — it derives every requirement.
- Establish today's baseline: who decides, how fast, how well, at what cost.
- Ask the cost of being wrong in each direction to derive metric and threshold.
- Propose the smallest time-boxed test rather than a full specification.
- Be willing to say ML is the wrong tool.
Avoid these mistakes
- Accepting a vague brief and building for months without a defined decision.
- Over-engineering to cover hypothetical future requirements.
- Never asking what would make the project a failure.
Expect these follow-ups
- Tell me about a time you talked a stakeholder out of using ML.
- How do you prioritise when three teams all want models from you?
- How do you set expectations about model accuracy before you have built anything?
30-second answer
Start by taking the specific case seriously rather than defending the model. Investigate the individual prediction, explain the drivers in their language using feature attributions, be explicit about what the model can and cannot do, and convert the complaint into either a fix or a monitored metric. Never open with 'the model is 94% accurate' — that answers a question they did not ask.
1 — Take the case seriously and look at it
“Let me pull up that specific prediction and see what drove it.” This single action converts an adversarial conversation into a collaborative one, and it is usually informative — individual complaints frequently reveal genuine systematic problems.2 — Explain the drivers in their vocabulary
Use SHAP or equivalent, then translate. Not “the SHAP value for tenure was −0.31” but “the biggest factor pushing this account toward high-risk was that they had three failed payments in the last month, which historically precedes cancellation about 40% of the time.”3 — Be explicit about what the model is and is not
“This model estimates probability from historical patterns. It has no knowledge that this customer just signed a new contract, because that data isn't in the system yet. That is a genuine limitation and it explains this case.” Honesty about limits builds far more trust than defensiveness.4 — Give them a path to act
An override mechanism, a feedback button, a documented escalation. Stakeholders tolerate imperfect models far better when they are not powerless in front of them.5 — Close the loop
“I've added this to our error review. If we see this pattern more than a handful of times I'll treat it as a feature gap rather than noise.” Then actually follow up — that is what converts a complaint into trust.
| Instead of | Say |
|---|---|
| “The model is 94% accurate” | “Let me look at this specific case with you” |
| “It's a black box” | “Here are the three factors that drove this prediction” |
| “That's just how ML works” | “This is a real limitation — here's why it happened and what we can do” |
| “The data said so” | “The model learned this pattern from historical cases; here's whether that generalises to yours” |
| “We'd need to retrain” | “Here's the fastest path: a rule for this case now, a data fix in the next cycle” |
Say these points
- Never open with aggregate accuracy — investigate the specific case.
- Translate feature attributions into domain language, not SHAP values.
- Be explicit and honest about the model's limitations.
- Provide an override path and close the feedback loop.
- Set expectations and show failure modes before launch, not after.
Avoid these mistakes
- Defending the model instead of examining the case.
- Explaining in ML vocabulary the stakeholder does not share.
- Promising a fix without checking whether it is feasible.
Expect these follow-ups
- How would you explain a false positive in a fraud system to the affected customer?
- How do you push back when a stakeholder demands 100% accuracy?
- How do you communicate model uncertainty in a dashboard?
Showing 5 of 5 questions for ml-behavioral-and-project-deep-dive.
Check your understanding
5 questions · no sign-up, nothing stored
Hands-on challenge
Build it — this is what you talk about in a deep-dive round.
Build your interview story bank
Prepare and stress-test the stories you will actually use, in writing.
Requirements
- Write up three projects using the five-beat structure, each under 250 words, with quantified impact.
- For each, list ten follow-up questions an interviewer could ask and write your answers.
- Prepare two failure stories — one technical, one judgement-based — each ending in a systemic change.
- Prepare one disagreement story where you were wrong and one where you disagreed and committed.
Stretch goals
- Record yourself answering each out loud and time it; the main narrative should be under two minutes.
- Have a peer play interviewer and drill three layers deep on each story; note where you ran out of substance.