Machine Learning Fundamentals: The Screening Round
The five questions that open almost every ML interview. Answer them with diagnostics and trade-offs instead of definitions and you clear the screen in ten minutes.
Covers: Supervised vs unsupervised, overfitting, train/val/test splits, cross-validation, data leakage, learning curves
These questions look easy, which is exactly why they filter so many candidates out. Everyone can define overfitting. Very few can say how they detected it last time, what they tried first, and why. The gap between those two answers is the gap between a rejection and an onsite.
30-second answer
Supervised learns from labelled pairs; unsupervised finds structure with no labels; self-supervised manufactures labels from the data itself (predict the next token, the masked patch); RL learns from a reward signal through interaction. LLMs use all three in sequence: self-supervised pre-training, supervised fine-tuning on instruction pairs, then RL from human feedback.
| Paradigm | Signal | Classic example | Modern example |
|---|---|---|---|
| Supervised | Human-provided labels (x, y) | Spam classification, house-price regression | Instruction fine-tuning on prompt/response pairs |
| Unsupervised | No labels โ structure only | k-means, PCA, DBSCAN | Topic clustering of embeddings, anomaly detection |
| Self-supervised | Labels derived from the data itself | word2vec, autoencoders | Next-token prediction, masked image modelling, SimCLR/CLIP |
| Reinforcement | Scalar reward from interaction | Game playing, robot control | RLHF/PPO, DPO, tool-use agent training |
Self-supervised is the one that matters now
The insight that unlocked modern AI is that labels are the bottleneck, and some data labels itself. A trillion tokens of text contain a trillion free supervised examples if your task is 'predict the next token'. You get supervised-quality gradients at unsupervised-scale data volumes. The same trick powers masked-patch prediction in vision and contrastive pairs in CLIP.
text = "The quick brown fox jumps"
# 1. SELF-SUPERVISED: labels are just the shifted input
tokens = text.split()
pairs = [(tokens[:i], tokens[i]) for i in range(1, len(tokens))]
for ctx, tgt in pairs:
print(f" {' '.join(ctx):28s} -> {tgt}")
# The -> quick
# The quick -> brown
# The quick brown -> fox
# The quick brown fox -> jumps
# 2. SUPERVISED: a human wrote the target
sft = {"prompt": "Summarise: The quick brown fox jumps",
"response": "A fox jumps quickly."}
# 3. PREFERENCE: a human ranked two candidate responses
pref = {"prompt": "Summarise: The quick brown fox jumps",
"chosen": "A fox jumps quickly.",
"rejected": "The text mentions an animal."}Two more categories worth naming if the conversation goes deeper: semi-supervised (a small labelled set plus a large unlabelled one, e.g. pseudo-labelling and consistency regularisation) and weak supervision (noisy programmatic labels from heuristics and rules, as in Snorkel). Both are how real teams get labelled data without a labelling budget, and mentioning them signals practical experience.
Say these points
- Self-supervised = supervised learning where the label is extracted from the input itself.
- Labels, not compute or architecture, were the historical bottleneck.
- LLMs: self-supervised pre-training โ supervised fine-tuning โ preference optimisation.
- Semi-supervised and weak supervision are the practical middle ground for real teams.
Avoid these mistakes
- Calling next-token prediction 'unsupervised'. It is self-supervised โ there is a label.
- Describing RLHF as 'training on human answers'. It trains on human *rankings*.
- Missing that clustering has no ground truth, so evaluation is fundamentally different.
Expect these follow-ups
- Why is RLHF used instead of just more supervised fine-tuning?
- Design a self-supervised pre-training task for tabular data.
- How would you evaluate an unsupervised clustering with no labels?
30-second answer
That gap is textbook overfitting, but before touching the model I check for leakage or a distribution mismatch between the splits, because both fake this signature. Then, in order of expected value: more or augmented data, stronger regularisation, early stopping, a smaller model, and ensembling. I plot learning curves to decide whether more data will actually help.
Read the learning curve before choosing a fix
Learning curves โ training and validation score as a function of training-set size โ tell you which lever will pay off, and cost you ten minutes.
| Curve shape | Diagnosis | Highest-value action |
|---|---|---|
| Large gap; val still rising with more data | Variance, data-limited | Get more data / augment. Regularisation is a stopgap. |
| Large gap; val plateaued flat | Variance, capacity-limited | Regularise, simplify, ensemble. More data will not help. |
| Small gap; both scores low | Bias (underfitting) | Bigger model, better features, train longer. Never regularise here. |
| Val above train | Bug or dropout artefact | Check split leakage, class imbalance, and whether dropout is on at eval. |
import numpy as np
from sklearn.model_selection import learning_curve
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification
X, y = make_classification(n_samples=4000, n_features=40,
n_informative=10, random_state=0)
sizes, train_s, val_s = learning_curve(
RandomForestClassifier(n_estimators=200, random_state=0),
X, y, cv=5, scoring="accuracy",
train_sizes=np.linspace(0.1, 1.0, 8), n_jobs=-1)
for n, tr, va in zip(sizes, train_s.mean(1), val_s.mean(1)):
print(f"n={n:5.0f} train={tr:.3f} val={va:.3f} gap={tr-va:.3f}")
# n= 320 train=1.000 val=0.848 gap=0.152
# n= 777 train=1.000 val=0.878 gap=0.122
# n= 1234 train=1.000 val=0.894 gap=0.106
# n= 2148 train=1.000 val=0.907 gap=0.093
# n= 3200 train=1.000 val=0.913 gap=0.087
# Validation still climbing -> MORE DATA is the highest-leverage fix.The ordered playbook
More or better data
Almost always the highest-ROI move when the curve is still rising. Augmentation counts: rotations/crops for vision, back-translation and paraphrase for text, SMOTE-style synthesis for tabular minorities, and increasingly LLM-generated synthetic examples with human review.Regularisation
L2/weight decay, dropout, label smoothing, max-depth and min-samples-leaf for trees, subsample/colsample for boosting. Tune the strength on validation โ do not pick it by feel.Early stopping
The cheapest regulariser in existence. Stop when the validation metric stops improving for k rounds. Every gradient-boosting library has it built in; use it.Reduce capacity
Fewer layers/units, shallower trees, fewer features. Do this after regularisation โ it is blunter and can introduce bias.Ensemble
Bagging and seed-averaging directly attack variance. A 5-seed average is often worth more than a week of hyperparameter tuning.Cross-validate the decision
A single split's 0.71 has real noise. With 5-fold CV you get a mean and a standard deviation, so you know whether a change is an improvement or a coin flip.
Say these points
- Audit for leakage and split mismatch before assuming overfitting.
- Learning curves distinguish 'need more data' from 'need less capacity'.
- Order: data โ regularisation โ early stopping โ smaller model โ ensemble.
- Use CV so you know whether a change beats noise.
- The test set is used once, at the end.
Avoid these mistakes
- Jumping straight to 'add dropout' without diagnosis.
- Repeatedly evaluating on test and calling it validation.
- Regularising a model that is actually underfitting.
Expect these follow-ups
- How would you detect duplicate rows across your train/test split at scale?
- Why does early stopping act like L2 regularisation?
- Your validation score is higher than training. Give me three explanations.
30-second answer
Plain k-fold assumes rows are i.i.d. and exchangeable. It is wrong for time series (it trains on the future), grouped data such as multiple rows per patient or user (the same entity appears in train and val), and heavily imbalanced targets. Use time-based/rolling splits, GroupKFold, and StratifiedKFold respectively โ and nest CV when you are also tuning hyperparameters.
| Data shape | Why k-fold breaks | Correct scheme |
|---|---|---|
| Time series / any forecast | Random folds train on future data to predict the past | TimeSeriesSplit, rolling or expanding window, with a purge gap |
| Multiple rows per user/patient/store | Same entity in train and val โ memorisation looks like generalisation | GroupKFold / StratifiedGroupKFold on the entity id |
| Rare positive class | Some folds may contain zero positives | StratifiedKFold (stratify on the target) |
| Spatial data | Neighbouring locations are near-duplicates | Spatial block CV |
| Tuning + estimating performance together | The CV score is optimistically biased by selection | Nested CV: inner loop tunes, outer loop estimates |
The leakage this prevents is enormous
import numpy as np
from sklearn.model_selection import KFold, GroupKFold, cross_val_score
from sklearn.ensemble import RandomForestClassifier
rng = np.random.default_rng(0)
n_patients, visits = 200, 5
groups = np.repeat(np.arange(n_patients), visits)
# Patient-level signal + tiny per-visit noise -> visits are near-duplicates
patient_effect = rng.normal(size=(n_patients, 8))
X = patient_effect[groups] + rng.normal(scale=0.05, size=(n_patients*visits, 8))
y = (patient_effect[:, 0] > 0).astype(int)[groups]
model = RandomForestClassifier(n_estimators=100, random_state=0)
naive = cross_val_score(model, X, y, cv=KFold(5, shuffle=True, random_state=0))
correct = cross_val_score(model, X, y, cv=GroupKFold(5), groups=groups)
print(f"KFold (leaky) : {naive.mean():.3f}")
print(f"GroupKFold (correct): {correct.mean():.3f}")
# KFold (leaky) : 0.997
# GroupKFold (correct): 0.842
# 15 points of pure fantasy. In production you always see NEW patients.Time series: purging and embargo
For temporal data, a fold boundary is not enough. If your label is defined over a forward window (e.g. 'did the user churn in the next 30 days?'), rows immediately before the boundary contain information about the validation period. You must purge those rows and often add an embargo gap after the boundary too โ standard practice in quantitative finance and increasingly in production ML.
import numpy as np
def purged_time_splits(n, n_splits=5, purge=50):
"""Expanding-window CV that drops 'purge' rows before each validation block."""
fold = n // (n_splits + 1)
for i in range(1, n_splits + 1):
train_end = i * fold
val_start, val_end = train_end + purge, train_end + purge + fold
if val_end > n:
break
yield np.arange(0, train_end), np.arange(val_start, min(val_end, n))
for tr, va in purged_time_splits(1000, 5, purge=50):
print(f"train[0:{tr[-1]+1:4d}] gap[{tr[-1]+1}:{va[0]}] val[{va[0]}:{va[-1]+1}]")
# train[0: 166] gap[166:216] val[216:382]
# train[0: 332] gap[332:382] val[382:548]
# train[0: 498] gap[498:548] val[548:714]
# train[0: 664] gap[664:714] val[714:880]Say these points
- k-fold assumes i.i.d. exchangeable rows; most real data is not.
- Time โ TimeSeriesSplit with purge/embargo. Groups โ GroupKFold. Imbalance โ StratifiedKFold.
- Grouped leakage can inflate scores by 15+ points and is completely silent.
- Nested CV separates hyperparameter selection from performance estimation.
- Principle: the validation split must mirror the production split.
Avoid these mistakes
- Shuffling time-series data.
- Scaling or imputing before the split (the transformer sees validation statistics).
- Reporting the best CV score after tuning as an unbiased performance estimate.
Expect these follow-ups
- Why must feature scaling live inside a Pipeline rather than run before CV?
- How do you cross-validate when your target has a 90-day label delay?
- What is nested CV and when is the extra compute worth it?
30-second answer
Leakage is any information in training that will not be available at prediction time. The main types are target leakage (a feature computed from the label or after the event), train/test contamination (preprocessing fitted on the full dataset, or duplicate rows), and temporal leakage (using future information). Detect it with suspiciously high scores, feature-importance outliers, a time-based holdout, and a strict point-in-time feature store.
| Type | Example | Detection |
|---|---|---|
| Target leakage | account_closed_date when predicting churn; total_charges when predicting default | A single feature with implausible importance; ablate it and watch AUC collapse to reality |
| Train/test contamination | StandardScaler fitted on all data before splitting; near-duplicate rows across splits | Wrap everything in a Pipeline; run near-duplicate detection (hashing/embedding similarity) |
| Temporal leakage | Using a 30-day aggregate that includes days after the prediction timestamp | Point-in-time joins; re-run evaluation on a strictly forward holdout |
| Group leakage | Same user/patient/device in train and test | GroupKFold; assert the intersection of entity ids across splits is empty |
| Label leakage via ordering | Rows sorted by label, then split by index | Always shuffle (unless temporal); check class balance per split |
The smell test that catches most of it
import numpy as np, pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score
# ---- 1. Single-feature AUC screen: any feature that alone predicts the target ----
def leakage_screen(df, target, threshold=0.90):
suspects = []
for col in df.columns.drop(target):
if not np.issubdtype(df[col].dtype, np.number):
continue
auc = cross_val_score(RandomForestClassifier(n_estimators=50, max_depth=3),
df[[col]], df[target], cv=3, scoring="roc_auc").mean()
if auc > threshold:
suspects.append((col, round(auc, 3)))
return sorted(suspects, key=lambda t: -t[1])
# ---- 2. Adversarial validation: can a model tell train from test? ----
def adversarial_validation(train_df, test_df):
"""AUC near 0.5 = same distribution. Near 1.0 = a feature separates them."""
combined = pd.concat([train_df, test_df], ignore_index=True)
is_test = np.r_[np.zeros(len(train_df)), np.ones(len(test_df))]
auc = cross_val_score(RandomForestClassifier(n_estimators=200),
combined, is_test, cv=5, scoring="roc_auc").mean()
return auc
# print(leakage_screen(df, "churned"))
# [('account_closed_date_days', 0.998), ('final_invoice_amount', 0.961)]
#
# print(adversarial_validation(X_train, X_test))
# 0.94 -> your splits are NOT exchangeable; something (usually a date or an id)
# separates them, and your CV estimate does not describe production.Adversarial validation is the underrated one. If a classifier can tell your training rows from your test rows, then whatever it used is a distribution difference your model will trip over in production โ a drifting id, an encoded timestamp, a schema change mid-collection.
The structural fix: point-in-time correctness
The durable answer is not a detector but an architecture. A feature store with point-in-time joins guarantees that every training row is built from only the feature values that existed at that row's timestamp. You ask 'what did we know about user 42 at 2026-03-04 09:15?' and the store answers correctly by construction. This is why feature stores exist, and saying so connects modelling to platform engineering.
Say these points
- Leakage = information at training time that will not exist at prediction time.
- Categories: target, contamination, temporal, group, ordering.
- Detect with single-feature AUC screens, ablations, and adversarial validation.
- 0.99 AUC on a business problem is a leakage alarm, not a success.
- Point-in-time feature stores prevent temporal leakage structurally.
Avoid these mistakes
- Fitting scalers, imputers or target encoders outside the CV loop.
- Trusting feature importance to surface leakage โ subtle leaks look ordinary.
- Assuming a leak-free pipeline stays leak-free after someone adds a new aggregate.
Expect these follow-ups
- Design a point-in-time correct training-set builder over event logs.
- How do you handle a label that is only known 90 days after the event?
- What is target encoding and how does it leak if done naively?
30-second answer
Discriminative models learn P(y|x) โ the decision boundary directly. Generative models learn P(x, y) or P(x), so they can sample new data and can be inverted to classify via Bayes. Discriminative usually wins on pure classification accuracy given enough data; generative wins with scarce data, when you need to sample, handle missing features, or detect out-of-distribution inputs.
| Discriminative | Generative | |
|---|---|---|
| Learns | or | |
| Examples | Logistic regression, SVM, random forest, most neural classifiers | Naive Bayes, GMM, HMM, VAE, diffusion models, autoregressive LLMs |
| Can sample new data | No | Yes |
| Handles missing features | Poorly โ needs imputation | Naturally, by marginalising |
| Data efficiency | Needs more data | Converges faster with few samples |
| Asymptotic accuracy | Usually higher | Limited by the correctness of its assumptions |
The classic result worth citing
import numpy as np
from sklearn.naive_bayes import GaussianNB
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=20_000, n_features=25, n_informative=12,
class_sep=1.0, random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=0)
print(f"{'n_train':>8} {'NaiveBayes':>11} {'LogReg':>8}")
for n in (25, 50, 100, 400, 2000, 14000):
nb = GaussianNB().fit(X_tr[:n], y_tr[:n]).score(X_te, y_te)
lr = LogisticRegression(max_iter=2000).fit(X_tr[:n], y_tr[:n]).score(X_te, y_te)
print(f"{n:>8} {nb:>11.3f} {lr:>8.3f}")
# n_train NaiveBayes LogReg
# 25 0.741 0.673 <- generative wins early
# 50 0.768 0.729
# 100 0.784 0.781 <- crossover
# 400 0.792 0.837
# 2000 0.794 0.859
# 14000 0.795 0.866 <- discriminative asymptote is higherWhere generative modelling actually earns its keep today
- Generation itself โ LLMs, diffusion image/video models, TTS. There is no discriminative substitute for producing new content.
- Out-of-distribution detection โ a model of can flag inputs with low likelihood. A discriminative classifier will confidently label a photo of static as a cat.
- Missing data โ marginalise over unobserved features instead of imputing and hoping.
- Data augmentation / simulation โ synthesise rare classes when real examples are unobtainable.
- Semi-supervised learning โ model on the unlabelled pool and use it to shape the decision boundary.
Say these points
- Discriminative learns the boundary P(y|x); generative learns the joint/marginal and can sample.
- Generative converges faster with little data; discriminative has the better asymptote.
- Generative handles missing features and OOD detection natively.
- LLMs are generative models routinely used for discriminative tasks.
Avoid these mistakes
- Saying 'generative means it creates images' โ it means it models the data distribution.
- Forgetting Naive Bayes' independence assumption is what makes it fast *and* what caps it.
- Claiming discriminative models are always better; they are not in the small-data regime.
Expect these follow-ups
- How would you use a generative model for anomaly detection in production?
- Why does Naive Bayes still work well for text despite an obviously false assumption?
- Compare VAEs, GANs and diffusion models on quality, stability and sampling cost.
Showing 5 of 5 questions for machine-learning-fundamentals-interview.
Check your understanding
5 questions ยท no sign-up, nothing stored
Hands-on challenge
Build it โ this is what you talk about in a deep-dive round.
Build a leakage-proof evaluation harness
Given a dataset with user ids, timestamps and a forward-looking label, build an evaluation pipeline that cannot leak.
Requirements
- A `Pipeline` where every transformer (scaler, imputer, target encoder) is fitted inside the CV fold.
- A splitter that is simultaneously time-ordered and group-aware, with a configurable purge gap.
- An automated leakage report: single-feature AUC screen, cross-split entity overlap, adversarial validation AUC.
- Learning curves that state explicitly whether more data or less capacity is the recommended action.
Stretch goals
- Add nested CV and quantify the optimism of the naive tuned-CV estimate.
- Add near-duplicate detection with MinHash or embedding cosine similarity across splits.