Machine learning interviews test fundamentals (bias-variance, overfitting, evaluation, regularisation), algorithm understanding (linear models, trees, boosting, clustering), deep learning and increasingly LLMs, practical judgement about data and metrics, and for experienced roles, end-to-end ML system design. The 30 questions below, with concise answers, cover what is asked most.

Fundamentals

1. What is the bias-variance trade-off? Bias is error from overly simple assumptions (underfitting); variance is error from sensitivity to the training data (overfitting). More flexible models reduce bias but increase variance; the goal is the best total error on new data.

2. How do you detect and reduce overfitting? Training score much higher than validation score. Reduce it with more data, regularisation, simpler models, early stopping, dropout, data augmentation or better features.

3. Supervised vs unsupervised vs reinforcement learning? Labelled examples; structure without labels; learning from rewards through interaction.

4. Why split data into train, validation and test sets? Train fits parameters, validation selects models and hyperparameters, test gives a final unbiased estimate. See cross-validation explained.

5. What is data leakage? Information unavailable at prediction time influencing training, inflating scores, such as scaling on the full dataset or including post-outcome features.

6. L1 vs L2 regularisation? L1 adds absolute weights and can zero out features (sparsity); L2 adds squared weights and shrinks them smoothly.

7. Precision vs recall? Precision: of predicted positives, how many are correct. Recall: of actual positives, how many were found. See evaluation metrics explained.

8. Why is accuracy misleading on imbalanced data? Predicting the majority class scores high while missing the rare class that matters.

9. How do you handle imbalanced classes? Appropriate metrics (PR-AUC, F1), class weights, resampling, threshold tuning and collecting more minority examples.

10. What is cross-validation and when shouldn't you use random k-fold? Rotating validation folds for stable estimates; avoid random folds for time series or grouped data.

Algorithms

11. How does linear regression learn? By minimising squared error, via least squares or gradient descent. See linear regression explained.

12. Why use log loss for logistic regression? It is convex for the model, gives well-behaved gradients and rewards calibrated probabilities.

13. How does a decision tree choose splits? By maximising impurity reduction (Gini or entropy) or variance reduction for regression.

14. Random forest vs gradient boosting? Forests average independent deep trees to reduce variance; boosting adds shallow trees sequentially to correct errors, often more accurate with tuning.

15. How does k-means work, and its limits? Alternating assignment to nearest centres and updating centres; assumes round, similar-sized clusters and needs scaled features and a chosen k.

16. What does PCA do? Finds orthogonal directions of maximum variance to reduce dimensions; fit it on training data only.

17. How does k-NN work? Predicts from the majority (or average) of the k nearest training points; needs scaling and is slow at prediction time on large data without indexing.

18. What is the curse of dimensionality? In high dimensions data becomes sparse and distances less meaningful, hurting distance-based methods.

Deep learning and LLMs

19. What does backpropagation do? Computes gradients of the loss for every weight using the chain rule; optimisers then update the weights. See backpropagation explained.

20. Why do gradients vanish, and how is it fixed? Repeated multiplication of small derivatives through deep layers; fixed with ReLU-family activations, good initialisation, normalisation and residual connections.

21. What do batch norm and dropout do? Batch norm stabilises activations for faster training; dropout randomly disables neurons during training to reduce overfitting.

22. Why CNNs for images? Local filters with shared weights capture spatial patterns efficiently and recognise them anywhere.

23. What is self-attention? Each token forms a weighted mix of all tokens' values, with weights from query-key similarity; it's the core of transformers. See attention explained.

24. BERT vs GPT? Bidirectional encoder for understanding tasks vs left-to-right decoder for generation.

25. RAG vs fine-tuning? RAG grounds answers in retrieved documents and suits changing knowledge; fine-tuning changes behaviour, format or style. See LoRA fine-tuning.

26. How would you reduce LLM hallucinations? Retrieval with citations, validation, tool use for exact data, abstention, and evaluation sets.

Practical and system design

27. How would you design a fraud detection system? Define the decision and metric (recall at a reviewer capacity), build features from transaction history in near real time, start with gradient boosting, validate on later time periods, serve with low latency, route high scores to review, and monitor drift and feedback labels.

28. What would you do if a deployed model's performance dropped? Check data pipelines and input distributions for drift or bugs, compare with recent labels, look at slices, retrain on recent data if needed, and add monitoring that would catch it earlier.

29. How do you choose between a simple and a complex model? Compare against a baseline on held-out data, weigh accuracy gains against latency, cost, interpretability and maintenance.

30. How do you explain a model's prediction? Coefficients for linear models; permutation importance and SHAP for complex ones; examples of similar cases; always as association, not proof of cause.

Which areas do these questions cover?

The 30 questions above by area
Fundamentals10
Algorithms8
Deep learning and LLMs8
Practical and system design4

Counted from this list. Senior roles weight system design much more heavily.

How should you talk about your projects?

PartWhat to say
ProblemWho needed what, and the success metric
DataSource, size, cleaning, leakage risks you avoided
ApproachBaseline first, then what you tried and why
ResultsHeld-out numbers, compared with the baseline
DeploymentHow it's served or would be, and monitored
ReflectionWhat failed, what you'd change

What does a good ML system design answer include?

Take "design a course-recommendation system for learners". A strong answer clarifies the goal (more completed courses, not just clicks), the metric (completion rate of recommended courses, measured with an A/B test), and constraints (cold start for new learners, latency on the home page). It proposes data (enrolments, progress, ratings), a two-stage approach (fast candidate retrieval with embeddings, then a ranking model with richer features), offline evaluation with time-based splits, online serving with cached candidates, and monitoring for popularity bias and drift. It ends with what to build first: a simple popularity-plus-similarity baseline. Interviewers reward this structure far more than naming advanced models.

What red flags do interviewers look for?

  • Reporting only training accuracy, or accuracy on imbalanced data.
  • No baseline: you can't tell whether a complex model helped.
  • Not noticing obvious leakage in a described dataset.
  • Jumping to deep learning for small tabular problems.
  • Being unable to explain the algorithms used in your own projects.

Avoiding these, and saying "I'd check for leakage and compare with a simple baseline first", signals practical maturity quickly, and it is exactly how experienced practitioners approach a new problem at work, before any modelling starts.

How should you prepare?

Pair each concept with a small experiment you've run yourself, prepare two projects you can discuss for 15 minutes each, practise one ML system design question a week, and keep Python and SQL sharp. Program Zero covers these topics in order across the machine learning, deep learning and LLM courses, with an end-to-end project in every phase. See also SQL and Python interview questions.