Logistic regression is a classification model. It computes a weighted sum of the inputs, like linear regression, then passes it through the sigmoid function to produce a probability between 0 and 1, such as "73% likely to be spam". It is trained by minimising log loss, and a decision threshold turns the probability into a yes/no decision.

Why can't you use linear regression for classification?

Suppose you want to predict whether a student will complete a course (1) or drop out (0). Linear regression could output 1.4 or −0.3, which make no sense as probabilities, and a few extreme examples would tilt the whole line. Logistic regression fixes both problems by mapping every output into the 0–1 range with an S-shaped curve, so predictions behave like probabilities and extreme inputs saturate rather than dominate.

How does the sigmoid work?

The model computes z = b + w₁x₁ + … + wₙxₙ, then the probability p = 1 / (1 + e−z). When z is 0 the probability is exactly 0.5; large positive z pushes p towards 1; large negative z towards 0.

Sigmoid output for different values of z
z = −40.02
z = −20.12
z = 00.50
z = 20.88
z = 40.98

Computed with p = 1 / (1 + e^−z).

The value z is the log-odds: the logarithm of p ÷ (1 − p). That's why logistic regression is described as "linear in the log-odds".

How is logistic regression trained?

It minimises log loss (binary cross-entropy). For a true positive example, the loss is −log(p): tiny if the model predicted 0.99, large if it predicted 0.01. For a negative example it's −log(1 − p). Confident wrong answers are punished very heavily, which pushes the model towards well-calibrated probabilities. There's no closed-form solution, so it is trained with gradient descent or related optimisers. The same cross-entropy loss trains image classifiers and, at a much larger scale, language models predicting the next token.

How do you interpret the coefficients?

Each weight is the change in log-odds for a one-unit increase in that input, holding others fixed. Taking ew gives an odds ratio. If the weight on "weekly practice hours" is 0.3, then e0.3 ≈ 1.35: each extra hour of practice multiplies the odds of completing the course by about 1.35, other things equal. Odds ratios are standard in medical and social research, and they make logistic regression one of the most explainable classifiers, which matters in lending and other regulated decisions.

How do you choose the decision threshold?

The model outputs probabilities; your application decides what to do with them. The default 0.5 threshold is rarely the right business choice.

Use caseCostlier mistakeThreshold tends to be
Fraud screeningMissing fraudLower (flag more for review)
Spam filterHiding a real emailHigher (only very likely spam)
Medical screeningMissing a caseLower, with follow-up tests
Marketing offerWasting budgetSet by expected return per contact

Lowering the threshold increases recall (you catch more positives) and usually lowers precision (more false alarms). Plot precision and recall across thresholds and choose with the people who own the decision. Our guide to model evaluation metrics explains these trade-offs in detail.

What about imbalanced classes?

If only 1% of transactions are fraud, a model that always says "not fraud" is 99% accurate and completely useless. With imbalanced data: judge models by precision, recall, F1 and the precision-recall curve rather than accuracy; use class weights (class_weight='balanced' in scikit-learn) so mistakes on the rare class count more; and use stratified splits so every fold contains rare cases.

How does logistic regression handle several classes?

One-vs-rest trains one binary model per class ("is it A or not?") and picks the most confident. Multinomial logistic regression uses the softmax function to output a probability for each class that sums to 1. Softmax is exactly the final layer of most neural network classifiers and of language models choosing the next token from their vocabulary, another reason this "simple" model is worth understanding deeply.

What preprocessing does it need?

  • Scale numeric features (standardisation), especially with regularisation.
  • One-hot encode categories; drop one level or rely on regularisation to avoid redundancy.
  • Handle missing values explicitly; consider a "was missing" indicator column.
  • Add interactions or transforms if the log-odds aren't linear in an input.
  • Regularise (L2 by default in scikit-learn) to prevent overfitting with many features.

When should you use logistic regression, and when not?

StrengthsLimitations
Fast to train and predict, even on large dataOnly linear decision boundaries without feature engineering
Probabilities are often well calibratedStruggles with complex interactions that trees capture automatically
Coefficients are explainable as odds ratiosSensitive to highly correlated features when interpreting
A strong, honest baseline for any classification taskUsually beaten by gradient boosting on complex tabular data

A good workflow: start with logistic regression as a baseline, then try random forests and gradient boosting. If the complex model only beats the baseline by a hair, the simpler, more explainable model may be the better product choice.

What does a worked example look like?

Suppose a trained model predicts course completion with z = −3 + 0.3 × (weekly practice hours) + 1.2 × (attended orientation, 1 or 0). A learner who practises 6 hours a week and attended orientation gets z = −3 + 1.8 + 1.2 = 0.0, so p = 0.50. With 10 hours, z = 1.2 and p ≈ 0.77. Without orientation and with 4 hours, z = −1.8 and p ≈ 0.14. The model's structure makes these numbers easy to explain to a non-technical colleague: practice and orientation both raise the odds, and you can say by how much.

What is calibration, and why does it matter?

A model is well calibrated if, among all the cases where it predicts 70%, about 70% turn out positive. Calibration matters whenever probabilities drive decisions: pricing risk, ranking leads, or deciding how many reviewers to staff for flagged cases. Logistic regression is often reasonably calibrated out of the box because log loss rewards honest probabilities, while many tree ensembles and neural networks need recalibration (Platt scaling or isotonic regression). Check it with a reliability plot that groups predictions into bins and compares the predicted with the observed rate in each.

How is logistic regression asked about in interviews?

Common questions: why is it called regression if it classifies (it regresses the log-odds); why use log loss instead of squared error (log loss is convex for this model and punishes confident mistakes, giving better probabilities); how do you handle imbalanced classes; and how do you choose a threshold. Strong candidates also explain the link to softmax and neural networks, and say when they'd prefer a tree ensemble. Being able to compute one prediction by hand, as in the worked example above, is a quick way to show you understand the model rather than only the library call.

How should you practise?

Build a spam or loan-default classifier, report precision and recall at three thresholds, and explain your top three coefficients as odds ratios. Then compare against a tree-based model. Classification, evaluation and tuning are all part of Program Zero's machine learning course; read linear regression explained first if you haven't.