Beginners should learn about ten machine learning algorithms well: linear and logistic regression, k-nearest neighbours, decision trees, random forests, gradient boosting, support vector machines, naive Bayes, k-means and PCA. Together they cover prediction, classification, grouping and simplification of data, and they teach every core idea deep learning later builds on.

What does each algorithm do?

AlgorithmTypeIn one lineUse when
Linear regressionSupervisedFits a straight-line relationship to predict a numberPredicting prices, demand, scores
Logistic regressionSupervisedEstimates the probability of a classYes/no outcomes; a strong baseline
k-nearest neighboursSupervisedPredicts from the most similar past examplesSmall datasets, simple similarity
Decision treeSupervisedAsks a sequence of yes/no questionsWhen you need an explainable model
Random forestSupervisedAverages many trees to reduce errorsRobust default on tabular data
Gradient boosting (XGBoost, LightGBM)SupervisedAdds trees that fix earlier mistakesOften top accuracy on tabular data
Support vector machineSupervisedFinds the widest boundary between classesMedium-sized, well-separated data
Naive BayesSupervisedApplies probability with a simple independence assumptionFast text classification
k-meansUnsupervisedGroups similar points into k clustersCustomer or product segments
PCAUnsupervisedCompresses many features into a fewVisualising or simplifying data

What goes wrong most often?

  • Overfitting: great on training data, poor on new data. Use validation sets and cross-validation.
  • Data leakage: information from the future or the test set sneaks into training.
  • Wrong metric: accuracy can mislead on imbalanced data; consider precision, recall and F1.
  • No baseline: always compare a fancy model with a simple one.

Where to learn these properly

Program Zero's Phase 6 covers NumPy, Pandas and EDA, supervised learning (regression, trees, SVM), unsupervised learning (k-means, PCA), ensembles (XGBoost, LightGBM) and model evaluation and tuning, ending in an end-to-end deployed project. For a study plan, see the month-by-month machine learning plan.