Beginners should learn about ten machine learning algorithms well: linear and logistic regression, k-nearest neighbours, decision trees, random forests, gradient boosting, support vector machines, naive Bayes, k-means and PCA. Together they cover prediction, classification, grouping and simplification of data, and they teach every core idea deep learning later builds on.
What does each algorithm do?
| Algorithm | Type | In one line | Use when |
|---|---|---|---|
| Linear regression | Supervised | Fits a straight-line relationship to predict a number | Predicting prices, demand, scores |
| Logistic regression | Supervised | Estimates the probability of a class | Yes/no outcomes; a strong baseline |
| k-nearest neighbours | Supervised | Predicts from the most similar past examples | Small datasets, simple similarity |
| Decision tree | Supervised | Asks a sequence of yes/no questions | When you need an explainable model |
| Random forest | Supervised | Averages many trees to reduce errors | Robust default on tabular data |
| Gradient boosting (XGBoost, LightGBM) | Supervised | Adds trees that fix earlier mistakes | Often top accuracy on tabular data |
| Support vector machine | Supervised | Finds the widest boundary between classes | Medium-sized, well-separated data |
| Naive Bayes | Supervised | Applies probability with a simple independence assumption | Fast text classification |
| k-means | Unsupervised | Groups similar points into k clusters | Customer or product segments |
| PCA | Unsupervised | Compresses many features into a few | Visualising or simplifying data |
What goes wrong most often?
- Overfitting: great on training data, poor on new data. Use validation sets and cross-validation.
- Data leakage: information from the future or the test set sneaks into training.
- Wrong metric: accuracy can mislead on imbalanced data; consider precision, recall and F1.
- No baseline: always compare a fancy model with a simple one.
Where to learn these properly
Program Zero's Phase 6 covers NumPy, Pandas and EDA, supervised learning (regression, trees, SVM), unsupervised learning (k-means, PCA), ensembles (XGBoost, LightGBM) and model evaluation and tuning, ending in an end-to-end deployed project. For a study plan, see the month-by-month machine learning plan.