Transfer learning reuses a model already trained on a large dataset (such as millions of images or billions of words) as the starting point for your own task. You either freeze the pretrained layers and train only a new output layer (feature extraction), or update some or all layers with a small learning rate (fine-tuning). It lets you build accurate models with far less data, time and compute, and it is the default way deep learning is applied today, from image classifiers to fine-tuned language models.

Why does transfer learning work?

Deep networks learn general features in their early layers and task-specific ones in later layers. An image model trained on a broad dataset learns edges, textures and shapes that are useful for almost any image task. A language model trained on vast text learns grammar, word meanings and facts that help with almost any language task. Starting from these learned features means your model only has to learn what's specific to your problem, such as which leaf patterns mean a particular crop disease, or how your customers phrase complaints.

How much does it save?

Training a large image or language model from scratch can require millions of labelled examples and large amounts of GPU time. Fine-tuning the same model for a new task often needs a few hundred to a few thousand examples and minutes to hours on a single GPU. For most teams, the choice is not between transfer learning and training from scratch; it's between transfer learning and not using deep learning at all.

What are the two main approaches?

ApproachWhat is trainedBest whenRisk
Feature extractionOnly a new head on frozen featuresVery little data; task similar to pretrainingCeiling on accuracy
Partial fine-tuningHead plus the last few layersModerate data; somewhat different taskSome overfitting
Full fine-tuningAll layers, small learning rateMore data; task differs moreOverfitting, forgetting general knowledge
Parameter-efficient (e.g. LoRA, adapters)Small added modules; base frozenLarge models; limited GPU memorySlightly lower ceiling than full fine-tuning in some tasks

How do you decide what to freeze?

Two questions guide the choice: how much labelled data you have, and how similar your task is to the pretraining data.

  • Little data, similar task (classifying common objects): freeze everything, train the head.
  • Little data, different task (medical scans): freeze early layers, fine-tune later ones carefully, augment heavily.
  • Lots of data, similar task: fine-tune all layers.
  • Lots of data, different task: fine-tune everything; pretrained weights still help convergence.

What does a typical vision example look like?

  1. Load a ResNet or EfficientNet pretrained on ImageNet.
  2. Replace its final classification layer with one sized for your classes (say, five leaf diseases).
  3. Freeze the backbone and train the new head for a few epochs.
  4. Unfreeze the last block and continue with a learning rate about ten times smaller.
  5. Use data augmentation and early stopping on a validation set.
  6. Evaluate per class and look at misclassified images.

With a few hundred images per class, this approach often reaches accuracy that training a CNN from scratch would need many times more data to match.

What does transfer learning look like in NLP?

Pretrained encoders such as BERT are fine-tuned for classification, named-entity recognition or question answering by adding a small task head and training on labelled examples. Decoder LLMs are adapted with instruction tuning or with LoRA and QLoRA for specific formats, tones or domains. The Hugging Face Transformers library makes loading and fine-tuning these models routine. For Indian languages, starting from multilingual pretrained models is especially valuable because labelled data in each language is limited.

Which practical tips matter most?

  • Match preprocessing to what the pretrained model expects (image size and normalisation, or the same tokenizer).
  • Use lower learning rates for pretrained layers than for the new head (discriminative learning rates).
  • Warm up the learning rate at the start of fine-tuning to avoid disrupting good weights.
  • Watch for catastrophic forgetting: aggressive fine-tuning can erase general abilities.
  • Keep a baseline: compare with feature extraction and a simple non-deep model.
  • Check licences: pretrained weights come with terms of use, especially for commercial products.

When does transfer learning fail?

When the source and target domains are very different and the pretrained features don't apply (for example, natural-image features applied to very unusual sensor data), fine-tuning may help little, a situation called negative transfer. Biases in the pretraining data can also carry over into your model. Always validate on data that reflects your real users, and compare with a model trained from scratch when you have enough data to do so.

How does this connect to foundation models?

Large pretrained models that many tasks build on are often called foundation models. Prompting, retrieval and fine-tuning are all ways of transferring their knowledge to a new task, from lightest-touch to deepest. Understanding classic transfer learning (what's frozen, what's trained, why learning rates matter) makes LLM fine-tuning much less mysterious.

How do you choose a pretrained model?

Consider four things. Task fit: a model pretrained on data like yours (images, multilingual text, code) transfers better. Size: larger models usually transfer better but cost more to fine-tune and run; start with a small or medium model and scale up only if results demand it. Licence: make sure commercial use is permitted if you need it. Community and documentation: well-used models have known pitfalls, examples and evaluation results, which saves time. Model hubs list these details; read the model card before downloading.

How does transfer learning help Indian-language AI?

Labelled datasets in most Indian languages are small compared with English, so training strong models from scratch for each language is rarely practical. Multilingual pretrained models, trained on text from many languages together, learn representations that are partly shared across languages, so fine-tuning on a few thousand labelled examples in Bengali, Marathi or Tamil can work surprisingly well, and knowledge sometimes transfers from higher-resource languages to related lower-resource ones. Government and research efforts that create open Indian-language datasets and models make this transfer more effective for everyone building in these languages.

What mistakes do people make with transfer learning?

  • Using a large learning rate for pretrained layers, which destroys useful features in the first few steps.
  • Forgetting to put the model in evaluation mode at inference, so dropout and batch-norm statistics behave differently.
  • Mismatched inputs: wrong image size, colour order or tokenizer.
  • Overfitting a tiny dataset by unfreezing everything at once.
  • Evaluating on images or text too similar to the training set, which hides poor generalisation.

A simple safeguard against most of these is to run the pretrained model on a few examples before fine-tuning and check its outputs make sense. If the untouched model gives nonsense on your inputs, the preprocessing is wrong, and no amount of fine-tuning will fix that. Then fine-tune in small steps, checking validation results after each change.

How should you practise?

Fine-tune a pretrained image model on a small dataset with feature extraction, then partial fine-tuning, and compare. Then fine-tune a small BERT model for text classification. Transfer learning appears in Program Zero's deep learning course and again, for language models, in the generative AI and LLM course, where you fine-tune with Hugging Face and LoRA.