To fine-tune an LLM affordably, use LoRA or QLoRA: keep the base model frozen and train small adapter weights on a few hundred to a few thousand high-quality examples of the behaviour you want. Fine-tune for consistent format, tone or task behaviour; use retrieval (RAG) for knowledge. Always evaluate against a held-out set before and after.

When is fine-tuning the right choice?

GoalBest first approach
Answer from your company documentsRAG (retrieval), not fine-tuning
Always output a strict JSON format or styleFine-tuning (after trying good prompts)
Classify or extract in a narrow domainFine-tuning a smaller model is often cheaper at scale
Teach new, changing factsRAG; fine-tuned facts go stale
Follow a brand voicePrompting first, fine-tuning if it is inconsistent

How do LoRA and QLoRA work?

Full fine-tuning updates every weight, which needs a lot of GPU memory and produces a full-size copy of the model. LoRA freezes the original weights and learns two small low-rank matrices for selected layers; at inference their product is added to the frozen weights. QLoRA does the same on a base model whose frozen weights are stored in 4-bit precision, so bigger models fit on a single GPU. The adapters are small files you can swap per task.

What are the steps to fine-tune an LLM?

  1. Define the task and write 30–50 test cases you will judge by.
  2. Choose a base model sized for your hardware and licence needs.
  3. Prepare data: consistent prompt/response pairs in the model's chat format; remove duplicates and errors.
  4. Configure LoRA: rank, alpha, target modules, learning rate, epochs.
  5. Train with a validation split; watch for overfitting.
  6. Evaluate the base and fine-tuned models on the same held-out tests.
  7. Serve the adapter with the base model, or merge them, and monitor in use.

What mistakes should you avoid?

  • Fine-tuning to add knowledge that RAG would handle better.
  • Noisy, inconsistent training examples: the model learns the noise.
  • No baseline: you can't tell if fine-tuning helped.
  • Training too long on a small dataset, so the model forgets general skills.

Where does Program Zero teach this?

In Phase 8 you fine-tune BERT with Hugging Face, then LoRA and QLoRA, and the phase project is a LoRA fine-tune of an open LLM alongside a small GPT you train from scratch. Phase 9 adds alignment (SFT, DPO) and quantization, and Phase 10 deploys the model. For the retrieval side, read what is RAG.