To fine-tune an LLM affordably, use LoRA or QLoRA: keep the base model frozen and train small adapter weights on a few hundred to a few thousand high-quality examples of the behaviour you want. Fine-tune for consistent format, tone or task behaviour; use retrieval (RAG) for knowledge. Always evaluate against a held-out set before and after.
When is fine-tuning the right choice?
| Goal | Best first approach |
|---|---|
| Answer from your company documents | RAG (retrieval), not fine-tuning |
| Always output a strict JSON format or style | Fine-tuning (after trying good prompts) |
| Classify or extract in a narrow domain | Fine-tuning a smaller model is often cheaper at scale |
| Teach new, changing facts | RAG; fine-tuned facts go stale |
| Follow a brand voice | Prompting first, fine-tuning if it is inconsistent |
How do LoRA and QLoRA work?
Full fine-tuning updates every weight, which needs a lot of GPU memory and produces a full-size copy of the model. LoRA freezes the original weights and learns two small low-rank matrices for selected layers; at inference their product is added to the frozen weights. QLoRA does the same on a base model whose frozen weights are stored in 4-bit precision, so bigger models fit on a single GPU. The adapters are small files you can swap per task.
What are the steps to fine-tune an LLM?
- Define the task and write 30–50 test cases you will judge by.
- Choose a base model sized for your hardware and licence needs.
- Prepare data: consistent prompt/response pairs in the model's chat format; remove duplicates and errors.
- Configure LoRA: rank, alpha, target modules, learning rate, epochs.
- Train with a validation split; watch for overfitting.
- Evaluate the base and fine-tuned models on the same held-out tests.
- Serve the adapter with the base model, or merge them, and monitor in use.
What mistakes should you avoid?
- Fine-tuning to add knowledge that RAG would handle better.
- Noisy, inconsistent training examples: the model learns the noise.
- No baseline: you can't tell if fine-tuning helped.
- Training too long on a small dataset, so the model forgets general skills.
Where does Program Zero teach this?
In Phase 8 you fine-tune BERT with Hugging Face, then LoRA and QLoRA, and the phase project is a LoRA fine-tune of an open LLM alongside a small GPT you train from scratch. Phase 9 adds alignment (SFT, DPO) and quantization, and Phase 10 deploys the model. For the retrieval side, read what is RAG.