When Fine-Tuning Beats Prompting
Fine-tune for narrow tasks, custom formats, or to shrink prompts — not to teach new facts.
Prompting First, Fine-Tuning Second
Always exhaust prompting before reaching for fine-tuning:
- Better system prompt
- Few-shot examples
- Tool descriptions
- Better model
- Structured outputs / Pydantic
- RAG
- ... THEN fine-tuning
When Fine-Tuning Helps
- Specific format / style — your bespoke YAML output language
- Narrow domain — medical, legal, code reviews
- Latency — short prompts (no few-shots) -> faster inference
- Cost — shorter prompts -> fewer tokens billed
- Consistent persona — character voice, brand tone
All lessons in this course
- When Fine-Tuning Beats Prompting
- Data Collection: Trajectories and Trace Replay
- LoRA and QLoRA for Cost-Efficient Tuning
- Evaluating Tuned Models vs Base