0Pricing
AI Agents · Lesson

When Fine-Tuning Beats Prompting

Fine-tune for narrow tasks, custom formats, or to shrink prompts — not to teach new facts.

When Fine-Tuning Beats Prompting is a free AI Agents lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Prompting First, Fine-Tuning Second

Always exhaust prompting before reaching for fine-tuning:

  1. Better system prompt
  2. Few-shot examples
  3. Tool descriptions
  4. Better model
  5. Structured outputs / Pydantic
  6. RAG
  7. ... THEN fine-tuning

When Fine-Tuning Helps

  • Specific format / style — your bespoke YAML output language
  • Narrow domain — medical, legal, code reviews
  • Latency — short prompts (no few-shots) -> faster inference
  • Cost — shorter prompts -> fewer tokens billed
  • Consistent persona — character voice, brand tone

When Fine-Tuning Hurts

  • Teaching facts — RAG is better. Fine-tunes forget facts
  • General reasoning — frontier models already excel
  • Frequent updates — re-training is slow
  • Tiny datasets — < 100 examples rarely move the needle

Closed-Model Fine-Tuning

OpenAI and Anthropic both offer fine-tuning APIs:

# OpenAI:
client.fine_tuning.jobs.create(
    training_file='file-abc',
    model='gpt-4o-mini-2024-07-18'
)

Open-Model Fine-Tuning

For Llama/Mistral/Qwen, you control everything. Tools:

  • Unsloth — fastest, low VRAM
  • Axolotl — config-driven, comprehensive
  • LLaMA-Factory — Chinese-team alternative
  • HuggingFace TRL — official, flexible

Full Fine-Tune vs PEFT

Two approaches:

  • Full — update every weight. Costly (need huge VRAM).
  • PEFT / LoRA — train tiny adapter layers. 100x cheaper, similar quality for most tasks.

Cost of Fine-Tuning (OpenAI)

Approximate (mid-2025):

  • Training: $0.003 / 1K tokens
  • Inference on tuned model: somewhat more expensive than base
  • Tens of thousands of examples: $20-200 total training cost

Cost of Fine-Tuning (Self-Hosted)

One 8xH100 hour: ~$100 with Lambda Labs. A LoRA on Llama 70B with 10k samples: ~$50-150 total.

Data Volume Rule of Thumb

  • 50 examples — useful only for style / format
  • 500 examples — meaningful improvements
  • 5,000+ examples — substantial new capability
  • 50,000+ examples — diminishing returns relative to dataset quality

Where Does the Data Come From?

  1. Production traces (with corrections)
  2. Synthetic generation by a stronger model
  3. Human labelling
  4. Public datasets

Fine-Tune vs RAG vs Prompting Decision

GoalBest approach
New factsRAG
New formatFine-tune (or strong prompt)
New styleFine-tune
Better reasoningStronger model
Lower cost / latencySmaller model + fine-tune

Eval Before and After

Always benchmark the tuned model vs base on YOUR eval set. Sometimes the tune is worse — be ready to throw it away.

Best Reason to Fine-Tune

Which is the BEST reason to fine-tune?

Recap

Prompt first; fine-tune only for narrow format/style/domain reasons. Use PEFT/LoRA for cost. Always eval before and after.

Frequently asked questions

Is the “When Fine-Tuning Beats Prompting” lesson free?

Yes — the full text of “When Fine-Tuning Beats Prompting” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “When Fine-Tuning Beats Prompting”?

Fine-tune for narrow tasks, custom formats, or to shrink prompts — not to teach new facts. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “When Fine-Tuning Beats Prompting” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. When Fine-Tuning Beats Prompting
  2. Data Collection: Trajectories and Trace Replay
  3. LoRA and QLoRA for Cost-Efficient Tuning
  4. Evaluating Tuned Models vs Base
← Back to AI Agents