0Pricing
AI Agents · Lesson

LoRA and QLoRA for Cost-Efficient Tuning

Train only low-rank adapters on a quantized base — fits a 70B fine-tune on a single GPU.

LoRA and QLoRA for Cost-Efficient Tuning is a free AI Agents lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Why PEFT?

Full fine-tuning of Llama 70B needs 8+ H100s and updates 70 billion parameters. PEFT (Parameter-Efficient Fine-Tuning) updates < 1% of parameters with similar quality. Massive cost savings.

LoRA: Low-Rank Adaptation

LoRA (Hu et al. 2021) trains tiny "adapter" matrices alongside frozen base weights:

  • Add A x B matrices where A is dxr and B is rxd, r is small (8, 16, 64)
  • Forward pass: y = Wx + (BA)x
  • Only train A and B — orders of magnitude fewer parameters

QLoRA: Quantised LoRA

QLoRA (Dettmers et al. 2023) further reduces cost by quantising the base model to 4-bit during training:

  • Base model in NF4 (4-bit)
  • LoRA adapters in higher precision
  • Fits a 70B fine-tune on a single A100/H100

Setup with peft + transformers

# pip install peft transformers bitsandbytes
from peft import LoraConfig, get_peft_model
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    'meta-llama/Llama-3.1-8B-Instruct',
    load_in_4bit=True,        # QLoRA
    device_map='auto'
)

lora_config = LoraConfig(
    r=16,                     # rank
    lora_alpha=32,
    target_modules=['q_proj', 'v_proj'],
    lora_dropout=0.05,
    bias='none'
)
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# trainable params: ~0.5% of total

Training Loop with TRL

from trl import SFTTrainer

trainer = SFTTrainer(
    model=model,
    train_dataset=dataset,
    tokenizer=tokenizer,
    args=TrainingArguments(
        per_device_train_batch_size=4,
        gradient_accumulation_steps=4,
        learning_rate=2e-4,
        num_train_epochs=3,
        output_dir='./lora-out'
    )
)
trainer.train()
model.save_pretrained('./adapter')

Using Unsloth

Unsloth is the fastest LoRA library — 2x faster than vanilla peft. Drop-in API:

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name='meta-llama/Llama-3.1-8B-Instruct',
    load_in_4bit=True
)
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=['q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj']
)

Cost Estimate

QLoRA on Llama 3.1 8B with 10k examples:

  • ~4 hours on a single H100
  • ~$5-10 in cloud GPU cost
  • Adapter file: 100-300 MB

Choosing Rank r

  • r = 8 — minimal, format/style changes
  • r = 16-32 — most cases
  • r = 64+ — substantial new behavior, more capacity

Target Modules

Default to attention projections (q, v). For more capacity, include all linear layers — better quality, more parameters.

Hyperparameters That Matter

  • Learning rate — 1e-4 to 5e-4 is typical
  • Epochs — 2-5 for SFT
  • Batch size — depends on VRAM; use grad accumulation to fake bigger

Merging Adapter Into Base

For inference, you can merge the LoRA back into the base weights:

merged = model.merge_and_unload()
merged.save_pretrained('./full-model')

Serving Multiple LoRAs

vLLM supports hot-swapping LoRAs at inference time. Useful for per-customer fine-tunes on shared base:

llm = LLM(model='base', enable_lora=True, max_lora_rank=64)
response = llm.generate(prompt, lora_request=LoRARequest('customer-a', 1, './customer-a-adapter'))

Eval the Adapter

Always run YOUR eval set on the tuned model. Sometimes the tune is worse on edge cases — be honest about regressions.

LoRA Advantage

What's the main practical advantage of LoRA over full fine-tuning?

Recap

LoRA + QLoRA = cost-efficient fine-tuning. Unsloth for speed, peft/TRL for breadth. Rank 16-32 for most cases. Eval against your gold set.

Frequently asked questions

Is the “LoRA and QLoRA for Cost-Efficient Tuning” lesson free?

Yes — the full text of “LoRA and QLoRA for Cost-Efficient Tuning” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “LoRA and QLoRA for Cost-Efficient Tuning”?

Train only low-rank adapters on a quantized base — fits a 70B fine-tune on a single GPU. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “LoRA and QLoRA for Cost-Efficient Tuning” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. When Fine-Tuning Beats Prompting
  2. Data Collection: Trajectories and Trace Replay
  3. LoRA and QLoRA for Cost-Efficient Tuning
  4. Evaluating Tuned Models vs Base
← Back to AI Agents