AI Engineering from Scratch: The Free 503-Lesson Curriculum With 41,900+ GitHub Stars — From Linear Algebra to Autonomous AI Agents

If you've ever felt like you're using AI without understanding it, you're not alone. A recent survey found that 84% of students already use AI tools — but only 18% feel prepared to use them professionally. The gap between "I can prompt ChatGPT" and "I can build, debug, and ship an AI system" is enormous.

AI Engineering from Scratch is the curriculum designed to close that gap — completely, for free, and from the ground up.

Created by Rohit Ghumare and a community of 500+ contributors, this open-source project has exploded to 41,900+ GitHub stars in just four months. It's not a collection of blog posts or YouTube links. It's a structured, linear curriculum that takes you from the mathematical foundations of AI all the way to building production-grade autonomous agent systems.

And every single lesson ends with something you can actually use — a prompt template, a Claude/Cursor skill, a deployable agent, or an MCP server you built by hand.

What Makes This Curriculum Different

Most AI learning resources teach in fragments. A paper here, a fine-tuning tutorial there, a flashy agent demo somewhere else. The pieces rarely connect. You can ship a chatbot but can't explain its loss curve. You can hook a function to an agent but can't describe what attention does inside the model calling it.

AI Engineering from Scratch is the spine that connects all those pieces:

• 503 lessons organized into 20 progressive phases
• ~320 hours of structured content
• 4 programming languages: Python, TypeScript, Rust, and Julia
• Every algorithm built from raw math first — backpropagation, tokenization, attention, agent loops
• MIT licensed — free forever, no paywall, no premium tier

The philosophy is simple: you don't truly understand something until you can build it from scratch. By the time PyTorch shows up in the curriculum, you already know what it's doing under the hood — because you wrote the smaller version yourself.

The 20-Phase Learning Path: From Math to Multi-Agent Swarms

The curriculum is organized as a dependency graph, not a flat list. Each phase builds on the previous ones:

Foundation Layer (Phases 0–2)

Start with dev environment setup, math foundations (linear algebra, calculus, probability, information theory), and ML fundamentals (regression, classification, clustering, evaluation). Every concept is implemented in code before moving on.

# Phase 1: Building gradient descent from scratch
def gradient_descent(f, df, x0, lr=0.01, epochs=1000):
    """Minimize f starting from x0 using gradient descent."""
    x = x0
    history = [x]
    for _ in range(epochs):
        grad = df(x)
        x = x - lr * grad
        history.append(x)
    return x, history

# Minimize f(x) = x^2 + 3x + 2
x_min, path = gradient_descent(
    f=lambda x: x**2 + 3*x + 2,
    df=lambda x: 2*x + 3,
    x0=5.0
)
print(f"Minimum at x = {x_min:.4f}")  # → x ≈ -1.5

Core AI Layer (Phases 3–9)

Deep learning core, computer vision, NLP, speech & audio, transformers, generative AI, and reinforcement learning. This is where you build neural networks, CNNs, RNNs, and attention mechanisms — all from raw numpy before touching any framework.

Modern AI Engineering Layer (Phases 10–16)

This is where it gets exciting: LLMs from scratch, LLM engineering (fine-tuning, RAG, evaluation), multimodal models, tools & protocols (MCP, function calling), agent engineering, autonomous systems, and multi-agent swarms.

Production Layer (Phases 17–19)

Infrastructure & production deployment, ethics & alignment, and capstone projects that tie everything together.

The Build It → Use It → Ship It Methodology

Every lesson follows the same six-beat structure, and this consistency is what makes the curriculum so effective:

1. Motto — One-line core idea of the lesson
2. Problem — The concrete pain point this concept solves
3. Concept — Diagrams, intuition, and mental models
4. Build It — Implement from raw math, no frameworks allowed
5. Use It — Do the same thing with PyTorch/sklearn/production libraries
6. Ship It — Produce a reusable artifact (prompt, skill, agent, or MCP server)

The Build It / Use It split is the spine. You implement the algorithm from scratch first, then run the same thing through the production library. You understand what the framework is doing because you wrote the smaller version yourself.

Here's what this looks like in practice — Phase 14, Lesson 1: building an agent loop from scratch in ~120 lines of pure Python:

# Phase 14, Lesson 1: The Agent Loop — built from scratch
def run(query, tools):
    """A minimal ReAct-style agent loop. No dependencies."""
    history = [user(query)]
    for step in range(MAX_STEPS):
        msg = llm(history)
        if msg.tool_calls:
            for call in msg.tool_calls:
                result = tools[call.name](**call.args)
                history.append(tool_result(call.id, result))
            continue
        return msg.content
    raise StepLimitExceeded(
        f"Agent exceeded {MAX_STEPS} steps"
    )

And the artifact it produces — a reusable skill you can install in Claude, Cursor, Codex, OpenClaw, or any agent that reads SKILL.md:

---
name: agent-loop
description: ReAct-style loop for any tool list
phase: 14
lesson: 01
---
Implement a minimal agent loop that:
1. Maintains conversation history
2. Processes tool calls iteratively
3. Respects a configurable step limit
4. Returns structured results

Real-World Example: From Gradient Descent to a Production Debugging Skill

Let's trace a real learning path through the curriculum to see how concepts compound:

Phase 1, Lesson 8 — You implement gradient descent variants (SGD, momentum, Adam) from scratch in Python. You understand why Adam converges faster, not just that it does.

Phase 3, Lesson 5 — You build backpropagation by hand through a 3-layer network. When PyTorch's loss.backward() fails silently, you know exactly where to look.

Phase 8, Lesson 3 — You implement a transformer's attention mechanism from numpy. Now you understand why your fine-tuned model is hallucinating on long contexts.

Phase 11, Lesson 7 — You build a loss debugging agent that uses your Phase 1 understanding of optimization to diagnose training failures:

# Artifact: prompt-loss-debugger.md
You are an ML training debugger. Given a loss curve that
plateaus or diverges, diagnose the issue by checking:
1. Learning rate schedule (ref: Phase 1, Lesson 8)
2. Gradient flow and vanishing/exploding gradients (Phase 3, Lesson 5)
3. Loss function suitability for the task (Phase 3, Lesson 5)
Provide specific recommendations with code fixes.

Each lesson's artifact builds on previous understanding. By the end, you have 503 reusable tools — and more importantly, you understand every single one of them because you built them.

Key Benefits for Developers

• True understanding, not API memorization — You'll know why things work, not just how to call them
• Language-agnostic — Python, TypeScript, Rust, and Julia implementations for every concept
• Portfolio-ready artifacts — 503 prompts, skills, agents, and MCP servers you actually built
• Self-paced and flexible — A built-in /find-your-level placement quiz maps you to the right starting phase
• Runs on your laptop — No cloud GPUs required for the vast majority of lessons
• Community-driven — 500+ contributors, active Discord, regular updates
• Completely free — MIT licensed, no premium tier, no strings attached
• AI-agent compatible — Works with Claude, Cursor, Codex, OpenClaw, and any SKILL.md-aware tool

How to Get Started

Three ways in, depending on your preference:

Option 1: Just Read

Open any completed lesson on aiengineeringfromscratch.com. No setup required.

Option 2: Clone and Run

git clone https://github.com/rohitg00/ai-engineering-from-scratch.git
cd ai-engineering-from-scratch
python phases/01-math-foundations/01-linear-algebra-intuition/code/vectors.py

Option 3: Find Your Level (Recommended)

Use the built-in /find-your-level skill inside any AI coding agent. Ten questions map your knowledge to a starting phase with personalized hour estimates. After each phase, use /check-understanding to quiz yourself.

Whether you're a junior developer wanting to understand what's under the hood of the AI tools you use daily, or a senior engineer looking to fill gaps in your ML theory knowledge — this curriculum meets you where you are.

Frequently Asked Questions

1. Do I need a math background to start AI Engineering from Scratch?

No. Phase 1 covers all the math you need — linear algebra, calculus, probability, and optimization — through code. You learn math by writing programs that implement it, which is far more effective than reading textbooks. If you can write a Python function, you can start at Phase 1.

2. How long does it take to complete the full curriculum?

The full curriculum is approximately 320 hours of structured content. At a pace of 10 hours per week, that's about 8 months. However, the /find-your-level placement quiz often lets experienced developers skip the first several phases, significantly reducing the time needed.

3. Is this curriculum suitable for production AI engineers, or just beginners?

Absolutely suitable for production engineers. Phases 10-19 cover LLM engineering, multimodal models, MCP protocols, agent engineering, autonomous systems, multi-agent swarms, and production infrastructure. Many senior engineers use the curriculum to fill gaps in their understanding — especially the "build from scratch" phases that frameworks like PyTorch normally hide.

4. What programming languages does the curriculum use?

The curriculum supports Python (primary), TypeScript, Rust, and Julia. Not every lesson is available in all four languages yet, but Python coverage is near-complete. TypeScript and Rust implementations are growing rapidly, with community contributions coming in daily.

5. Can I use the artifacts (prompts, skills, agents) in commercial projects?

Yes. The entire curriculum is MIT licensed, which means you can use, modify, and distribute any artifact — including in commercial products. The prompts, skills, and MCP servers you build are yours to keep and deploy however you want.

6. How does this compare to fast.ai or Andrej Karpathy's courses?

Fast.ai and Karpathy's courses are excellent but focus on specific areas (deep learning and LLMs, respectively). AI Engineering from Scratch covers the full spectrum — from math foundations through multi-agent swarm systems — and produces reusable artifacts at every step. Think of it as the spine that connects all the specialized courses. Many learners use it alongside those resources.

7. Does it work with AI coding assistants like Cursor, Claude, or Copilot?

Yes — this is one of the curriculum's standout features. The skills produced by each lesson follow the SKILL.md format, which is compatible with Claude, Cursor, Codex, OpenClaw, Hermes, and any agent that supports custom skills. You can install the entire skill set with python3 scripts/install_skills.py and immediately use them in your AI-powered workflow.