0PricingLogin▼
Deep Learning Academy · Lesson

Backprop a Tiny Net by Hand

Compute gradients on paper, confirm in code.

Our Tiny Network

Let's hand-trace the smallest net: one input x, one weight w, no bias. The output is simply y = w times x. Tiny enough to do on paper.

y = w * x

Add a Simple Loss

We compare the output to a target t with squared error. This loss punishes being far from the target and is easy to differentiate by hand.

loss = (y - t) ** 2

All lessons in this course

  1. The Chain Rule, Layer by Layer
  2. Forward Caches, Backward Reuses
  3. Backprop a Tiny Net by Hand
  4. Vanishing & Exploding Gradients
← Back to Deep Learning Academy