Функция потерь как ландшафт для спуска

Your model's loss is a single number that says how wrong its predictions are. Smaller loss means better predictions. Training is just the quest to shrink it. 📉

The same data gives different loss depending on your weights. Change a weight and the loss moves up or down. So loss is really a function of the weights.

Imagine every weight setting as a spot on a hilly landscape, and the height at that spot is the loss. Training means walking downhill to lower ground.

The lowest point in this landscape is the minimum: the weights where your model predicts best. Your whole job is to reach that valley floor.

With a single weight, the landscape is just a 2D curve. You can literally plot loss against the weight and see the dip where loss is smallest.

Add a second weight and the curve becomes a 3D surface with hills and valleys. Real models have millions of weights, so the landscape lives in millions of dimensions.

In high dimensions you can never view the whole landscape at once. You only feel the slope right under your feet and step accordingly.

A steep slope means loss changes fast here, so a small step helps a lot. Flat ground means you are near a valley or a plateau and progress slows.

The landscape can have many dips. A local minimum is a small valley that traps you above the deepest one. Deep nets usually still train well anyway.

The whole training algorithm is called gradient descent for a reason: you repeatedly step downhill on this loss landscape until you settle near the bottom.

Thinking of loss as terrain makes training intuitive: where am I, which way is down, and how big a step should I take? Every other idea builds on this picture.

Loss is a number that depends on your weights, so it forms a landscape of hills and valleys. Training means walking downhill toward the lowest loss you can find. 🏔️