用 Python 手动最小化函数

Let's run gradient descent ourselves on one simple curve. We will minimize a basic function in plain Python before trusting any framework to do it.

We use a single-dip parabola. Its lowest point sits at x equal to 3, so that is the minimum our code should discover on its own.

For this curve the slope, or gradient, is two times x minus three. In real models autograd computes this for you, but here we write it directly.

Descent needs a starting point. We pick an arbitrary initial value far from the answer so we can watch the steps march toward it.

We set a modest learning rate so steps are big enough to make progress but small enough to avoid overshooting the dip.

Each iteration applies the same rule from before: subtract the scaled gradient from x. One line does all the downhill work.

One step is not enough, so we wrap the update in a loop. Twenty or so iterations let x slide steadily toward the minimum.

Print x and f(x) each pass and you will see them converge: x creeps toward 3 and the function value shrinks toward zero.

Notice the moves shrink as you approach the bottom. Near the minimum the gradient is tiny, so each step naturally slows down without any extra code.

Set the learning rate to something like 1.1 and rerun. x bounces away instead of settling, showing live how a too-big step diverges.

This handful of lines is exactly what deep learning does, just with millions of weights and an automatic gradient. The core loop never changes.

What signals that descent is converging?

You coded gradient descent by hand: define a function and its gradient, start somewhere, then loop the update rule until x converges on the minimum. ✅