Minimize a Function by Hand in Python
Code gradient descent on a simple curve.
Minimize a Function by Hand in Python is a free Deep Learning Academy lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Deep Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
A Tiny Practice Problem
Let's run gradient descent ourselves on one simple curve. We will minimize a basic function in plain Python before trusting any framework to do it.
Pick a Function
We use a single-dip parabola. Its lowest point sits at x equal to 3, so that is the minimum our code should discover on its own.
def f(x):
return (x - 3) ** 2Know the Gradient
For this curve the slope, or gradient, is two times x minus three. In real models autograd computes this for you, but here we write it directly.
def grad(x):
return 2 * (x - 3)Start Somewhere
Descent needs a starting point. We pick an arbitrary initial value far from the answer so we can watch the steps march toward it.
x = 0.0Choose a Learning Rate
We set a modest learning rate so steps are big enough to make progress but small enough to avoid overshooting the dip.
lr = 0.1The Update Step
Each iteration applies the same rule from before: subtract the scaled gradient from x. One line does all the downhill work.
x = x - lr * grad(x)Loop It
One step is not enough, so we wrap the update in a loop. Twenty or so iterations let x slide steadily toward the minimum.
for i in range(20):
x = x - lr * grad(x)Watch It Converge
Print x and f(x) each pass and you will see them converge: x creeps toward 3 and the function value shrinks toward zero.
print(round(x, 4), round(f(x), 6))Steps Get Smaller
Notice the moves shrink as you approach the bottom. Near the minimum the gradient is tiny, so each step naturally slows down without any extra code.
Try a Bad Rate
Set the learning rate to something like 1.1 and rerun. x bounces away instead of settling, showing live how a too-big step diverges.
lr = 1.1Same Idea, Bigger Scale
This handful of lines is exactly what deep learning does, just with millions of weights and an automatic gradient. The core loop never changes.
Quick Check
What signals that descent is converging?
Recap
You coded gradient descent by hand: define a function and its gradient, start somewhere, then loop the update rule until x converges on the minimum. ✅
Frequently asked questions
Is the “Minimize a Function by Hand in Python” lesson free?
Yes — the full text of “Minimize a Function by Hand in Python” is free to read here on the web, and the Deep Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Deep Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “Minimize a Function by Hand in Python”?
Code gradient descent on a simple curve. You practise Deep Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Deep Learning Academy?
No prior experience is required. Deep Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Minimize a Function by Hand in Python” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Deep Learning Academy lesson?
Yes. Every Deep Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Loss as a Landscape to Descend
- Gradients Point Uphill — So Step the Other Way
- Learning Rate: Too Big, Too Small, Just Right
- Minimize a Function by Hand in Python