Learning Rate Schedules & Warmup

One fixed learning rate is rarely ideal for the whole run. A schedule changes it over time, usually large early and small near the end.

Big early steps cover ground fast. Shrinking the rate later lets the model settle precisely into a minimum instead of bouncing around it.