Калибруйте уверенность

When a model says 90 percent, it should be right about 90 percent of those times. Calibration measures whether that promise holds.

Modern deep nets often output probabilities that are too high. They sound certain even when they are frequently wrong.

In medicine or finance, decisions hinge on the number, not just the label. A trustworthy probability is as vital as the prediction.

Plot predicted confidence against actual accuracy in bins. A perfectly calibrated model traces the diagonal line.

ECE sums the gap between confidence and accuracy across bins into one number. Lower means better-aligned probabilities.

A confident-looking softmax output is not a guarantee. The raw scores need a correction step to mean what they say.

The simplest fix divides the logits by a learned temperature before softmax, softening overconfident scores.

You tune the single temperature on a held-out validation set, leaving the model weights and its accuracy untouched.

Temperature scaling only rescales confidence; it never changes which class wins, so your accuracy is preserved.

Beyond temperature, Platt scaling and isotonic regression can map raw scores to honest probabilities too.

Always verify calibration before you act on probabilities. A well-calibrated model lets you set thresholds with confidence. ✅

Good calibration means confidence matches accuracy. Check it with reliability diagrams and ECE, then fix it with temperature scaling. 🎯