Softmax for Probabilities
Turn raw scores into a class distribution.
Softmax for Probabilities is a free Deep Learning Academy lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Deep Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
From Scores to Choices
A classifier's last layer spits out raw scores, one per class. These unbounded numbers are called logits, and they are hard to read directly. 🔢
Enter Softmax
Softmax turns a vector of logits into a clean probability distribution over all your classes. Now each score becomes a chance.
import torch.nn.functional as F
probs = F.softmax(logits, dim=-1)Two Guarantees
Softmax promises two things: every output is between 0 and 1, and all of them sum to one. That is exactly what a distribution needs.
How It Works
Softmax exponentiates each logit, then divides by the total. The exponential makes bigger logits dominate, sharpening the winner.
exp = logits.exp()
probs = exp / exp.sum()Relative, Not Absolute
Softmax cares about the gaps between logits, not their raw size. Adding the same constant to every logit leaves the probabilities unchanged.
Sigmoid vs Softmax
Use sigmoid for one yes-or-no output. Use softmax when classes compete and exactly one of several should win.
Picking the Winner
To get the predicted class, take the argmax of the softmax output. That index is the class with the highest probability.
pred = probs.argmax(dim=-1)Mind the Dimension
Always set the right dim so softmax normalizes across classes, not across the batch. A wrong axis quietly ruins your probabilities.
Skip It While Training
During training you usually do not apply softmax yourself. The cross-entropy loss expects raw logits and applies it internally.
loss = F.cross_entropy(logits, target) # logits, not probsWhy That Is Safer
Combining softmax and the log inside one function is more numerically stable. Doing both steps by hand can overflow or lose precision.
Temperature Tweaks Sharpness
Dividing logits by a temperature before softmax controls confidence. Low values sharpen the peak; high values flatten it out.
Quick Check
Think about what cross-entropy loss expects as its input.
Recap
Softmax converts logits into probabilities that sum to one, perfect for multiclass output. But let cross-entropy handle it during training. 🏁
Frequently asked questions
Is the “Softmax for Probabilities” lesson free?
Yes — the full text of “Softmax for Probabilities” is free to read here on the web, and the Deep Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Deep Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “Softmax for Probabilities”?
Turn raw scores into a class distribution. You practise Deep Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Deep Learning Academy?
No prior experience is required. Deep Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Softmax for Probabilities” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Deep Learning Academy lesson?
Yes. Every Deep Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.