Cross-Entropy for Multiclass
Why it expects raw logits, not softmax.
Cross-Entropy for Multiclass is a free Deep Learning Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Deep Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
More Than Two Choices
When your model must pick one label from many, like a digit 0 through 9, you use cross-entropy, the standard multiclass classification loss. 🔢
One Logit Per Class
For C classes, your network outputs a vector of C raw scores called logits, one per possible class, for every sample in the batch.
Softmax Turns Scores Into Odds
Softmax exponentiates each logit and normalizes them so they sum to 1, turning raw scores into a clean probability distribution.
import torch
probs = torch.softmax(logits, dim=1)The Loss for Many Classes
PyTorch bundles softmax and the loss into nn.CrossEntropyLoss. It scores how much probability your model put on the correct class.
import torch.nn as nn
loss_fn = nn.CrossEntropyLoss()Pass Raw Logits Directly
This is the key rule: feed CrossEntropyLoss the raw logits, never a softmax output. It applies log-softmax internally for you.
loss = loss_fn(logits, labels)Why Not Softmax First
Applying softmax yourself, then this loss, squashes the values twice. That double pass distorts gradients and quietly wrecks training.
Labels Are Class Indices
Your targets are plain integer indices, like 3 for class three, not one-hot vectors. PyTorch handles the lookup for you.
labels = torch.tensor([3, 0, 7])Mind the Output Shape
Logits have shape batch by classes, while labels have shape batch only. A shape mismatch here is the most common cross-entropy error.
It Is Built From Two Pieces
Under the hood, CrossEntropyLoss equals log-softmax followed by negative log-likelihood. Fusing them keeps the math numerically stable.
Predict With Argmax
To get the final answer, take the argmax over the logits. The largest score is the predicted class, no softmax needed.
preds = logits.argmax(dim=1)Smooth Labels for Generalization
Setting label_smoothing nudges targets slightly away from a hard 1, which discourages overconfidence and often improves test accuracy.
loss_fn = nn.CrossEntropyLoss(label_smoothing=0.1)Quick Check
You are using nn.CrossEntropyLoss for 10 classes. What should the model's final output be?
Recap: One Winner Per Sample
For multiclass tasks you feed raw logits and integer labels to CrossEntropyLoss, which softmaxes internally and scores the right class. Argmax gives the prediction. 🏆
Frequently asked questions
Is the “Cross-Entropy for Multiclass” lesson free?
Yes — the full text of “Cross-Entropy for Multiclass” is free to read here on the web, and the Deep Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Deep Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “Cross-Entropy for Multiclass”?
Why it expects raw logits, not softmax. You practise Deep Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Deep Learning Academy?
No prior experience is required. Deep Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Cross-Entropy for Multiclass” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Deep Learning Academy lesson?
Yes. Every Deep Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- MSE & MAE for Regression
- Binary Cross-Entropy with Logits
- Cross-Entropy for Multiclass
- Class Weights for Imbalanced Data