Evaluation and Inference: From Logits to Predicted Labels
Learners will compute softmax over logits, decode predicted class indices to labels, and evaluate the fine-tuned model with accuracy and F1 on a held-out test set.
Evaluation and Inference: From Logits to Predicted Labels is a free Machine Learning Academy lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Machine Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
What Are Logits?
The output of a classification model's final linear layer is called logits: raw, un-normalised scores for each class. For a 2-class problem (negative/positive), a logit vector might be [-1.2, 3.5], meaning the model strongly favours class 1 (positive). Logits can be any real number; they are not probabilities. We need an additional step (softmax or sigmoid) to convert them into interpretable probability values that sum to 1.
import torch
import torch.nn.functional as F
# Example logits for 2-class problem (batch of 3)
logits = torch.tensor([[-1.2, 3.5], [2.1, -0.3], [0.4, 0.6]])
print('Logits:\n', logits)
# Convert to probabilities with softmax
probs = F.softmax(logits, dim=1)
print('Probabilities:\n', probs)
# Each row sums to 1
print('Row sums:', probs.sum(dim=1))Softmax: Converting Logits to Probabilities
The softmax function converts a vector of real-valued logits into a probability distribution. For class i: softmax(z_i) = exp(z_i) / sum(exp(z_j)). The exponential amplifies differences: a logit difference of 2 leads to one class being about 7x more probable than the other. Softmax is used for multi-class classification where exactly one class is correct.
import torch
import torch.nn.functional as F
import numpy as np
logits = torch.tensor([1.0, 3.0, 0.5]) # 3 classes
probs = F.softmax(logits, dim=0)
print('Class probabilities:', probs.numpy().round(4))
# [0.0900, 0.6652, 0.2449] -- class 1 is most likely
print('Sum:', probs.sum().item()) # 1.0
# argmax gives the predicted class index
pred_class = torch.argmax(probs).item()
print('Predicted class:', pred_class) # 1argmax: Extracting Predicted Labels
Once we have logit vectors for a batch of examples, torch.argmax(logits, dim=1) returns the index of the highest-scoring class for each example. This is the predicted class label. For BERT fine-tuned with num_labels=2, the output is 0 (negative) or 1 (positive). We do not need to apply softmax before argmax because softmax is monotone: the largest logit always maps to the largest probability.
import torch
# Logits from model output for a batch of 4 examples
logits = torch.tensor([
[-2.1, 3.4], # strongly positive
[ 1.8, -0.5], # negative
[ 0.1, 0.2], # uncertain, leans positive
[-3.0, -1.0] # both negative, less-negative wins
])
predictions = torch.argmax(logits, dim=1)
print('Predictions:', predictions.tolist()) # [1, 0, 1, 1]
labels = {'0': 'negative', '1': 'positive'}
for pred in predictions.tolist():
print(labels[str(pred)])Decoding Predicted Indices to Class Names
Model outputs are integer indices. In a production system you maintain a mapping from index to human-readable label. Store this as a list or dictionary and index into it with the predicted integer. For BERT fine-tuned on IMDB, the mapping is simply {0: 'NEGATIVE', 1: 'POSITIVE'}. For multi-class tasks like topic classification, the mapping might have 20 or more entries.
import torch
from transformers import BertForSequenceClassification, BertTokenizer
# Assume model and tokenizer are already loaded and fine-tuned
id2label = {0: 'NEGATIVE', 1: 'POSITIVE'}
def predict(text, model, tokenizer, device):
model.eval()
inputs = tokenizer(text, return_tensors='pt',
truncation=True, max_length=256)
inputs = {k: v.to(device) for k, v in inputs.items()}
with torch.no_grad():
logits = model(**inputs).logits
pred_id = torch.argmax(logits, dim=1).item()
return id2label[pred_id]
# result = predict('This film was absolutely wonderful!', model, tokenizer, device)
# print(result) # POSITIVEConfidence Scores from Probabilities
Returning the predicted label alone often is not enough for real applications. A confidence score (the probability of the predicted class) tells users how certain the model is. A prediction of POSITIVE with 99% confidence is very different from one with 52% confidence. Flagging low-confidence predictions for human review is a common practice in production systems handling sensitive decisions.
import torch
import torch.nn.functional as F
def predict_with_confidence(text, model, tokenizer, device):
model.eval()
id2label = {0: 'NEGATIVE', 1: 'POSITIVE'}
inputs = tokenizer(text, return_tensors='pt',
truncation=True, max_length=256)
inputs = {k: v.to(device) for k, v in inputs.items()}
with torch.no_grad():
logits = model(**inputs).logits
probs = F.softmax(logits, dim=1).squeeze()
pred_id = torch.argmax(probs).item()
confidence = probs[pred_id].item()
return id2label[pred_id], round(confidence, 4)
# label, conf = predict_with_confidence('Terrible movie.', model, tokenizer, device)
# print(label, conf) # NEGATIVE 0.9732Evaluating with Accuracy and F1
After collecting predictions for the entire test set, compute standard metrics using scikit-learn. Accuracy is the fraction of correct predictions — useful when classes are balanced. F1-score is the harmonic mean of precision and recall — important when one class (e.g., positive reviews) matters more or classes are imbalanced. classification_report prints all metrics in one readable table.
from sklearn.metrics import classification_report, accuracy_score
import torch
all_preds, all_labels = [], []
model.eval()
with torch.no_grad():
for batch in test_loader:
outputs = model(
input_ids=batch['input_ids'].to(device),
attention_mask=batch['attention_mask'].to(device)
)
preds = torch.argmax(outputs.logits, dim=1).cpu().numpy()
all_preds.extend(preds)
all_labels.extend(batch['label'].numpy())
print('Accuracy:', accuracy_score(all_labels, all_preds))
print(classification_report(all_labels, all_preds,
target_names=['NEGATIVE', 'POSITIVE']))Confusion Matrix for BERT
A confusion matrix shows the breakdown of predictions versus true labels. For sentiment analysis, the off-diagonal cells show false positives (predicted POSITIVE but actually NEGATIVE) and false negatives (predicted NEGATIVE but actually POSITIVE). Visualising this with seaborn heatmap reveals systematic errors: does the model over-predict one class? Are certain types of reviews consistently mis-classified?
from sklearn.metrics import confusion_matrix
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
cm = confusion_matrix(all_labels, all_preds)
plt.figure(figsize=(6, 5))
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',
xticklabels=['NEG', 'POS'],
yticklabels=['NEG', 'POS'])
plt.ylabel('True Label')
plt.xlabel('Predicted Label')
plt.title('BERT Confusion Matrix on IMDB Test Set')
plt.tight_layout()
plt.savefig('bert_confusion.png')Batch Inference for Efficiency
Running inference one example at a time is slow. Process examples in batches to take advantage of GPU parallelism. Set a fixed batch_size in your DataLoader, pass entire batches through the model, and collect results. On a GPU, batched inference can be 50x faster than single-example inference. Use torch.no_grad() and move tensors to the correct device for optimal performance.
import torch
from torch.utils.data import DataLoader, TensorDataset
from transformers import BertTokenizer
def batch_predict(texts, model, tokenizer, device, batch_size=32):
model.eval()
all_predictions = []
for i in range(0, len(texts), batch_size):
batch_texts = texts[i:i+batch_size]
enc = tokenizer(batch_texts, truncation=True, padding=True,
max_length=256, return_tensors='pt')
enc = {k: v.to(device) for k, v in enc.items()}
with torch.no_grad():
logits = model(**enc).logits
preds = torch.argmax(logits, dim=1).cpu().tolist()
all_predictions.extend(preds)
return all_predictionsSigmoid for Multi-Label Classification
When a text can belong to multiple classes simultaneously (e.g., a review that is both 'funny' and 'emotional'), use sigmoid instead of softmax. Sigmoid applies independently to each class logit, producing a probability between 0 and 1 per class that does not sum to 1. Apply a threshold (usually 0.5) to each probability to get a binary prediction per class. Set problem_type='multi_label_classification' in BertForSequenceClassification.
import torch
import torch.nn.functional as F
# 5 classes, multi-label: a text can have multiple labels
logits = torch.tensor([[1.2, -0.5, 2.1, -1.8, 0.3]])
probs = torch.sigmoid(logits)
print('Per-class probabilities:', probs)
threshold = 0.5
predicted_labels = (probs > threshold).int()
print('Predicted labels (multi-hot):', predicted_labels)
# e.g., [1, 0, 1, 0, 0] -- classes 0 and 2 are predictedHandling Misclassified Examples
Always inspect misclassified examples manually. Print examples where the model predicted 0 but the true label is 1 (false negatives) and vice versa. Common patterns include sarcasm ('Oh, what a brilliant disaster of a film'), domain shift (old-fashioned vocabulary), or long reviews where the model only sees the first 256 tokens. Understanding failure modes guides feature engineering or data collection efforts.
import numpy as np
all_preds = np.array(all_preds)
all_labels = np.array(all_labels)
texts = test_dataset['text'] if hasattr(test_dataset, '__getitem__') else []
false_negatives = np.where((all_preds == 0) & (all_labels == 1))[0]
false_positives = np.where((all_preds == 1) & (all_labels == 0))[0]
print('False negatives (predicted NEG, true POS):')
for idx in false_negatives[:3]:
print(' -', str(texts[idx])[:120] if texts else idx)
print('False positives (predicted POS, true NEG):')
for idx in false_positives[:3]:
print(' -', str(texts[idx])[:120] if texts else idx)Exporting Predictions to CSV
In production or competition settings, you often need to export predictions to a CSV file. Use pandas to create a DataFrame with the original text, true label, predicted label, and confidence score. This format is easy to share with stakeholders, audit, and use as input to downstream reporting pipelines. Always include the model version and prediction timestamp in the metadata.
import pandas as pd
import torch.nn.functional as F
import torch
results = []
id2label = {0: 'NEGATIVE', 1: 'POSITIVE'}
model.eval()
for i, text in enumerate(sample_texts):
inputs = tokenizer(text, return_tensors='pt',
truncation=True, max_length=256)
with torch.no_grad():
logits = model(**inputs).logits
probs = F.softmax(logits, dim=1).squeeze()
pred = torch.argmax(probs).item()
results.append({
'text': text[:80],
'true_label': id2label[sample_labels[i]],
'predicted_label': id2label[pred],
'confidence': round(probs[pred].item(), 4)
})
df = pd.DataFrame(results)
df.to_csv('bert_predictions.csv', index=False)
print(df.head())Quick Check
Test your understanding of Machine Learning with Python concepts from this lesson.
Lesson Recap
In this lesson you learned: logits are raw model scores that are converted to probabilities via softmax for multi-class or sigmoid for multi-label tasks, argmax gives the predicted class index which maps to a human-readable label via an id2label dictionary, and classification_report and confusion matrix together give a full picture of model performance beyond accuracy. Next up we explore MLflow for tracking experiments, parameters, and metrics across multiple training runs.
Frequently asked questions
Is the “Evaluation and Inference: From Logits to Predicted Labels” lesson free?
Yes — the full text of “Evaluation and Inference: From Logits to Predicted Labels” is free to read here on the web, and the Machine Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Machine Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “Evaluation and Inference: From Logits to Predicted Labels”?
Learners will compute softmax over logits, decode predicted class indices to labels, and evaluate the fine-tuned model with accuracy and F1 on a held-out test set. You practise Machine Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Machine Learning Academy?
No prior experience is required. Machine Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Evaluation and Inference: From Logits to Predicted Labels” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Machine Learning Academy lesson?
Yes. Every Machine Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Transformer Architecture: Attention, Tokens, and Context
- Hugging Face Tokenizers: Encoding Text for BERT
- Fine-Tuning BertForSequenceClassification
- Evaluation and Inference: From Logits to Predicted Labels