Machine Learning Academy · 课时

评估与推理：从 Logits 到预测标签

您将对 logits 计算 softmax，将预测的类别索引解码为标签，并在保留的测试集上使用准确率和 F1 评估微调后的模型。

评估与推理：从 Logits 到预测标签 是 CoddyKit 上的免费 Machine Learning Academy 课时。这是第 4 节课，共 4 节。你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。这是 Machine Learning Academy 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Machine Learning Academy 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

What Are Logits?

The output of a classification model's final linear layer is called logits: raw, un-normalised scores for each class. For a 2-class problem (negative/positive), a logit vector might be [-1.2, 3.5], meaning the model strongly favours class 1 (positive). Logits can be any real number; they are not probabilities. We need an additional step (softmax or sigmoid) to convert them into interpretable probability values that sum to 1.

import torch
import torch.nn.functional as F

# Example logits for 2-class problem (batch of 3)
logits = torch.tensor([[-1.2, 3.5], [2.1, -0.3], [0.4, 0.6]])
print('Logits:\n', logits)

# Convert to probabilities with softmax
probs = F.softmax(logits, dim=1)
print('Probabilities:\n', probs)
# Each row sums to 1
print('Row sums:', probs.sum(dim=1))

Softmax: Converting Logits to Probabilities

The softmax function converts a vector of real-valued logits into a probability distribution. For class i: softmax(z_i) = exp(z_i) / sum(exp(z_j)). The exponential amplifies differences: a logit difference of 2 leads to one class being about 7x more probable than the other. Softmax is used for multi-class classification where exactly one class is correct.

import torch
import torch.nn.functional as F
import numpy as np

logits = torch.tensor([1.0, 3.0, 0.5])  # 3 classes
probs = F.softmax(logits, dim=0)
print('Class probabilities:', probs.numpy().round(4))
# [0.0900, 0.6652, 0.2449] -- class 1 is most likely
print('Sum:', probs.sum().item())  # 1.0

# argmax gives the predicted class index
pred_class = torch.argmax(probs).item()
print('Predicted class:', pred_class)  # 1

argmax: Extracting Predicted Labels

Once we have logit vectors for a batch of examples, torch.argmax(logits, dim=1) returns the index of the highest-scoring class for each example. This is the predicted class label. For BERT fine-tuned with num_labels=2, the output is 0 (negative) or 1 (positive). We do not need to apply softmax before argmax because softmax is monotone: the largest logit always maps to the largest probability.

import torch

# Logits from model output for a batch of 4 examples
logits = torch.tensor([
    [-2.1,  3.4],  # strongly positive
    [ 1.8, -0.5],  # negative
    [ 0.1,  0.2],  # uncertain, leans positive
    [-3.0, -1.0]   # both negative, less-negative wins
])

predictions = torch.argmax(logits, dim=1)
print('Predictions:', predictions.tolist())  # [1, 0, 1, 1]

labels = {'0': 'negative', '1': 'positive'}
for pred in predictions.tolist():
    print(labels[str(pred)])

Decoding Predicted Indices to Class Names

Model outputs are integer indices. In a production system you maintain a mapping from index to human-readable label. Store this as a list or dictionary and index into it with the predicted integer. For BERT fine-tuned on IMDB, the mapping is simply {0: 'NEGATIVE', 1: 'POSITIVE'}. For multi-class tasks like topic classification, the mapping might have 20 or more entries.

import torch
from transformers import BertForSequenceClassification, BertTokenizer

# Assume model and tokenizer are already loaded and fine-tuned
id2label = {0: 'NEGATIVE', 1: 'POSITIVE'}

def predict(text, model, tokenizer, device):
    model.eval()
    inputs = tokenizer(text, return_tensors='pt',
                       truncation=True, max_length=256)
    inputs = {k: v.to(device) for k, v in inputs.items()}
    with torch.no_grad():
        logits = model(**inputs).logits
    pred_id = torch.argmax(logits, dim=1).item()
    return id2label[pred_id]

# result = predict('This film was absolutely wonderful!', model, tokenizer, device)
# print(result)  # POSITIVE

Confidence Scores from Probabilities

Returning the predicted label alone often is not enough for real applications. A confidence score (the probability of the predicted class) tells users how certain the model is. A prediction of POSITIVE with 99% confidence is very different from one with 52% confidence. Flagging low-confidence predictions for human review is a common practice in production systems handling sensitive decisions.

import torch
import torch.nn.functional as F

def predict_with_confidence(text, model, tokenizer, device):
    model.eval()
    id2label = {0: 'NEGATIVE', 1: 'POSITIVE'}
    inputs = tokenizer(text, return_tensors='pt',
                       truncation=True, max_length=256)
    inputs = {k: v.to(device) for k, v in inputs.items()}
    with torch.no_grad():
        logits = model(**inputs).logits
    probs = F.softmax(logits, dim=1).squeeze()
    pred_id = torch.argmax(probs).item()
    confidence = probs[pred_id].item()
    return id2label[pred_id], round(confidence, 4)

# label, conf = predict_with_confidence('Terrible movie.', model, tokenizer, device)
# print(label, conf)  # NEGATIVE 0.9732

Evaluating with Accuracy and F1

After collecting predictions for the entire test set, compute standard metrics using scikit-learn. Accuracy is the fraction of correct predictions — useful when classes are balanced. F1-score is the harmonic mean of precision and recall — important when one class (e.g., positive reviews) matters more or classes are imbalanced. classification_report prints all metrics in one readable table.

from sklearn.metrics import classification_report, accuracy_score
import torch

all_preds, all_labels = [], []

model.eval()
with torch.no_grad():
    for batch in test_loader:
        outputs = model(
            input_ids=batch['input_ids'].to(device),
            attention_mask=batch['attention_mask'].to(device)
        )
        preds = torch.argmax(outputs.logits, dim=1).cpu().numpy()
        all_preds.extend(preds)
        all_labels.extend(batch['label'].numpy())

print('Accuracy:', accuracy_score(all_labels, all_preds))
print(classification_report(all_labels, all_preds,
      target_names=['NEGATIVE', 'POSITIVE']))

Confusion Matrix for BERT

A confusion matrix shows the breakdown of predictions versus true labels. For sentiment analysis, the off-diagonal cells show false positives (predicted POSITIVE but actually NEGATIVE) and false negatives (predicted NEGATIVE but actually POSITIVE). Visualising this with seaborn heatmap reveals systematic errors: does the model over-predict one class? Are certain types of reviews consistently mis-classified?

from sklearn.metrics import confusion_matrix
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np

cm = confusion_matrix(all_labels, all_preds)

plt.figure(figsize=(6, 5))
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',
            xticklabels=['NEG', 'POS'],
            yticklabels=['NEG', 'POS'])
plt.ylabel('True Label')
plt.xlabel('Predicted Label')
plt.title('BERT Confusion Matrix on IMDB Test Set')
plt.tight_layout()
plt.savefig('bert_confusion.png')

Batch Inference for Efficiency

Running inference one example at a time is slow. Process examples in batches to take advantage of GPU parallelism. Set a fixed batch_size in your DataLoader, pass entire batches through the model, and collect results. On a GPU, batched inference can be 50x faster than single-example inference. Use torch.no_grad() and move tensors to the correct device for optimal performance.

import torch
from torch.utils.data import DataLoader, TensorDataset
from transformers import BertTokenizer

def batch_predict(texts, model, tokenizer, device, batch_size=32):
    model.eval()
    all_predictions = []
    for i in range(0, len(texts), batch_size):
        batch_texts = texts[i:i+batch_size]
        enc = tokenizer(batch_texts, truncation=True, padding=True,
                        max_length=256, return_tensors='pt')
        enc = {k: v.to(device) for k, v in enc.items()}
        with torch.no_grad():
            logits = model(**enc).logits
        preds = torch.argmax(logits, dim=1).cpu().tolist()
        all_predictions.extend(preds)
    return all_predictions

Sigmoid for Multi-Label Classification

When a text can belong to multiple classes simultaneously (e.g., a review that is both 'funny' and 'emotional'), use sigmoid instead of softmax. Sigmoid applies independently to each class logit, producing a probability between 0 and 1 per class that does not sum to 1. Apply a threshold (usually 0.5) to each probability to get a binary prediction per class. Set problem_type='multi_label_classification' in BertForSequenceClassification.

import torch
import torch.nn.functional as F

# 5 classes, multi-label: a text can have multiple labels
logits = torch.tensor([[1.2, -0.5, 2.1, -1.8, 0.3]])
probs = torch.sigmoid(logits)
print('Per-class probabilities:', probs)

threshold = 0.5
predicted_labels = (probs > threshold).int()
print('Predicted labels (multi-hot):', predicted_labels)
# e.g., [1, 0, 1, 0, 0] -- classes 0 and 2 are predicted

Handling Misclassified Examples

Always inspect misclassified examples manually. Print examples where the model predicted 0 but the true label is 1 (false negatives) and vice versa. Common patterns include sarcasm ('Oh, what a brilliant disaster of a film'), domain shift (old-fashioned vocabulary), or long reviews where the model only sees the first 256 tokens. Understanding failure modes guides feature engineering or data collection efforts.

import numpy as np

all_preds = np.array(all_preds)
all_labels = np.array(all_labels)
texts = test_dataset['text'] if hasattr(test_dataset, '__getitem__') else []

false_negatives = np.where((all_preds == 0) & (all_labels == 1))[0]
false_positives = np.where((all_preds == 1) & (all_labels == 0))[0]

print('False negatives (predicted NEG, true POS):')
for idx in false_negatives[:3]:
    print(' -', str(texts[idx])[:120] if texts else idx)

print('False positives (predicted POS, true NEG):')
for idx in false_positives[:3]:
    print(' -', str(texts[idx])[:120] if texts else idx)

Exporting Predictions to CSV

In production or competition settings, you often need to export predictions to a CSV file. Use pandas to create a DataFrame with the original text, true label, predicted label, and confidence score. This format is easy to share with stakeholders, audit, and use as input to downstream reporting pipelines. Always include the model version and prediction timestamp in the metadata.

import pandas as pd
import torch.nn.functional as F
import torch

results = []
id2label = {0: 'NEGATIVE', 1: 'POSITIVE'}

model.eval()
for i, text in enumerate(sample_texts):
    inputs = tokenizer(text, return_tensors='pt',
                       truncation=True, max_length=256)
    with torch.no_grad():
        logits = model(**inputs).logits
    probs = F.softmax(logits, dim=1).squeeze()
    pred = torch.argmax(probs).item()
    results.append({
        'text': text[:80],
        'true_label': id2label[sample_labels[i]],
        'predicted_label': id2label[pred],
        'confidence': round(probs[pred].item(), 4)
    })

df = pd.DataFrame(results)
df.to_csv('bert_predictions.csv', index=False)
print(df.head())

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: logits are raw model scores that are converted to probabilities via softmax for multi-class or sigmoid for multi-label tasks, argmax gives the predicted class index which maps to a human-readable label via an id2label dictionary, and classification_report and confusion matrix together give a full picture of model performance beyond accuracy. Next up we explore MLflow for tracking experiments, parameters, and metrics across multiple training runs.

常见问题解答

「评估与推理：从 Logits 到预测标签」课时是免费的吗？

是的 — 「评估与推理：从 Logits 到预测标签」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Machine Learning Academy 课程的其余内容，请升级到 CoddyKit PRO。 Machine Learning Academy 课程共包含 4 节课。

「评估与推理：从 Logits 到预测标签」这节课中我会学到什么？

您将对 logits 计算 softmax，将预测的类别索引解码为标签，并在保留的测试集上使用准确率和 F1 评估微调后的模型。你通过在浏览器中直接运行的动手代码来练习 Machine Learning Academy，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 Machine Learning Academy 需要有经验吗？

无需任何先前经验。CoddyKit 上的 Machine Learning Academy 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。这是第 4 节课，共 4 节。

「评估与推理：从 Logits 到预测标签」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 Machine Learning Academy 课中编写并运行代码吗？

能。每节 Machine Learning Academy 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

← 返回 Machine Learning Academy