0Pricing
Machine Learning Academy · Lekcja

Ocena i inferencja: od logitów do przewidywanych etykiet

Uczą się Państwo obliczać softmax dla logitów, dekodować indeksy przewidywanych klas do etykiet oraz oceniać dostrojony model za pomocą accuracy i F1 na wydzielonym zbiorze testowym.

Ocena i inferencja: od logitów do przewidywanych etykiet to bezpłatna lekcja Machine Learning Academy na CoddyKit. To lekcja 4 z 4. Możesz przeczytać całą lekcję poniżej za darmo — a potem ćwiczyć ją interaktywnie w przeglądarce z wbudowanym edytorem kodu i tutorem AI dostępnym 24/7. To część ścieżki edukacyjnej Machine Learning Academy, a Twój postęp synchronizuje się między webem a aplikacją CoddyKit. Kurs Machine Learning Academy zawiera 4 lekcji w sumie.

Części tej lekcji nie zostały jeszcze przetłumaczone i są wyświetlane po angielsku.

What Are Logits?

The output of a classification model's final linear layer is called logits: raw, un-normalised scores for each class. For a 2-class problem (negative/positive), a logit vector might be [-1.2, 3.5], meaning the model strongly favours class 1 (positive). Logits can be any real number; they are not probabilities. We need an additional step (softmax or sigmoid) to convert them into interpretable probability values that sum to 1.

import torch
import torch.nn.functional as F

# Example logits for 2-class problem (batch of 3)
logits = torch.tensor([[-1.2, 3.5], [2.1, -0.3], [0.4, 0.6]])
print('Logits:\n', logits)

# Convert to probabilities with softmax
probs = F.softmax(logits, dim=1)
print('Probabilities:\n', probs)
# Each row sums to 1
print('Row sums:', probs.sum(dim=1))

Softmax: Converting Logits to Probabilities

The softmax function converts a vector of real-valued logits into a probability distribution. For class i: softmax(z_i) = exp(z_i) / sum(exp(z_j)). The exponential amplifies differences: a logit difference of 2 leads to one class being about 7x more probable than the other. Softmax is used for multi-class classification where exactly one class is correct.

import torch
import torch.nn.functional as F
import numpy as np

logits = torch.tensor([1.0, 3.0, 0.5])  # 3 classes
probs = F.softmax(logits, dim=0)
print('Class probabilities:', probs.numpy().round(4))
# [0.0900, 0.6652, 0.2449] -- class 1 is most likely
print('Sum:', probs.sum().item())  # 1.0

# argmax gives the predicted class index
pred_class = torch.argmax(probs).item()
print('Predicted class:', pred_class)  # 1

argmax: Extracting Predicted Labels

Once we have logit vectors for a batch of examples, torch.argmax(logits, dim=1) returns the index of the highest-scoring class for each example. This is the predicted class label. For BERT fine-tuned with num_labels=2, the output is 0 (negative) or 1 (positive). We do not need to apply softmax before argmax because softmax is monotone: the largest logit always maps to the largest probability.

import torch

# Logits from model output for a batch of 4 examples
logits = torch.tensor([
    [-2.1,  3.4],  # strongly positive
    [ 1.8, -0.5],  # negative
    [ 0.1,  0.2],  # uncertain, leans positive
    [-3.0, -1.0]   # both negative, less-negative wins
])

predictions = torch.argmax(logits, dim=1)
print('Predictions:', predictions.tolist())  # [1, 0, 1, 1]

labels = {'0': 'negative', '1': 'positive'}
for pred in predictions.tolist():
    print(labels[str(pred)])

Decoding Predicted Indices to Class Names

Model outputs are integer indices. In a production system you maintain a mapping from index to human-readable label. Store this as a list or dictionary and index into it with the predicted integer. For BERT fine-tuned on IMDB, the mapping is simply {0: 'NEGATIVE', 1: 'POSITIVE'}. For multi-class tasks like topic classification, the mapping might have 20 or more entries.

import torch
from transformers import BertForSequenceClassification, BertTokenizer

# Assume model and tokenizer are already loaded and fine-tuned
id2label = {0: 'NEGATIVE', 1: 'POSITIVE'}

def predict(text, model, tokenizer, device):
    model.eval()
    inputs = tokenizer(text, return_tensors='pt',
                       truncation=True, max_length=256)
    inputs = {k: v.to(device) for k, v in inputs.items()}
    with torch.no_grad():
        logits = model(**inputs).logits
    pred_id = torch.argmax(logits, dim=1).item()
    return id2label[pred_id]

# result = predict('This film was absolutely wonderful!', model, tokenizer, device)
# print(result)  # POSITIVE

Confidence Scores from Probabilities

Returning the predicted label alone often is not enough for real applications. A confidence score (the probability of the predicted class) tells users how certain the model is. A prediction of POSITIVE with 99% confidence is very different from one with 52% confidence. Flagging low-confidence predictions for human review is a common practice in production systems handling sensitive decisions.

import torch
import torch.nn.functional as F

def predict_with_confidence(text, model, tokenizer, device):
    model.eval()
    id2label = {0: 'NEGATIVE', 1: 'POSITIVE'}
    inputs = tokenizer(text, return_tensors='pt',
                       truncation=True, max_length=256)
    inputs = {k: v.to(device) for k, v in inputs.items()}
    with torch.no_grad():
        logits = model(**inputs).logits
    probs = F.softmax(logits, dim=1).squeeze()
    pred_id = torch.argmax(probs).item()
    confidence = probs[pred_id].item()
    return id2label[pred_id], round(confidence, 4)

# label, conf = predict_with_confidence('Terrible movie.', model, tokenizer, device)
# print(label, conf)  # NEGATIVE 0.9732

Evaluating with Accuracy and F1

After collecting predictions for the entire test set, compute standard metrics using scikit-learn. Accuracy is the fraction of correct predictions — useful when classes are balanced. F1-score is the harmonic mean of precision and recall — important when one class (e.g., positive reviews) matters more or classes are imbalanced. classification_report prints all metrics in one readable table.

from sklearn.metrics import classification_report, accuracy_score
import torch

all_preds, all_labels = [], []

model.eval()
with torch.no_grad():
    for batch in test_loader:
        outputs = model(
            input_ids=batch['input_ids'].to(device),
            attention_mask=batch['attention_mask'].to(device)
        )
        preds = torch.argmax(outputs.logits, dim=1).cpu().numpy()
        all_preds.extend(preds)
        all_labels.extend(batch['label'].numpy())

print('Accuracy:', accuracy_score(all_labels, all_preds))
print(classification_report(all_labels, all_preds,
      target_names=['NEGATIVE', 'POSITIVE']))

Confusion Matrix for BERT

A confusion matrix shows the breakdown of predictions versus true labels. For sentiment analysis, the off-diagonal cells show false positives (predicted POSITIVE but actually NEGATIVE) and false negatives (predicted NEGATIVE but actually POSITIVE). Visualising this with seaborn heatmap reveals systematic errors: does the model over-predict one class? Are certain types of reviews consistently mis-classified?

from sklearn.metrics import confusion_matrix
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np

cm = confusion_matrix(all_labels, all_preds)

plt.figure(figsize=(6, 5))
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',
            xticklabels=['NEG', 'POS'],
            yticklabels=['NEG', 'POS'])
plt.ylabel('True Label')
plt.xlabel('Predicted Label')
plt.title('BERT Confusion Matrix on IMDB Test Set')
plt.tight_layout()
plt.savefig('bert_confusion.png')

Batch Inference for Efficiency

Running inference one example at a time is slow. Process examples in batches to take advantage of GPU parallelism. Set a fixed batch_size in your DataLoader, pass entire batches through the model, and collect results. On a GPU, batched inference can be 50x faster than single-example inference. Use torch.no_grad() and move tensors to the correct device for optimal performance.

import torch
from torch.utils.data import DataLoader, TensorDataset
from transformers import BertTokenizer

def batch_predict(texts, model, tokenizer, device, batch_size=32):
    model.eval()
    all_predictions = []
    for i in range(0, len(texts), batch_size):
        batch_texts = texts[i:i+batch_size]
        enc = tokenizer(batch_texts, truncation=True, padding=True,
                        max_length=256, return_tensors='pt')
        enc = {k: v.to(device) for k, v in enc.items()}
        with torch.no_grad():
            logits = model(**enc).logits
        preds = torch.argmax(logits, dim=1).cpu().tolist()
        all_predictions.extend(preds)
    return all_predictions

Sigmoid for Multi-Label Classification

When a text can belong to multiple classes simultaneously (e.g., a review that is both 'funny' and 'emotional'), use sigmoid instead of softmax. Sigmoid applies independently to each class logit, producing a probability between 0 and 1 per class that does not sum to 1. Apply a threshold (usually 0.5) to each probability to get a binary prediction per class. Set problem_type='multi_label_classification' in BertForSequenceClassification.

import torch
import torch.nn.functional as F

# 5 classes, multi-label: a text can have multiple labels
logits = torch.tensor([[1.2, -0.5, 2.1, -1.8, 0.3]])
probs = torch.sigmoid(logits)
print('Per-class probabilities:', probs)

threshold = 0.5
predicted_labels = (probs > threshold).int()
print('Predicted labels (multi-hot):', predicted_labels)
# e.g., [1, 0, 1, 0, 0] -- classes 0 and 2 are predicted

Handling Misclassified Examples

Always inspect misclassified examples manually. Print examples where the model predicted 0 but the true label is 1 (false negatives) and vice versa. Common patterns include sarcasm ('Oh, what a brilliant disaster of a film'), domain shift (old-fashioned vocabulary), or long reviews where the model only sees the first 256 tokens. Understanding failure modes guides feature engineering or data collection efforts.

import numpy as np

all_preds = np.array(all_preds)
all_labels = np.array(all_labels)
texts = test_dataset['text'] if hasattr(test_dataset, '__getitem__') else []

false_negatives = np.where((all_preds == 0) & (all_labels == 1))[0]
false_positives = np.where((all_preds == 1) & (all_labels == 0))[0]

print('False negatives (predicted NEG, true POS):')
for idx in false_negatives[:3]:
    print(' -', str(texts[idx])[:120] if texts else idx)

print('False positives (predicted POS, true NEG):')
for idx in false_positives[:3]:
    print(' -', str(texts[idx])[:120] if texts else idx)

Exporting Predictions to CSV

In production or competition settings, you often need to export predictions to a CSV file. Use pandas to create a DataFrame with the original text, true label, predicted label, and confidence score. This format is easy to share with stakeholders, audit, and use as input to downstream reporting pipelines. Always include the model version and prediction timestamp in the metadata.

import pandas as pd
import torch.nn.functional as F
import torch

results = []
id2label = {0: 'NEGATIVE', 1: 'POSITIVE'}

model.eval()
for i, text in enumerate(sample_texts):
    inputs = tokenizer(text, return_tensors='pt',
                       truncation=True, max_length=256)
    with torch.no_grad():
        logits = model(**inputs).logits
    probs = F.softmax(logits, dim=1).squeeze()
    pred = torch.argmax(probs).item()
    results.append({
        'text': text[:80],
        'true_label': id2label[sample_labels[i]],
        'predicted_label': id2label[pred],
        'confidence': round(probs[pred].item(), 4)
    })

df = pd.DataFrame(results)
df.to_csv('bert_predictions.csv', index=False)
print(df.head())

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: logits are raw model scores that are converted to probabilities via softmax for multi-class or sigmoid for multi-label tasks, argmax gives the predicted class index which maps to a human-readable label via an id2label dictionary, and classification_report and confusion matrix together give a full picture of model performance beyond accuracy. Next up we explore MLflow for tracking experiments, parameters, and metrics across multiple training runs.

Często zadawane pytania

Czy lekcja „Ocena i inferencja: od logitów do przewidywanych etykiet” jest bezpłatna?

Tak — pełny tekst „Ocena i inferencja: od logitów do przewidywanych etykiet” jest dostępny za darmo tutaj w sieci. Aby ćwiczyć ją interaktywnie (wbudowany edytor kodu i tutor AI dostępny 24/7) i odblokować resztę kursu Machine Learning Academy, przejdź na CoddyKit PRO. Kurs Machine Learning Academy zawiera 4 lekcji w sumie.

Co nauczysz się w „Ocena i inferencja: od logitów do przewidywanych etykiet”?

Uczą się Państwo obliczać softmax dla logitów, dekodować indeksy przewidywanych klas do etykiet oraz oceniać dostrojony model za pomocą accuracy i F1 na wydzielonym zbiorze testowym. Ćwiczysz Machine Learning Academy z praktycznym kodem, który uruchamiasz bezpośrednio w przeglądarce, a tutor AI dostępny 24/7 odpowiada na Twoje pytania podczas pracy nad lekcją.

Czy potrzebuję doświadczenia, aby zacząć Machine Learning Academy?

Nie wymagamy żadnego doświadczenia. Machine Learning Academy w CoddyKit jest strukturyzowany dla początkujących i zaawansowanych użytkowników, więc możesz zacząć tutaj lub od początku i uczyć się w swoim tempie. To lekcja 4 z 4.

Ile czasu zajmuje lekcja „Ocena i inferencja: od logitów do przewidywanych etykiet”?

Większość lekcji CoddyKit trwa około 5–10 minut. Każda lekcja to mały, interaktywny krok, dzięki czemu robisz systematyczne postępy i zawsze wracasz dokładnie do tego samego miejsca — na webie i w aplikacji.

Czy mogę pisać i uruchamiać kod w tej lekcji Machine Learning Academy?

Tak. Każda lekcja Machine Learning Academy zawiera wbudowany edytor kodu, więc piszesz i uruchamiasz prawdziwy kod bezpośrednio w przeglądarce i od razu otrzymujesz sprzężenie zwrotne od AI — bez konfiguracji na komputerze.

Wszystkie lekcje w tym kursie

  1. Architektura Transformer: attention, tokeny i kontekst
  2. Tokenizery Hugging Face: kodowanie tekstu dla BERT
  3. Dostrajanie BertForSequenceClassification
  4. Ocena i inferencja: od logitów do przewidywanych etykiet
← Powrót do Machine Learning Academy