Precyzja, czułość i F1-score w praktyce

When classes are imbalanced or when different error types have different costs, we need metrics that focus specifically on how the model handles the positive class. Precision and Recall are the two complementary metrics that fill this role.

Together, precision and recall give a complete picture of classifier behaviour on the positive class without being distorted by a large number of true negatives. This matters enormously in fraud detection, disease screening, and content moderation — all domains where the positive class is rare but critical.

Precision answers: Of all the examples the model predicted as positive, what fraction were actually positive?

Precision = TP / (TP + FP)

High precision means few false alarms. A spam filter with precision 0.99 means 99% of emails sent to the spam folder are actually spam — very few legitimate emails are misclassified. However, precision says nothing about how many real spam emails were missed. A model that flags only one email as spam and that email is spam has precision 1.0, even if it missed thousands of spam emails.

Recall (also called sensitivity or true positive rate) answers: Of all the actual positive examples, what fraction did the model correctly identify?

Recall = TP / (TP + FN)

High recall means few missed positives. A cancer screening test with recall 0.95 catches 95% of patients who actually have cancer — only 5% are missed. However, recall says nothing about how many healthy patients were incorrectly flagged. High recall is essential when the cost of missing a positive (False Negative) is very high.

Precision and recall are in tension with each other. You can always increase recall to 1.0 by predicting positive for every single example — but precision will collapse because most of those predictions are false positives. Conversely, predicting positive only when you are extremely confident (very high threshold) improves precision but misses many real positives, reducing recall.

Moving the decision threshold changes the balance. Lowering the threshold flags more examples as positive: recall goes up, precision goes down. Raising the threshold makes the model more selective: precision goes up, recall goes down. There is no threshold that maximises both simultaneously in most real problems.

The F1-score combines precision and recall into a single number using the harmonic mean:

F1 = 2 × (Precision × Recall) / (Precision + Recall)

The harmonic mean penalises extreme imbalances: a model with precision 0.9 and recall 0.1 gets F1 = 2×0.9×0.1/1.0 = 0.18, not 0.5. This makes F1 much harder to game than a simple average. F1 = 1.0 only when both precision and recall are 1.0.

The F1-score treats precision and recall as equally important. The Fβ score lets you weight recall β times more than precision:

Fβ = (1 + β²) × Precision × Recall / (β² × Precision + Recall)

• β = 1: standard F1 (equal weight)
• β = 2 (F2): recall is twice as important — use for cancer screening where missing a case is critical
• β = 0.5 (F0.5): precision is twice as important — use for spam filters where false alarms are costly

Instead of evaluating at one threshold, a precision-recall curve plots precision vs recall across all possible thresholds. The shape of the curve tells you the best achievable trade-off for your model.

A model with high area under the precision-recall curve (PR-AUC) is better at finding positives without generating too many false alarms. PR-AUC is preferred over ROC-AUC when the positive class is rare, because it focuses on performance on the minority class rather than being influenced by the large pool of true negatives.

For multi-class classification, scikit-learn reports several types of averages for precision, recall, and F1:

• Macro average: compute metric per class, then take unweighted mean. Treats all classes equally regardless of size.
• Weighted average: compute metric per class, then take mean weighted by the number of examples per class. Reflects overall dataset performance.
• Micro average: aggregate TP/FP/FN across all classes, then compute the metric. For accuracy this equals overall accuracy.

A decision framework for metric selection:

• Balanced classes, equal error costs → Accuracy
• Imbalanced classes, cost of FP high → Precision (e.g., spam filter — don't trash good emails)
• Imbalanced classes, cost of FN high → Recall (e.g., disease screening — don't miss sick patients)
• Both FP and FN costly, need single metric → F1
• FN much more costly than FP → F2 (cancer detection, fraud where you can investigate)
• Need to evaluate across all thresholds → PR-AUC or ROC-AUC

Two additional metrics complete the picture of binary classifier performance:

• Specificity (True Negative Rate) = TN / (TN + FP): of all actual negatives, what fraction did the model correctly identify? High specificity means few false alarms. Used together with sensitivity (recall) in medical testing.
• Fall-out (False Positive Rate) = FP / (FP + TN): the complement of specificity. The x-axis of the ROC curve that you will study in the evaluation metrics lesson.

The default threshold of 0.5 is not always optimal. A workflow for finding the best threshold: compute precision, recall, and F1 at many thresholds, then pick the threshold that maximises the metric relevant to your problem.

For a fraud detection scenario, you might maximise F2 to prioritise recall. For a content moderation system, you might maximise precision to avoid incorrectly removing legitimate content. The precision_recall_curve function gives you all the data you need for this analysis.

In this lesson you learned: precision measures quality of positive predictions (TP/(TP+FP)), recall measures coverage of actual positives (TP/(TP+FN)), and the F1-score harmonic mean balances both — while Fβ lets you weight recall or precision more heavily based on your application's error cost structure. Next up we tackle the fundamental question of model evaluation: why you cannot evaluate a model on the data it was trained on, and how to properly estimate generalisation performance with held-out test sets.