Precision, Recall, and F1-Score in Practice
Learners will calculate precision and recall, understand the precision-recall trade-off, and choose the right metric for imbalanced real-world tasks.
Precision, Recall, and F1-Score in Practice is a free Machine Learning Academy lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Machine Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
The Precision-Recall Framework
When classes are imbalanced or when different error types have different costs, we need metrics that focus specifically on how the model handles the positive class. Precision and Recall are the two complementary metrics that fill this role.
Together, precision and recall give a complete picture of classifier behaviour on the positive class without being distorted by a large number of true negatives. This matters enormously in fraud detection, disease screening, and content moderation — all domains where the positive class is rare but critical.
Precision: Quality of Positive Predictions
Precision answers: Of all the examples the model predicted as positive, what fraction were actually positive?
Precision = TP / (TP + FP)
High precision means few false alarms. A spam filter with precision 0.99 means 99% of emails sent to the spam folder are actually spam — very few legitimate emails are misclassified. However, precision says nothing about how many real spam emails were missed. A model that flags only one email as spam and that email is spam has precision 1.0, even if it missed thousands of spam emails.
from sklearn.metrics import precision_score
from sklearn.metrics import confusion_matrix
import numpy as np
y_true = np.array([1, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0])
y_pred = np.array([1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0])
cm = confusion_matrix(y_true, y_pred)
TN, FP, FN, TP = cm.ravel()
precision_manual = TP / (TP + FP)
precision_sklearn = precision_score(y_true, y_pred)
print(f'TP={TP}, FP={FP}, FN={FN}, TN={TN}')
print(f'Precision (manual): {precision_manual:.3f}')
print(f'Precision (sklearn): {precision_sklearn:.3f}')Recall: Coverage of Actual Positives
Recall (also called sensitivity or true positive rate) answers: Of all the actual positive examples, what fraction did the model correctly identify?
Recall = TP / (TP + FN)
High recall means few missed positives. A cancer screening test with recall 0.95 catches 95% of patients who actually have cancer — only 5% are missed. However, recall says nothing about how many healthy patients were incorrectly flagged. High recall is essential when the cost of missing a positive (False Negative) is very high.
from sklearn.metrics import recall_score
import numpy as np
y_true = np.array([1, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0])
y_pred = np.array([1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0])
TP = ((y_pred == 1) & (y_true == 1)).sum()
FN = ((y_pred == 0) & (y_true == 1)).sum()
recall_manual = TP / (TP + FN)
recall_sklearn = recall_score(y_true, y_pred)
print(f'TP={TP}, FN={FN}')
print(f'Recall (manual): {recall_manual:.3f}')
print(f'Recall (sklearn): {recall_sklearn:.3f}')The Precision-Recall Trade-off
Precision and recall are in tension with each other. You can always increase recall to 1.0 by predicting positive for every single example — but precision will collapse because most of those predictions are false positives. Conversely, predicting positive only when you are extremely confident (very high threshold) improves precision but misses many real positives, reducing recall.
Moving the decision threshold changes the balance. Lowering the threshold flags more examples as positive: recall goes up, precision goes down. Raising the threshold makes the model more selective: precision goes up, recall goes down. There is no threshold that maximises both simultaneously in most real problems.
from sklearn.metrics import precision_score, recall_score
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
import numpy as np
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
X_train_s = StandardScaler().fit_transform(X_train)
X_test_s = StandardScaler().fit_transform(X_test)
model = LogisticRegression(max_iter=1000)
model.fit(X_train_s, y_train)
scores = model.predict_proba(X_test_s)[:, 1]
for threshold in [0.3, 0.5, 0.7, 0.9]:
y_pred = (scores >= threshold).astype(int)
p = precision_score(y_test, y_pred, zero_division=0)
r = recall_score(y_test, y_pred, zero_division=0)
print(f'Threshold {threshold}: Precision={p:.3f} Recall={r:.3f}')F1-Score: Balancing Precision and Recall
The F1-score combines precision and recall into a single number using the harmonic mean:
F1 = 2 × (Precision × Recall) / (Precision + Recall)
The harmonic mean penalises extreme imbalances: a model with precision 0.9 and recall 0.1 gets F1 = 2×0.9×0.1/1.0 = 0.18, not 0.5. This makes F1 much harder to game than a simple average. F1 = 1.0 only when both precision and recall are 1.0.
from sklearn.metrics import f1_score
import numpy as np
# Compare two classifiers
clf_a_prec, clf_a_rec = 0.9, 0.5 # high precision, low recall
clf_b_prec, clf_b_rec = 0.7, 0.7 # balanced
def f1_from_pr(precision, recall):
return 2 * precision * recall / (precision + recall)
f1_a = f1_from_pr(clf_a_prec, clf_a_rec)
f1_b = f1_from_pr(clf_b_prec, clf_b_rec)
print(f'Classifier A: P={clf_a_prec} R={clf_a_rec} F1={f1_a:.3f}')
print(f'Classifier B: P={clf_b_prec} R={clf_b_rec} F1={f1_b:.3f}')
print('\nClassifier B has better F1 despite lower precision.')F-beta Score: Weighting Precision and Recall
The F1-score treats precision and recall as equally important. The Fβ score lets you weight recall β times more than precision:
Fβ = (1 + β²) × Precision × Recall / (β² × Precision + Recall)
- β = 1: standard F1 (equal weight)
- β = 2 (F2): recall is twice as important — use for cancer screening where missing a case is critical
- β = 0.5 (F0.5): precision is twice as important — use for spam filters where false alarms are costly
from sklearn.metrics import fbeta_score
import numpy as np
y_true = np.array([1, 0, 1, 1, 0, 0, 1, 0, 1, 0])
y_pred = np.array([1, 0, 1, 0, 1, 0, 1, 0, 0, 0])
# beta=1 (F1): equal weight
f1 = fbeta_score(y_true, y_pred, beta=1)
# beta=2: emphasise recall
f2 = fbeta_score(y_true, y_pred, beta=2)
# beta=0.5: emphasise precision
f_half = fbeta_score(y_true, y_pred, beta=0.5)
print(f'F1 (beta=1.0): {f1:.3f}')
print(f'F2 (beta=2.0): {f2:.3f} <- emphasises recall')
print(f'F0.5 (beta=0.5): {f_half:.3f} <- emphasises precision')Precision-Recall Curves
Instead of evaluating at one threshold, a precision-recall curve plots precision vs recall across all possible thresholds. The shape of the curve tells you the best achievable trade-off for your model.
A model with high area under the precision-recall curve (PR-AUC) is better at finding positives without generating too many false alarms. PR-AUC is preferred over ROC-AUC when the positive class is rare, because it focuses on performance on the minority class rather than being influenced by the large pool of true negatives.
from sklearn.metrics import precision_recall_curve, average_precision_score
import matplotlib.pyplot as plt
# Get probability scores for positive class
scores = model.predict_proba(X_test_s)[:, 1]
precision_vals, recall_vals, _ = precision_recall_curve(y_test, scores)
ap = average_precision_score(y_test, scores)
plt.plot(recall_vals, precision_vals, 'b-', linewidth=2)
plt.xlabel('Recall')
plt.ylabel('Precision')
plt.title(f'Precision-Recall Curve (AP={ap:.3f})')
plt.grid(True, alpha=0.3)
plt.show()Macro, Micro, and Weighted Averages
For multi-class classification, scikit-learn reports several types of averages for precision, recall, and F1:
- Macro average: compute metric per class, then take unweighted mean. Treats all classes equally regardless of size.
- Weighted average: compute metric per class, then take mean weighted by the number of examples per class. Reflects overall dataset performance.
- Micro average: aggregate TP/FP/FN across all classes, then compute the metric. For accuracy this equals overall accuracy.
from sklearn.metrics import classification_report
from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
clf = DecisionTreeClassifier(max_depth=4)
clf.fit(X_train, y_train)
y_pred = clf.predict(X_test)
print(classification_report(y_test, y_pred,
target_names=['setosa', 'versicolor', 'virginica']))Choosing the Right Metric for Your Problem
A decision framework for metric selection:
- Balanced classes, equal error costs → Accuracy
- Imbalanced classes, cost of FP high → Precision (e.g., spam filter — don't trash good emails)
- Imbalanced classes, cost of FN high → Recall (e.g., disease screening — don't miss sick patients)
- Both FP and FN costly, need single metric → F1
- FN much more costly than FP → F2 (cancer detection, fraud where you can investigate)
- Need to evaluate across all thresholds → PR-AUC or ROC-AUC
Specificity and the Confusion Matrix Complete
Two additional metrics complete the picture of binary classifier performance:
- Specificity (True Negative Rate) = TN / (TN + FP): of all actual negatives, what fraction did the model correctly identify? High specificity means few false alarms. Used together with sensitivity (recall) in medical testing.
- Fall-out (False Positive Rate) = FP / (FP + TN): the complement of specificity. The x-axis of the ROC curve that you will study in the evaluation metrics lesson.
import numpy as np
from sklearn.metrics import confusion_matrix
y_true = np.array([1, 0, 1, 1, 0, 0, 1, 0, 1, 0])
y_pred = np.array([1, 0, 1, 0, 1, 0, 1, 0, 0, 0])
cm = confusion_matrix(y_true, y_pred)
TN, FP, FN, TP = cm.ravel()
sensitivity = TP / (TP + FN) # recall
specificity = TN / (TN + FP)
fpr = FP / (FP + TN) # false positive rate
print(f'Sensitivity (Recall): {sensitivity:.3f}')
print(f'Specificity (TNR): {specificity:.3f}')
print(f'False Positive Rate: {fpr:.3f}')
print(f'Sensitivity + Specificity should be > 1 for a useful model')Classification Report with Custom Thresholds
The default threshold of 0.5 is not always optimal. A workflow for finding the best threshold: compute precision, recall, and F1 at many thresholds, then pick the threshold that maximises the metric relevant to your problem.
For a fraud detection scenario, you might maximise F2 to prioritise recall. For a content moderation system, you might maximise precision to avoid incorrectly removing legitimate content. The precision_recall_curve function gives you all the data you need for this analysis.
import numpy as np
from sklearn.metrics import precision_recall_fscore_support
scores = model.predict_proba(X_test_s)[:, 1]
thresholds = np.arange(0.1, 0.95, 0.05)
best_f1, best_threshold = 0, 0.5
for t in thresholds:
y_pred = (scores >= t).astype(int)
p, r, f, _ = precision_recall_fscore_support(y_test, y_pred, average='binary', zero_division=0)
if f > best_f1:
best_f1, best_threshold = f, t
print(f't={t:.2f}: P={p:.3f} R={r:.3f} F1={f:.3f}')
print(f'\nBest F1={best_f1:.3f} at threshold={best_threshold:.2f}')Quick Check
Test your understanding of Machine Learning with Python concepts from this lesson.
Lesson Recap
In this lesson you learned: precision measures quality of positive predictions (TP/(TP+FP)), recall measures coverage of actual positives (TP/(TP+FN)), and the F1-score harmonic mean balances both — while Fβ lets you weight recall or precision more heavily based on your application's error cost structure. Next up we tackle the fundamental question of model evaluation: why you cannot evaluate a model on the data it was trained on, and how to properly estimate generalisation performance with held-out test sets.
Frequently asked questions
Is the “Precision, Recall, and F1-Score in Practice” lesson free?
Yes — the full text of “Precision, Recall, and F1-Score in Practice” is free to read here on the web, and the Machine Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Machine Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “Precision, Recall, and F1-Score in Practice”?
Learners will calculate precision and recall, understand the precision-recall trade-off, and choose the right metric for imbalanced real-world tasks. You practise Machine Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Machine Learning Academy?
No prior experience is required. Machine Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Precision, Recall, and F1-Score in Practice” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Machine Learning Academy lesson?
Yes. Every Machine Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- From Regression to Classification: Threshold Decisions
- Logistic Regression and the Sigmoid Function
- The Confusion Matrix Explained
- Precision, Recall, and F1-Score in Practice