Pick Metrics That Matter
Choose business KPIs, not just offline accuracy.
Accuracy Is Not the Goal
A challenger can win on offline accuracy yet lose money. The metric that decides your A/B test should be the business outcome you actually care about. 💡
Offline vs Online Metrics
Offline metrics like F1 or AUC come from a static test set. Online metrics come from live user behavior, and only those tell you the real production impact.