Hybrid Systems and Evaluation Metrics
Combining collaborative + content-based, RMSE, MAE, Precision@K, Recall@K.
Hybrid Systems and Evaluation Metrics is a free Learn AI with Python lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Learn AI with Python learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Why Hybrid Systems?
Collaborative filtering struggles with cold-start; content-based filtering over-specializes. A hybrid system combines them so each method covers the other weaknesses, usually beating either alone.
Weighted Combination
The simplest hybrid blends scores with weights that sum to 1. For example 0.6 * CF + 0.4 * CB leans on collaborative signals but keeps content as a fallback and diversifier.
final_score = 0.6 * cf_score + 0.4 * cb_scoreTuning the Weights
The 0.6 / 0.4 split is a starting point. Tune the weights on a validation set: when users have rich history, raise the CF weight; for new items, raise the CB weight. You can even switch weights per user.
Other Hybrid Strategies
- Weighted: blend scores (shown above).
- Switching: pick CF or CB depending on data availability.
- Feature combination: feed both signals into one model.
Weighted blending is the most common and easiest to reason about.
Evaluating: Two Question Types
Evaluation splits into two questions:
- Rating prediction: how close are predicted ratings? Use RMSE/MAE.
- Ranking quality: are the right items near the top? Use Precision@K, Recall@K, NDCG@K.
RMSE for Rating Prediction
RMSE measures average squared error between predicted and actual ratings, then square-roots it. It heavily penalizes large misses and is the standard rating-accuracy metric.
import numpy as np
def rmse(pred, actual):
return np.sqrt(np.mean((np.array(pred) - np.array(actual)) ** 2))Precision@K
Precision@K = (relevant items in the top K) / K. It answers: of the K things I recommended, what fraction did the user actually like? Higher means fewer wasted slots.
def precision_at_k(recommended, relevant, k):
top_k = recommended[:k]
hits = len(set(top_k) & set(relevant))
return hits / kRecall@K
Recall@K = (relevant items in the top K) / (total relevant items). It answers: of everything the user would like, what fraction did I surface in the top K?
def recall_at_k(recommended, relevant, k):
top_k = recommended[:k]
hits = len(set(top_k) & set(relevant))
return hits / len(relevant) if relevant else 0.0Precision vs Recall Trade-off
Precision favors being right about what you show; recall favors covering everything relevant. Recommenders usually emphasize Precision@K because users only see a short list, so the top slots matter most.
NDCG@K for Ranking Quality
NDCG@K (Normalized Discounted Cumulative Gain) rewards putting highly relevant items near the top. A relevant item at rank 1 counts more than the same item at rank 10, because gains are discounted by position.
How NDCG Works
DCG sums each item relevance divided by log of its rank. NDCG divides DCG by the ideal DCG (perfect ordering), giving a 0-1 score where 1 is a perfectly ranked list.
import numpy as np
def dcg(rels):
return sum(r / np.log2(i + 2) for i, r in enumerate(rels))
def ndcg_at_k(rels, k):
ideal = sorted(rels, reverse=True)
return dcg(rels[:k]) / dcg(ideal[:k]) if dcg(ideal[:k]) else 0.0Quick Check
Test your evaluation-metrics knowledge.
Recap
You built a hybrid recommender (e.g. 0.6 * CF + 0.4 * CB) and learned its evaluation toolkit: RMSE for rating accuracy, Precision@K and Recall@K for retrieval, and NDCG@K for ranking quality. That ends the Recommendation Systems course. Next course: MLOps Fundamentals.
Frequently asked questions
Is the “Hybrid Systems and Evaluation Metrics” lesson free?
Yes — the full text of “Hybrid Systems and Evaluation Metrics” is free to read here on the web, and the Learn AI with Python course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Learn AI with Python course, upgrade to CoddyKit PRO.
What will I learn in “Hybrid Systems and Evaluation Metrics”?
Combining collaborative + content-based, RMSE, MAE, Precision@K, Recall@K. You practise Learn AI with Python with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Learn AI with Python?
No prior experience is required. Learn AI with Python on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Hybrid Systems and Evaluation Metrics” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Learn AI with Python lesson?
Yes. Every Learn AI with Python lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Collaborative Filtering: User-Based and Item-Based
- Matrix Factorization with SVD
- Content-Based Filtering
- Hybrid Systems and Evaluation Metrics