用通俗语言理解贝叶斯定理
您将通过具体的医学检验示例，在不涉及复杂数学的情况下建立对先验概率、似然和后验概率的直觉
用通俗语言理解贝叶斯定理 是 CoddyKit 上的免费 Machine Learning Academy 课时。 这是第 1 节课，共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 Machine Learning Academy 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Machine Learning Academy 课程共包含 4 节课。
本课时的部分内容尚未翻译，以英文显示。
What Is Bayes' Theorem?
Bayes' theorem is a mathematical rule for updating beliefs in light of new evidence. It answers the question: Given that I observed X, how probable is hypothesis H? The formula is: P(H|X) = P(X|H) * P(H) / P(X). In plain English: the posterior probability of H given X equals the likelihood of observing X if H were true, times the prior probability of H, divided by the overall probability of observing X. Bayes' theorem is fundamental to machine learning, statistics, and rational reasoning under uncertainty.
# Bayes' theorem components:
# P(H|X) = Posterior -- What we want to know
# 'Probability of H given we observed X'
# P(X|H) = Likelihood -- How likely is X if H is true?
# P(H) = Prior -- Our belief in H before seeing X
# P(X) = Evidence -- Overall probability of observing X
# Rearranged:
# Posterior = (Likelihood * Prior) / Evidence
print('Posterior = (Likelihood * Prior) / Evidence')A Medical Test Example
Let's build intuition with a classic example. A rare disease affects 1% of the population. A test is 95% accurate: if you have the disease, it tests positive 95% of the time; if you don't, it tests negative 95% of the time (5% false positive rate). If you test positive, what is the probability you actually have the disease? Most people's intuition says 95%, but Bayes' theorem reveals the answer is much lower — because the disease is so rare, most positives are actually false positives.
# Medical test: Bayes' theorem applied
P_disease = 0.01 # Prior: 1% have the disease
P_no_disease = 0.99 # 99% are healthy
P_pos_given_disease = 0.95 # True positive rate (sensitivity)
P_pos_given_no_disease = 0.05 # False positive rate
# P(positive) = total probability of a positive test
P_positive = (P_pos_given_disease * P_disease +
P_pos_given_no_disease * P_no_disease)
# Bayes: P(disease | positive)
P_disease_given_pos = (P_pos_given_disease * P_disease) / P_positive
print(f'P(disease | positive test) = {P_disease_given_pos:.2%}')
# Only ~16%! Not 95% -- the low prior dominatesPrior, Likelihood, and Posterior Explained
The three key terms in Bayes' theorem: Prior P(H) is your belief about H before seeing any evidence — in the medical example, the 1% disease prevalence. Likelihood P(X|H) is how well the evidence X is explained by hypothesis H — the 95% true positive rate. Posterior P(H|X) is your updated belief after incorporating the evidence. The posterior from one observation becomes the prior for the next — Bayes' theorem describes a continuous learning process where beliefs are updated as evidence accumulates.
# Sequential Bayesian updating
# Start with 1% prior, observe 3 positive tests
prior = 0.01
P_pos_given_disease = 0.95
P_pos_given_no_disease = 0.05
for test_num in range(1, 4):
likelihood_pos = P_pos_given_disease
P_positive = likelihood_pos * prior + P_pos_given_no_disease * (1 - prior)
posterior = (likelihood_pos * prior) / P_positive
print(f'After test {test_num}: P(disease) = {posterior:.3%}')
prior = posterior # Posterior becomes new prior
# Three positive tests raise probability substantiallyBayes' Theorem for Classification
In machine learning classification, Bayes' theorem gives us the probability of each class given the observed features: P(class | features) ∝ P(features | class) * P(class). We predict the class with the highest posterior probability. P(class) is estimated from class frequencies in training data (prior). P(features | class) is estimated from the feature distribution within each class (likelihood). Naive Bayes simplifies the likelihood calculation by assuming features are independent given the class — a 'naive' assumption that often works surprisingly well in practice.
# Classification with Bayes' theorem
# Predict P(spam | email_features) vs P(ham | email_features)
# Prior (from training data)
P_spam = 0.3 # 30% of emails are spam
P_ham = 0.7 # 70% are ham
# Likelihood: P(features | class) from training
# (simplified: single word 'offer' seen)
P_offer_given_spam = 0.6 # 'offer' appears in 60% of spam
P_offer_given_ham = 0.1 # 'offer' appears in 10% of ham
# Unnormalised posteriors (ignore P(offer) -- same denominator)
posterior_spam = P_offer_given_spam * P_spam # 0.18
posterior_ham = P_offer_given_ham * P_ham # 0.07
print('Unnormalised: spam=', posterior_spam, 'ham=', posterior_ham)
print('Predicted class: spam' if posterior_spam > posterior_ham else 'ham')The Law of Total Probability
The denominator in Bayes' theorem, P(X), is often computed using the law of total probability: P(X) = sum over all classes c of P(X|c) * P(c). This ensures the posterior probabilities across all classes sum to 1. In the medical example, P(positive) = P(positive|disease)*P(disease) + P(positive|no disease)*P(no disease). In Naive Bayes classification, we usually skip computing P(X) because it is the same for all classes — we just compare unnormalised posteriors to find the most probable class.
# Law of total probability: P(X) = sum_c P(X|c) * P(c)
classes = ['spam', 'ham', 'newsletter']
priors = [0.3, 0.5, 0.2] # Must sum to 1.0
P_word_given_class = [0.5, 0.1, 0.3] # Likelihood of 'free' in each class
# Total probability of seeing the word 'free'
P_free = sum(lik * pri for lik, pri in zip(P_word_given_class, priors))
print(f'P(free) = {P_free}')
# Posteriors (normalised)
for cls, pri, lik in zip(classes, priors, P_word_given_class):
posterior = lik * pri / P_free
print(f'P({cls} | free) = {posterior:.3f}')From Formula to Algorithm
Bayes' theorem translates directly into a classification algorithm. Given a labelled training set: Step 1: estimate P(class) for each class from class frequencies. Step 2: for each feature-class combination, estimate P(feature | class). Step 3: for a new sample, compute P(class | features) ∝ P(features | class) * P(class) for each class. Step 4: predict the class with the highest posterior. The 'naive' approximation in Naive Bayes simplifies Step 2: assume P(features|class) = product of P(feature_i | class) for all features.
# Naive Bayes algorithm in pseudocode
# Training:
# For each class c:
# prior[c] = count(class==c) / total_samples
# For each feature f:
# likelihood[c][f] = P(feature_f | class==c) # from training data
# Prediction for new sample x:
# For each class c:
# log_prob[c] = log(prior[c])
# For each feature f:
# log_prob[c] += log(likelihood[c][f=x_f]) # naive: assume independence
# return argmax(log_prob) # class with highest log-posterior
print('Naive independence assumption: P(f1,f2,...|c) = P(f1|c)*P(f2|c)*...*P(fn|c)')Why Log Probabilities Avoid Underflow
When multiplying many small probabilities together, the result quickly underflows to 0 in floating-point arithmetic — even though mathematically the value is a tiny but non-zero number. For example, a document with 100 words, each with P(word|class)=0.01, gives 0.01^100 = 10^{-200}, which is below the smallest float64 value. Working in log space converts multiplication to addition: log(a*b) = log(a) + log(b). Since we compare log posteriors (not actual posteriors), the argmax decision is identical. Naive Bayes implementations always operate in log space for numerical stability.
import numpy as np
# Direct multiplication: underflows
probs = [0.1] * 100 # 100 features, each P=0.1
product = np.prod(probs)
print('Direct product:', product) # 0.0 -- underflow!
# Log space: numerically stable
log_sum = np.sum(np.log(probs))
print('Log sum:', log_sum) # -100 * log(10) -- valid
# Compare two classes
log_prob_A = np.sum(np.log([0.1] * 100))
log_prob_B = np.sum(np.log([0.2] * 100))
print('Class A log-prob:', log_prob_A)
print('Class B log-prob:', log_prob_B)
print('Predicted:', 'A' if log_prob_A > log_prob_B else 'B')Bayes Optimal Classifier: The Gold Standard
The Bayes optimal classifier is the theoretically best possible classifier for a given data distribution. It predicts the class with the highest true posterior probability P(class|features). No other classifier can achieve a lower expected error rate on that distribution. In practice, we cannot use the Bayes optimal classifier because we do not know the true distributions — we can only estimate them from finite data. Naive Bayes is an approximation of the Bayes optimal classifier under the independence assumption. When that assumption holds, Naive Bayes is the optimal classifier.
# The Bayes error rate is the irreducible error
# Even a perfect model cannot beat it on a given distribution
# Example: predicting coin flip from noisy signal
import numpy as np
np.random.seed(42)
# True label: 50/50 coin
y_true = np.random.choice([0, 1], size=1000)
# Signal: 70% correlated with true label
signal = np.where(np.random.rand(1000) < 0.7, y_true, 1-y_true)
# Optimal prediction: just use the signal
acc = (signal == y_true).mean()
print(f'Optimal classifier accuracy: {acc:.3f}')
print(f'Bayes error rate (irreducible): {1-0.70:.3f} = 30%')Real-World Applications of Bayes' Theorem
Bayes' theorem is not just a classroom formula — it powers many real systems. Email spam filters compute P(spam | words) from word frequencies in known spam and ham. Medical diagnosis systems update disease probabilities as test results arrive. Search engines use Bayesian ranking to combine query relevance and document popularity. Autonomous vehicles use Bayesian filtering (Kalman filter, particle filter) to track their position given noisy sensor readings. A/B testing in industry increasingly uses Bayesian frameworks that output probability-of-being-best rather than p-values.
# Real-world Bayes: A/B test - which version is better?
# After 100 conversions from 1000 visitors for version A
# and 120 conversions from 1000 visitors for version B:
from scipy import stats
# Beta distribution as posterior for conversion rate
alpha_A, beta_A = 100 + 1, 900 + 1 # Beta(successes+1, failures+1)
alpha_B, beta_B = 120 + 1, 880 + 1
# Monte Carlo: P(B is better than A)
samples_A = stats.beta(alpha_A, beta_A).rvs(100000)
samples_B = stats.beta(alpha_B, beta_B).rvs(100000)
P_B_better = (samples_B > samples_A).mean()
print(f'P(B is better than A): {P_B_better:.3f}')The Independence Assumption: When Is It Valid?
The 'naive' part of Naive Bayes assumes features are conditionally independent given the class. For text data, this means assuming each word's presence is independent of other words (given the class). In reality, words like 'credit' and 'card' appear together — they are correlated. However, empirical studies show Naive Bayes works well despite violated independence because: (1) ranking for classification only needs the correct ordering of posteriors, not accurate probabilities; (2) with many features, independent signals combine well; (3) the simplicity prevents overfitting on small datasets.
# When does naive independence work?
# Even with correlated features, naive bayes often ranks classes correctly
import numpy as np
# Two correlated features that both indicate spam
words = {'buy': 0, 'now': 1, 'click': 2, 'here': 3}
# Spam email often has all four; ham rarely does
# Independence assumption ignores that 'buy' and 'now' co-occur
# But the combined signal is still strong for detecting spam
P_word_spam = [0.7, 0.6, 0.5, 0.6]
P_word_ham = [0.1, 0.1, 0.05, 0.1]
# For email with all 4 words
log_spam = np.sum(np.log(P_word_spam))
log_ham = np.sum(np.log(P_word_ham))
print('Log P(features|spam):', log_spam.round(2))
print('Log P(features|ham): ', log_ham.round(2))
print('Predicted: SPAM' if log_spam > log_ham else 'HAM')Bayes Factor: Comparing Hypotheses
The Bayes factor is the ratio of likelihoods for two competing hypotheses: BF = P(evidence | H1) / P(evidence | H0). Unlike p-values, which only tell you whether to reject the null hypothesis, the Bayes factor quantifies how much the evidence favours one hypothesis over another. BF > 10 is strong evidence for H1; BF < 0.1 is strong evidence for H0; values between 1/3 and 3 are inconclusive. Bayes factors are increasingly used in science and industry for A/B testing and hypothesis evaluation because they naturally incorporate prior information and give interpretable evidence strength.
# Bayes Factor example: comparing two models
# H0: coin is fair (p=0.5)
# H1: coin is biased (p=0.7)
# Observed: 8 heads in 10 flips
from scipy.stats import binom
observed_heads = 8
n_flips = 10
# Likelihood under each hypothesis
L_H0 = binom.pmf(observed_heads, n_flips, p=0.5)
L_H1 = binom.pmf(observed_heads, n_flips, p=0.7)
BF = L_H1 / L_H0
print(f'P(8 heads | fair coin) = {L_H0:.4f}')
print(f'P(8 heads | p=0.7) = {L_H1:.4f}')
print(f'Bayes Factor (H1/H0) = {BF:.2f}')
print('BF > 3: moderate evidence for biased coin')Quick Check
Test your understanding of Machine Learning with Python concepts from this lesson.
Lesson Recap
In this lesson you learned: Bayes' theorem updates prior beliefs with evidence to produce posterior beliefs, the three components (prior, likelihood, posterior) and how they interact, and why low priors can dominate even high-accuracy tests (as in the medical diagnosis example). Next up we explore Bag of Words with CountVectorizer and TfidfVectorizer to convert text into numbers for Naive Bayes classification.
常见问题解答
「用通俗语言理解贝叶斯定理」课时是免费的吗？
是的 — 「用通俗语言理解贝叶斯定理」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Machine Learning Academy 课程的其余内容，请升级到 CoddyKit PRO。 Machine Learning Academy 课程共包含 4 节课。
「用通俗语言理解贝叶斯定理」这节课中我会学到什么？
您将通过具体的医学检验示例，在不涉及复杂数学的情况下建立对先验概率、似然和后验概率的直觉 你通过在浏览器中直接运行的动手代码来练习 Machine Learning Academy，全天候 AI 导师会在你学习这节课的过程中回答你的问题。
学习 Machine Learning Academy 需要有经验吗？
无需任何先前经验。CoddyKit 上的 Machine Learning Academy 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。 这是第 1 节课，共 4 节。
「用通俗语言理解贝叶斯定理」课时需要多长时间？
大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。
我能在这节 Machine Learning Academy 课中编写并运行代码吗？
能。每节 Machine Learning Academy 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。