O teorema de Bayes em linguagem simples
Trabalhe com um exemplo concreto de teste médico para desenvolver a intuição sobre probabilidades a priori, de verossimilhança e a posteriori sem matemática avançada.
O teorema de Bayes em linguagem simples é uma aula grátis de Machine Learning Academy no CoddyKit. Esta é a aula 1 de 4. Você pode ler a aula completa abaixo gratuitamente — depois pratica ao vivo no navegador com um editor de código integrado e um tutor de IA 24/7. Faz parte do caminho de aprendizado de Machine Learning Academy, e seu progresso é sincronizado entre a web e o app CoddyKit. O curso de Machine Learning Academy inclui 4 aulas no total.
Partes desta aula ainda não foram traduzidas e aparecem em inglês.
What Is Bayes' Theorem?
Bayes' theorem is a mathematical rule for updating beliefs in light of new evidence. It answers the question: Given that I observed X, how probable is hypothesis H? The formula is: P(H|X) = P(X|H) * P(H) / P(X). In plain English: the posterior probability of H given X equals the likelihood of observing X if H were true, times the prior probability of H, divided by the overall probability of observing X. Bayes' theorem is fundamental to machine learning, statistics, and rational reasoning under uncertainty.
# Bayes' theorem components:
# P(H|X) = Posterior -- What we want to know
# 'Probability of H given we observed X'
# P(X|H) = Likelihood -- How likely is X if H is true?
# P(H) = Prior -- Our belief in H before seeing X
# P(X) = Evidence -- Overall probability of observing X
# Rearranged:
# Posterior = (Likelihood * Prior) / Evidence
print('Posterior = (Likelihood * Prior) / Evidence')A Medical Test Example
Let's build intuition with a classic example. A rare disease affects 1% of the population. A test is 95% accurate: if you have the disease, it tests positive 95% of the time; if you don't, it tests negative 95% of the time (5% false positive rate). If you test positive, what is the probability you actually have the disease? Most people's intuition says 95%, but Bayes' theorem reveals the answer is much lower — because the disease is so rare, most positives are actually false positives.
# Medical test: Bayes' theorem applied
P_disease = 0.01 # Prior: 1% have the disease
P_no_disease = 0.99 # 99% are healthy
P_pos_given_disease = 0.95 # True positive rate (sensitivity)
P_pos_given_no_disease = 0.05 # False positive rate
# P(positive) = total probability of a positive test
P_positive = (P_pos_given_disease * P_disease +
P_pos_given_no_disease * P_no_disease)
# Bayes: P(disease | positive)
P_disease_given_pos = (P_pos_given_disease * P_disease) / P_positive
print(f'P(disease | positive test) = {P_disease_given_pos:.2%}')
# Only ~16%! Not 95% -- the low prior dominatesPrior, Likelihood, and Posterior Explained
The three key terms in Bayes' theorem: Prior P(H) is your belief about H before seeing any evidence — in the medical example, the 1% disease prevalence. Likelihood P(X|H) is how well the evidence X is explained by hypothesis H — the 95% true positive rate. Posterior P(H|X) is your updated belief after incorporating the evidence. The posterior from one observation becomes the prior for the next — Bayes' theorem describes a continuous learning process where beliefs are updated as evidence accumulates.
# Sequential Bayesian updating
# Start with 1% prior, observe 3 positive tests
prior = 0.01
P_pos_given_disease = 0.95
P_pos_given_no_disease = 0.05
for test_num in range(1, 4):
likelihood_pos = P_pos_given_disease
P_positive = likelihood_pos * prior + P_pos_given_no_disease * (1 - prior)
posterior = (likelihood_pos * prior) / P_positive
print(f'After test {test_num}: P(disease) = {posterior:.3%}')
prior = posterior # Posterior becomes new prior
# Three positive tests raise probability substantiallyBayes' Theorem for Classification
In machine learning classification, Bayes' theorem gives us the probability of each class given the observed features: P(class | features) ∝ P(features | class) * P(class). We predict the class with the highest posterior probability. P(class) is estimated from class frequencies in training data (prior). P(features | class) is estimated from the feature distribution within each class (likelihood). Naive Bayes simplifies the likelihood calculation by assuming features are independent given the class — a 'naive' assumption that often works surprisingly well in practice.
# Classification with Bayes' theorem
# Predict P(spam | email_features) vs P(ham | email_features)
# Prior (from training data)
P_spam = 0.3 # 30% of emails are spam
P_ham = 0.7 # 70% are ham
# Likelihood: P(features | class) from training
# (simplified: single word 'offer' seen)
P_offer_given_spam = 0.6 # 'offer' appears in 60% of spam
P_offer_given_ham = 0.1 # 'offer' appears in 10% of ham
# Unnormalised posteriors (ignore P(offer) -- same denominator)
posterior_spam = P_offer_given_spam * P_spam # 0.18
posterior_ham = P_offer_given_ham * P_ham # 0.07
print('Unnormalised: spam=', posterior_spam, 'ham=', posterior_ham)
print('Predicted class: spam' if posterior_spam > posterior_ham else 'ham')The Law of Total Probability
The denominator in Bayes' theorem, P(X), is often computed using the law of total probability: P(X) = sum over all classes c of P(X|c) * P(c). This ensures the posterior probabilities across all classes sum to 1. In the medical example, P(positive) = P(positive|disease)*P(disease) + P(positive|no disease)*P(no disease). In Naive Bayes classification, we usually skip computing P(X) because it is the same for all classes — we just compare unnormalised posteriors to find the most probable class.
# Law of total probability: P(X) = sum_c P(X|c) * P(c)
classes = ['spam', 'ham', 'newsletter']
priors = [0.3, 0.5, 0.2] # Must sum to 1.0
P_word_given_class = [0.5, 0.1, 0.3] # Likelihood of 'free' in each class
# Total probability of seeing the word 'free'
P_free = sum(lik * pri for lik, pri in zip(P_word_given_class, priors))
print(f'P(free) = {P_free}')
# Posteriors (normalised)
for cls, pri, lik in zip(classes, priors, P_word_given_class):
posterior = lik * pri / P_free
print(f'P({cls} | free) = {posterior:.3f}')From Formula to Algorithm
Bayes' theorem translates directly into a classification algorithm. Given a labelled training set: Step 1: estimate P(class) for each class from class frequencies. Step 2: for each feature-class combination, estimate P(feature | class). Step 3: for a new sample, compute P(class | features) ∝ P(features | class) * P(class) for each class. Step 4: predict the class with the highest posterior. The 'naive' approximation in Naive Bayes simplifies Step 2: assume P(features|class) = product of P(feature_i | class) for all features.
# Naive Bayes algorithm in pseudocode
# Training:
# For each class c:
# prior[c] = count(class==c) / total_samples
# For each feature f:
# likelihood[c][f] = P(feature_f | class==c) # from training data
# Prediction for new sample x:
# For each class c:
# log_prob[c] = log(prior[c])
# For each feature f:
# log_prob[c] += log(likelihood[c][f=x_f]) # naive: assume independence
# return argmax(log_prob) # class with highest log-posterior
print('Naive independence assumption: P(f1,f2,...|c) = P(f1|c)*P(f2|c)*...*P(fn|c)')Why Log Probabilities Avoid Underflow
When multiplying many small probabilities together, the result quickly underflows to 0 in floating-point arithmetic — even though mathematically the value is a tiny but non-zero number. For example, a document with 100 words, each with P(word|class)=0.01, gives 0.01^100 = 10^{-200}, which is below the smallest float64 value. Working in log space converts multiplication to addition: log(a*b) = log(a) + log(b). Since we compare log posteriors (not actual posteriors), the argmax decision is identical. Naive Bayes implementations always operate in log space for numerical stability.
import numpy as np
# Direct multiplication: underflows
probs = [0.1] * 100 # 100 features, each P=0.1
product = np.prod(probs)
print('Direct product:', product) # 0.0 -- underflow!
# Log space: numerically stable
log_sum = np.sum(np.log(probs))
print('Log sum:', log_sum) # -100 * log(10) -- valid
# Compare two classes
log_prob_A = np.sum(np.log([0.1] * 100))
log_prob_B = np.sum(np.log([0.2] * 100))
print('Class A log-prob:', log_prob_A)
print('Class B log-prob:', log_prob_B)
print('Predicted:', 'A' if log_prob_A > log_prob_B else 'B')Bayes Optimal Classifier: The Gold Standard
The Bayes optimal classifier is the theoretically best possible classifier for a given data distribution. It predicts the class with the highest true posterior probability P(class|features). No other classifier can achieve a lower expected error rate on that distribution. In practice, we cannot use the Bayes optimal classifier because we do not know the true distributions — we can only estimate them from finite data. Naive Bayes is an approximation of the Bayes optimal classifier under the independence assumption. When that assumption holds, Naive Bayes is the optimal classifier.
# The Bayes error rate is the irreducible error
# Even a perfect model cannot beat it on a given distribution
# Example: predicting coin flip from noisy signal
import numpy as np
np.random.seed(42)
# True label: 50/50 coin
y_true = np.random.choice([0, 1], size=1000)
# Signal: 70% correlated with true label
signal = np.where(np.random.rand(1000) < 0.7, y_true, 1-y_true)
# Optimal prediction: just use the signal
acc = (signal == y_true).mean()
print(f'Optimal classifier accuracy: {acc:.3f}')
print(f'Bayes error rate (irreducible): {1-0.70:.3f} = 30%')Real-World Applications of Bayes' Theorem
Bayes' theorem is not just a classroom formula — it powers many real systems. Email spam filters compute P(spam | words) from word frequencies in known spam and ham. Medical diagnosis systems update disease probabilities as test results arrive. Search engines use Bayesian ranking to combine query relevance and document popularity. Autonomous vehicles use Bayesian filtering (Kalman filter, particle filter) to track their position given noisy sensor readings. A/B testing in industry increasingly uses Bayesian frameworks that output probability-of-being-best rather than p-values.
# Real-world Bayes: A/B test - which version is better?
# After 100 conversions from 1000 visitors for version A
# and 120 conversions from 1000 visitors for version B:
from scipy import stats
# Beta distribution as posterior for conversion rate
alpha_A, beta_A = 100 + 1, 900 + 1 # Beta(successes+1, failures+1)
alpha_B, beta_B = 120 + 1, 880 + 1
# Monte Carlo: P(B is better than A)
samples_A = stats.beta(alpha_A, beta_A).rvs(100000)
samples_B = stats.beta(alpha_B, beta_B).rvs(100000)
P_B_better = (samples_B > samples_A).mean()
print(f'P(B is better than A): {P_B_better:.3f}')The Independence Assumption: When Is It Valid?
The 'naive' part of Naive Bayes assumes features are conditionally independent given the class. For text data, this means assuming each word's presence is independent of other words (given the class). In reality, words like 'credit' and 'card' appear together — they are correlated. However, empirical studies show Naive Bayes works well despite violated independence because: (1) ranking for classification only needs the correct ordering of posteriors, not accurate probabilities; (2) with many features, independent signals combine well; (3) the simplicity prevents overfitting on small datasets.
# When does naive independence work?
# Even with correlated features, naive bayes often ranks classes correctly
import numpy as np
# Two correlated features that both indicate spam
words = {'buy': 0, 'now': 1, 'click': 2, 'here': 3}
# Spam email often has all four; ham rarely does
# Independence assumption ignores that 'buy' and 'now' co-occur
# But the combined signal is still strong for detecting spam
P_word_spam = [0.7, 0.6, 0.5, 0.6]
P_word_ham = [0.1, 0.1, 0.05, 0.1]
# For email with all 4 words
log_spam = np.sum(np.log(P_word_spam))
log_ham = np.sum(np.log(P_word_ham))
print('Log P(features|spam):', log_spam.round(2))
print('Log P(features|ham): ', log_ham.round(2))
print('Predicted: SPAM' if log_spam > log_ham else 'HAM')Bayes Factor: Comparing Hypotheses
The Bayes factor is the ratio of likelihoods for two competing hypotheses: BF = P(evidence | H1) / P(evidence | H0). Unlike p-values, which only tell you whether to reject the null hypothesis, the Bayes factor quantifies how much the evidence favours one hypothesis over another. BF > 10 is strong evidence for H1; BF < 0.1 is strong evidence for H0; values between 1/3 and 3 are inconclusive. Bayes factors are increasingly used in science and industry for A/B testing and hypothesis evaluation because they naturally incorporate prior information and give interpretable evidence strength.
# Bayes Factor example: comparing two models
# H0: coin is fair (p=0.5)
# H1: coin is biased (p=0.7)
# Observed: 8 heads in 10 flips
from scipy.stats import binom
observed_heads = 8
n_flips = 10
# Likelihood under each hypothesis
L_H0 = binom.pmf(observed_heads, n_flips, p=0.5)
L_H1 = binom.pmf(observed_heads, n_flips, p=0.7)
BF = L_H1 / L_H0
print(f'P(8 heads | fair coin) = {L_H0:.4f}')
print(f'P(8 heads | p=0.7) = {L_H1:.4f}')
print(f'Bayes Factor (H1/H0) = {BF:.2f}')
print('BF > 3: moderate evidence for biased coin')Quick Check
Test your understanding of Machine Learning with Python concepts from this lesson.
Lesson Recap
In this lesson you learned: Bayes' theorem updates prior beliefs with evidence to produce posterior beliefs, the three components (prior, likelihood, posterior) and how they interact, and why low priors can dominate even high-accuracy tests (as in the medical diagnosis example). Next up we explore Bag of Words with CountVectorizer and TfidfVectorizer to convert text into numbers for Naive Bayes classification.
Perguntas Frequentes
A aula “O teorema de Bayes em linguagem simples” é grátis?
Sim — o texto completo de “O teorema de Bayes em linguagem simples” é grátis para ler aqui na web. Para praticá-la interativamente (um editor de código integrado e um tutor de IA 24/7) e desbloquear o restante do curso de Machine Learning Academy, atualize para CoddyKit PRO. O curso de Machine Learning Academy inclui 4 aulas no total.
O que vou aprender em “O teorema de Bayes em linguagem simples”?
Trabalhe com um exemplo concreto de teste médico para desenvolver a intuição sobre probabilidades a priori, de verossimilhança e a posteriori sem matemática avançada. Você pratica Machine Learning Academy com código prático que executa diretamente no navegador, e um tutor de IA 24/7 responde suas dúvidas enquanto trabalha na aula.
Preciso ter experiência prévia para começar Machine Learning Academy?
Nenhuma experiência prévia é necessária. Machine Learning Academy no CoddyKit é estruturado para alunos iniciantes até avançados, então você pode começar aqui ou desde o início e aprender no seu ritmo. Esta é a aula 1 de 4.
Quanto tempo leva a aula “O teorema de Bayes em linguagem simples”?
A maioria das aulas CoddyKit leva cerca de 5–10 minutos. Cada uma é compacta e interativa, então você faz progresso constante e retoma exatamente de onde parou entre web e app.
Posso escrever e executar código nesta aula de Machine Learning Academy?
Sim. Cada aula de Machine Learning Academy inclui um editor de código integrado, então você escreve e executa código real direto no navegador e recebe feedback de IA instantaneamente — nenhuma configuração local necessária.
Todas as aulas deste curso
- O teorema de Bayes em linguagem simples
- Saco de palavras: CountVectorizer e TfidfVectorizer
- Treinamento de um classificador Naive Bayes multinomial
- Suavização de Laplace e o problema da probabilidade zero