Machine Learning Academy · 课时

逻辑回归与 Sigmoid 函数

您将应用 Sigmoid 生成概率，将输出解读为类别置信度，并使用 scikit-learn 训练逻辑模型

逻辑回归与 Sigmoid 函数 是 CoddyKit 上的免费 Machine Learning Academy 课时。这是第 2 节课，共 4 节。你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。这是 Machine Learning Academy 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Machine Learning Academy 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

Logistic Regression: A Probabilistic Classifier

Logistic regression is the go-to algorithm for binary classification. Despite its name, it is a classifier, not a regressor. It extends the linear model by squashing the output through a special function called the sigmoid, ensuring predictions are always valid probabilities between 0 and 1.

Logistic regression is one of the most widely deployed ML models in production. It is fast to train, interpretable, and provides calibrated probability estimates. It is often the first classifier you should try on a new binary classification problem before reaching for more complex methods.

The Sigmoid Function

The sigmoid function (also called the logistic function) maps any real number to the range (0, 1):

σ(z) = 1 / (1 + e^(-z))

Key properties: σ(0) = 0.5, σ(+∞) → 1, σ(-∞) → 0. The S-shaped curve smoothly transitions from 0 to 1. When z is a large positive number (strong signal for class 1), the output approaches 1. When z is a large negative number (strong signal for class 0), the output approaches 0.

import numpy as np
import matplotlib.pyplot as plt

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

z = np.linspace(-10, 10, 200)
y = sigmoid(z)

plt.plot(z, y, 'b-', linewidth=2)
plt.axhline(y=0.5, color='red', linestyle='--', alpha=0.5, label='threshold=0.5')
plt.axvline(x=0, color='gray', linestyle='--', alpha=0.5)
plt.xlabel('z (linear score)')
plt.ylabel('sigmoid(z) = probability')
plt.title('The Sigmoid Function')
plt.legend()
plt.show()

print('sigmoid(0):', sigmoid(0))   # 0.5
print('sigmoid(5):', sigmoid(5))   # ~0.993
print('sigmoid(-5):', sigmoid(-5)) # ~0.007

How Logistic Regression Computes Probability

Logistic regression first computes the same linear combination as linear regression (z = w₁x₁ + w₂x₂ + ... + b), then passes it through the sigmoid to get a probability:

P(y=1|x) = σ(wᵀx + b) = 1 / (1 + e^(-(wᵀx+b)))

The model outputs a probability that the input belongs to class 1. If this probability exceeds 0.5, the model predicts class 1; otherwise class 0. The weights w are learned during training to maximise the likelihood of the observed labels — a process called maximum likelihood estimation.

import numpy as np

def logistic_predict_proba(X, weights, bias):
    z = X @ weights + bias  # linear score
    probability = 1 / (1 + np.exp(-z))  # sigmoid
    return probability

# Example: spam classification
# Features: word_count, exclamation_marks, capital_ratio
weights = np.array([0.05, 0.3, 2.0])
bias = -3.0

new_email = np.array([50, 5, 0.6])  # 50 words, 5 '!', 60% capitals
prob_spam = logistic_predict_proba(new_email, weights, bias)
print(f'P(spam): {prob_spam:.3f}')
print('Predicted label:', 1 if prob_spam >= 0.5 else 0)

Log Loss: The Right Cost Function

Logistic regression does not use MSE as its cost function. Instead, it uses Binary Cross-Entropy (log loss):

Loss = -(y × log(ŷ) + (1-y) × log(1-ŷ))

This penalises confident wrong predictions extremely harshly. If the true label is 1 and the model predicts probability 0.001, the log loss is -log(0.001) ≈ 6.9 — enormous. If the model predicts 0.99, the loss is -log(0.99) ≈ 0.01 — tiny. This asymmetric penalisation is exactly right for probability calibration.

import numpy as np

def log_loss_single(y_true, y_pred_proba, epsilon=1e-9):
    # Clip to avoid log(0)
    p = np.clip(y_pred_proba, epsilon, 1 - epsilon)
    return -(y_true * np.log(p) + (1 - y_true) * np.log(1 - p))

# True label = 1 (spam)
print('P(spam)=0.95, loss:', log_loss_single(1, 0.95).round(3))  # small
print('P(spam)=0.5,  loss:', log_loss_single(1, 0.50).round(3))  # moderate
print('P(spam)=0.05, loss:', log_loss_single(1, 0.05).round(3))  # large!
print('P(spam)=0.001,loss:', log_loss_single(1, 0.001).round(3)) # very large!

Training Logistic Regression with scikit-learn

Scikit-learn's LogisticRegression uses a gradient-based optimiser (LBFGS by default) to minimise log loss and find the optimal weights. The API is identical to LinearRegression: instantiate, fit, predict.

Important parameters include C (inverse of regularisation strength — smaller C means stronger regularisation) and max_iter (maximum number of optimisation steps). If you see a convergence warning, increase max_iter or scale your features first.

from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Scale features (important for logistic regression)
scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)

model = LogisticRegression(C=1.0, max_iter=1000)
model.fit(X_train_s, y_train)
print('Test accuracy:', model.score(X_test_s, y_test).round(3))

Probability Outputs and Confidence

Logistic regression is one of the few classifiers that produces well-calibrated probability estimates. A probability of 0.85 for class 1 means roughly 85% of predictions with that confidence level should actually be class 1. This calibration is valuable for risk management applications (credit scoring, medical diagnosis) where the probability itself matters, not just the binary label.

Use predict_proba() to retrieve the full probability vector. The two columns represent P(class 0) and P(class 1). They always sum to 1.0.

import numpy as np

# Get class probabilities
probas = model.predict_proba(X_test_s[:8])
labels = model.predict(X_test_s[:8])

print('Sample | P(benign) | P(malignant) | Predicted')
for i, (proba, label) in enumerate(zip(probas, labels)):
    print(f'  {i+1}    |   {proba[0]:.3f}   |    {proba[1]:.3f}     | {label}')

print('\nClass names:', model.classes_)  # [0, 1] or ['benign', 'malignant']

Interpreting Logistic Regression Coefficients

Logistic regression coefficients are not as directly interpretable as linear regression coefficients, but they still carry meaningful information. The coefficient w for a feature represents the change in the log-odds of the positive class for a one-unit increase in that feature:

log(P(y=1)/P(y=0)) = wᵀx + b

The odds ratio for a feature is exp(w). If exp(w) = 2, that feature doubles the odds of the positive class. Large positive coefficients indicate features that strongly predict class 1; large negative coefficients strongly predict class 0.

import numpy as np
import pandas as pd
from sklearn.datasets import load_breast_cancer

data = load_breast_cancer()
feature_names = data.feature_names

# Coefficient table with odds ratios
coef_df = pd.DataFrame({
    'Feature': feature_names,
    'Coefficient': model.coef_[0],
    'Odds_Ratio': np.exp(model.coef_[0])
}).sort_values('Coefficient', key=abs, ascending=False)

print(coef_df.head(5).to_string(index=False))
# Features with largest |coef| drive predictions most strongly

Regularisation in Logistic Regression

Logistic regression in scikit-learn applies L2 regularisation by default, controlled by the parameter C. Unlike most regularisation parameters, C is the inverse of regularisation strength: a smaller C means stronger regularisation (more shrinkage of weights), and a larger C means weaker regularisation.

Use penalty='l1' and solver 'liblinear' for L1 regularisation, which performs automatic feature selection by zeroing out irrelevant feature weights. L1 logistic regression is particularly useful when you have many features and suspect most are irrelevant.

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

# Compare regularisation strengths
for C in [0.01, 0.1, 1.0, 10.0, 100.0]:
    model_c = LogisticRegression(C=C, max_iter=1000)
    scores = cross_val_score(model_c, X_train_s, y_train, cv=5)
    print(f'C={C:6}: CV Accuracy = {scores.mean():.3f} (+/- {scores.std():.3f})')

Classification Report

A single accuracy number hides important information. Scikit-learn's classification_report() prints precision, recall, and F1-score for each class, plus macro and weighted averages. This is the standard way to report classification performance in research and industry.

Reading the report: the 'support' column shows how many examples of each class are in the test set. If supports are very unequal (class imbalance), look at per-class recall rather than overall accuracy to understand where the model fails.

from sklearn.metrics import classification_report

y_pred = model.predict(X_test_s)
print(classification_report(y_test, y_pred, target_names=['benign', 'malignant']))
# Shows for each class:
#   precision: of all predicted positive, how many were actually positive
#   recall: of all actual positive, how many did we catch
#   f1-score: harmonic mean of precision and recall
#   support: number of true instances of each class in test set

Logistic Regression vs Linear Regression Compared

A clear comparison of when to use each:

Target type: Linear regression → continuous number; Logistic regression → probability / binary label.
Output range: Linear → (-∞, +∞); Logistic → (0, 1).
Loss function: Linear → MSE; Logistic → Binary Cross-Entropy (log loss).
Evaluation: Linear → RMSE, R²; Logistic → Accuracy, F1, AUC-ROC.
Shared property: Both are linear models — the decision boundary is a hyperplane, and both benefit from feature scaling and regularisation.

Multi-Class Logistic Regression

For problems with more than two classes, logistic regression generalises to Softmax regression (also called multinomial logistic regression). Instead of one sigmoid, it computes one linear score per class and passes them all through the softmax function, producing a probability distribution over all classes.

In scikit-learn, set multi_class='multinomial' and use solver 'lbfgs' or 'saga'. The model outputs a probability for each class, and the predicted class is the one with the highest probability.

from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
X_train_s = StandardScaler().fit_transform(X_train)
X_test_s = StandardScaler().fit_transform(X_test)

# Multinomial for 3 classes
model = LogisticRegression(multi_class='multinomial', solver='lbfgs', max_iter=1000)
model.fit(X_train_s, y_train)
print('Test accuracy:', model.score(X_test_s, y_test).round(3))
print('Proba shape:', model.predict_proba(X_test_s[:1]).shape)  # (1, 3)

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: the sigmoid function squashes any linear score to a valid probability between 0 and 1, logistic regression uses binary cross-entropy loss which harshly penalises confident wrong predictions, and coefficients represent changes in log-odds with odds ratio exp(w) giving a more intuitive scale-invariant interpretation. Next up we build a confusion matrix from predictions and ground truth, providing a detailed breakdown of where the classifier succeeds and fails.

常见问题解答

「逻辑回归与 Sigmoid 函数」课时是免费的吗？

是的 — 「逻辑回归与 Sigmoid 函数」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Machine Learning Academy 课程的其余内容，请升级到 CoddyKit PRO。 Machine Learning Academy 课程共包含 4 节课。

「逻辑回归与 Sigmoid 函数」这节课中我会学到什么？

您将应用 Sigmoid 生成概率，将输出解读为类别置信度，并使用 scikit-learn 训练逻辑模型你通过在浏览器中直接运行的动手代码来练习 Machine Learning Academy，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 Machine Learning Academy 需要有经验吗？

无需任何先前经验。CoddyKit 上的 Machine Learning Academy 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。这是第 2 节课，共 4 节。

「逻辑回归与 Sigmoid 函数」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 Machine Learning Academy 课中编写并运行代码吗？

能。每节 Machine Learning Academy 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

← 返回 Machine Learning Academy