0Pricing
Machine Learning Academy · レッスン

ロジスティック回帰とシグモイド関数

シグモイド関数で確率を生成し、出力をクラスの確信度として解釈して、scikit-learnでロジスティックモデルを訓練します。

「ロジスティック回帰とシグモイド関数」はCoddyKit上の無料Machine Learning Academyレッスンです。 これはレッスン2/4です。 下記で完全なレッスンを無料で読むことができます。その後、ブラウザ内の組み込みコードエディタと24時間対応のAIチューターでハンズオン演習できます。 これはMachine Learning Academy学習パスの一部であり、ウェブとCoddyKitアプリ全体で進捗が同期されます。 Machine Learning Academyコースには全4レッスンが含まれています。

このレッスンの一部はまだ翻訳されておらず、英語で表示されています。

Logistic Regression: A Probabilistic Classifier

Logistic regression is the go-to algorithm for binary classification. Despite its name, it is a classifier, not a regressor. It extends the linear model by squashing the output through a special function called the sigmoid, ensuring predictions are always valid probabilities between 0 and 1.

Logistic regression is one of the most widely deployed ML models in production. It is fast to train, interpretable, and provides calibrated probability estimates. It is often the first classifier you should try on a new binary classification problem before reaching for more complex methods.

The Sigmoid Function

The sigmoid function (also called the logistic function) maps any real number to the range (0, 1):

σ(z) = 1 / (1 + e^(-z))

Key properties: σ(0) = 0.5, σ(+∞) → 1, σ(-∞) → 0. The S-shaped curve smoothly transitions from 0 to 1. When z is a large positive number (strong signal for class 1), the output approaches 1. When z is a large negative number (strong signal for class 0), the output approaches 0.

import numpy as np
import matplotlib.pyplot as plt

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

z = np.linspace(-10, 10, 200)
y = sigmoid(z)

plt.plot(z, y, 'b-', linewidth=2)
plt.axhline(y=0.5, color='red', linestyle='--', alpha=0.5, label='threshold=0.5')
plt.axvline(x=0, color='gray', linestyle='--', alpha=0.5)
plt.xlabel('z (linear score)')
plt.ylabel('sigmoid(z) = probability')
plt.title('The Sigmoid Function')
plt.legend()
plt.show()

print('sigmoid(0):', sigmoid(0))   # 0.5
print('sigmoid(5):', sigmoid(5))   # ~0.993
print('sigmoid(-5):', sigmoid(-5)) # ~0.007

How Logistic Regression Computes Probability

Logistic regression first computes the same linear combination as linear regression (z = w₁x₁ + w₂x₂ + ... + b), then passes it through the sigmoid to get a probability:

P(y=1|x) = σ(wᵀx + b) = 1 / (1 + e^(-(wᵀx+b)))

The model outputs a probability that the input belongs to class 1. If this probability exceeds 0.5, the model predicts class 1; otherwise class 0. The weights w are learned during training to maximise the likelihood of the observed labels — a process called maximum likelihood estimation.

import numpy as np

def logistic_predict_proba(X, weights, bias):
    z = X @ weights + bias  # linear score
    probability = 1 / (1 + np.exp(-z))  # sigmoid
    return probability

# Example: spam classification
# Features: word_count, exclamation_marks, capital_ratio
weights = np.array([0.05, 0.3, 2.0])
bias = -3.0

new_email = np.array([50, 5, 0.6])  # 50 words, 5 '!', 60% capitals
prob_spam = logistic_predict_proba(new_email, weights, bias)
print(f'P(spam): {prob_spam:.3f}')
print('Predicted label:', 1 if prob_spam >= 0.5 else 0)

Log Loss: The Right Cost Function

Logistic regression does not use MSE as its cost function. Instead, it uses Binary Cross-Entropy (log loss):

Loss = -(y × log(ŷ) + (1-y) × log(1-ŷ))

This penalises confident wrong predictions extremely harshly. If the true label is 1 and the model predicts probability 0.001, the log loss is -log(0.001) ≈ 6.9 — enormous. If the model predicts 0.99, the loss is -log(0.99) ≈ 0.01 — tiny. This asymmetric penalisation is exactly right for probability calibration.

import numpy as np

def log_loss_single(y_true, y_pred_proba, epsilon=1e-9):
    # Clip to avoid log(0)
    p = np.clip(y_pred_proba, epsilon, 1 - epsilon)
    return -(y_true * np.log(p) + (1 - y_true) * np.log(1 - p))

# True label = 1 (spam)
print('P(spam)=0.95, loss:', log_loss_single(1, 0.95).round(3))  # small
print('P(spam)=0.5,  loss:', log_loss_single(1, 0.50).round(3))  # moderate
print('P(spam)=0.05, loss:', log_loss_single(1, 0.05).round(3))  # large!
print('P(spam)=0.001,loss:', log_loss_single(1, 0.001).round(3)) # very large!

Training Logistic Regression with scikit-learn

Scikit-learn's LogisticRegression uses a gradient-based optimiser (LBFGS by default) to minimise log loss and find the optimal weights. The API is identical to LinearRegression: instantiate, fit, predict.

Important parameters include C (inverse of regularisation strength — smaller C means stronger regularisation) and max_iter (maximum number of optimisation steps). If you see a convergence warning, increase max_iter or scale your features first.

from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Scale features (important for logistic regression)
scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)

model = LogisticRegression(C=1.0, max_iter=1000)
model.fit(X_train_s, y_train)
print('Test accuracy:', model.score(X_test_s, y_test).round(3))

Probability Outputs and Confidence

Logistic regression is one of the few classifiers that produces well-calibrated probability estimates. A probability of 0.85 for class 1 means roughly 85% of predictions with that confidence level should actually be class 1. This calibration is valuable for risk management applications (credit scoring, medical diagnosis) where the probability itself matters, not just the binary label.

Use predict_proba() to retrieve the full probability vector. The two columns represent P(class 0) and P(class 1). They always sum to 1.0.

import numpy as np

# Get class probabilities
probas = model.predict_proba(X_test_s[:8])
labels = model.predict(X_test_s[:8])

print('Sample | P(benign) | P(malignant) | Predicted')
for i, (proba, label) in enumerate(zip(probas, labels)):
    print(f'  {i+1}    |   {proba[0]:.3f}   |    {proba[1]:.3f}     | {label}')

print('\nClass names:', model.classes_)  # [0, 1] or ['benign', 'malignant']

Interpreting Logistic Regression Coefficients

Logistic regression coefficients are not as directly interpretable as linear regression coefficients, but they still carry meaningful information. The coefficient w for a feature represents the change in the log-odds of the positive class for a one-unit increase in that feature:

log(P(y=1)/P(y=0)) = wᵀx + b

The odds ratio for a feature is exp(w). If exp(w) = 2, that feature doubles the odds of the positive class. Large positive coefficients indicate features that strongly predict class 1; large negative coefficients strongly predict class 0.

import numpy as np
import pandas as pd
from sklearn.datasets import load_breast_cancer

data = load_breast_cancer()
feature_names = data.feature_names

# Coefficient table with odds ratios
coef_df = pd.DataFrame({
    'Feature': feature_names,
    'Coefficient': model.coef_[0],
    'Odds_Ratio': np.exp(model.coef_[0])
}).sort_values('Coefficient', key=abs, ascending=False)

print(coef_df.head(5).to_string(index=False))
# Features with largest |coef| drive predictions most strongly

Regularisation in Logistic Regression

Logistic regression in scikit-learn applies L2 regularisation by default, controlled by the parameter C. Unlike most regularisation parameters, C is the inverse of regularisation strength: a smaller C means stronger regularisation (more shrinkage of weights), and a larger C means weaker regularisation.

Use penalty='l1' and solver 'liblinear' for L1 regularisation, which performs automatic feature selection by zeroing out irrelevant feature weights. L1 logistic regression is particularly useful when you have many features and suspect most are irrelevant.

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

# Compare regularisation strengths
for C in [0.01, 0.1, 1.0, 10.0, 100.0]:
    model_c = LogisticRegression(C=C, max_iter=1000)
    scores = cross_val_score(model_c, X_train_s, y_train, cv=5)
    print(f'C={C:6}: CV Accuracy = {scores.mean():.3f} (+/- {scores.std():.3f})')

Classification Report

A single accuracy number hides important information. Scikit-learn's classification_report() prints precision, recall, and F1-score for each class, plus macro and weighted averages. This is the standard way to report classification performance in research and industry.

Reading the report: the 'support' column shows how many examples of each class are in the test set. If supports are very unequal (class imbalance), look at per-class recall rather than overall accuracy to understand where the model fails.

from sklearn.metrics import classification_report

y_pred = model.predict(X_test_s)
print(classification_report(y_test, y_pred, target_names=['benign', 'malignant']))
# Shows for each class:
#   precision: of all predicted positive, how many were actually positive
#   recall: of all actual positive, how many did we catch
#   f1-score: harmonic mean of precision and recall
#   support: number of true instances of each class in test set

Logistic Regression vs Linear Regression Compared

A clear comparison of when to use each:

  • Target type: Linear regression → continuous number; Logistic regression → probability / binary label.
  • Output range: Linear → (-∞, +∞); Logistic → (0, 1).
  • Loss function: Linear → MSE; Logistic → Binary Cross-Entropy (log loss).
  • Evaluation: Linear → RMSE, R²; Logistic → Accuracy, F1, AUC-ROC.
  • Shared property: Both are linear models — the decision boundary is a hyperplane, and both benefit from feature scaling and regularisation.

Multi-Class Logistic Regression

For problems with more than two classes, logistic regression generalises to Softmax regression (also called multinomial logistic regression). Instead of one sigmoid, it computes one linear score per class and passes them all through the softmax function, producing a probability distribution over all classes.

In scikit-learn, set multi_class='multinomial' and use solver 'lbfgs' or 'saga'. The model outputs a probability for each class, and the predicted class is the one with the highest probability.

from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
X_train_s = StandardScaler().fit_transform(X_train)
X_test_s = StandardScaler().fit_transform(X_test)

# Multinomial for 3 classes
model = LogisticRegression(multi_class='multinomial', solver='lbfgs', max_iter=1000)
model.fit(X_train_s, y_train)
print('Test accuracy:', model.score(X_test_s, y_test).round(3))
print('Proba shape:', model.predict_proba(X_test_s[:1]).shape)  # (1, 3)

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: the sigmoid function squashes any linear score to a valid probability between 0 and 1, logistic regression uses binary cross-entropy loss which harshly penalises confident wrong predictions, and coefficients represent changes in log-odds with odds ratio exp(w) giving a more intuitive scale-invariant interpretation. Next up we build a confusion matrix from predictions and ground truth, providing a detailed breakdown of where the classifier succeeds and fails.

よくある質問

「ロジスティック回帰とシグモイド関数」レッスンは無料ですか？

はい。「ロジスティック回帰とシグモイド関数」の完全なテキストはこのウェブで無料で読めます。インタラクティブに演習し（組み込みコードエディタと24時間対応のAIチューター）、Machine Learning Academyコースの残りをアンロックするには、CoddyKit PROにアップグレードしてください。 Machine Learning Academyコースには全4レッスンが含まれています。

「ロジスティック回帰とシグモイド関数」で何を学びますか？

シグモイド関数で確率を生成し、出力をクラスの確信度として解釈して、scikit-learnでロジスティックモデルを訓練します。 ブラウザで直接実行するハンズオンコードでMachine Learning Academyを演習し、24時間対応のAIチューターがレッスンを進める中での質問に答えます。

Machine Learning Academyを始めるのに経験は必要ですか？

事前経験は必要ありません。CoddyKitのMachine Learning Academyは初級者から上級者向けに構成されているため、ここから始めるか最初から始めて、自分のペースで進むことができます。 これはレッスン2/4です。

「ロジスティック回帰とシグモイド関数」レッスンにはどのくらい時間がかかりますか？

ほとんどのCoddyKitレッスンは約5～10分かかります。各レッスンはコンパクトでインタラクティブなので、着実に進歩し、ウェブとアプリ全体で正確に前回の場所から再開できます。

このMachine Learning Academyレッスンでコードを書いて実行できますか？

はい。すべてのMachine Learning Academyレッスンに組み込みコードエディタが含まれているため、ブラウザでリアルコードを書いて実行し、即座のAIフィードバックを取得できます。ローカル設定は不要です。

このコースのすべてのレッスン

  1. 回帰から分類へ：しきい値による判定
  2. ロジスティック回帰とシグモイド関数
  3. 混同行列を理解する
  4. 実践で学ぶ適合率、再現率、F1スコア
← Machine Learning Academyに戻る