Logistic Regression and the Sigmoid Function
Learners will apply the sigmoid to produce probabilities, interpret the output as class confidence, and train a logistic model with scikit-learn.
Logistic Regression: A Probabilistic Classifier
Logistic regression is the go-to algorithm for binary classification. Despite its name, it is a classifier, not a regressor. It extends the linear model by squashing the output through a special function called the sigmoid, ensuring predictions are always valid probabilities between 0 and 1.
Logistic regression is one of the most widely deployed ML models in production. It is fast to train, interpretable, and provides calibrated probability estimates. It is often the first classifier you should try on a new binary classification problem before reaching for more complex methods.
The Sigmoid Function
The sigmoid function (also called the logistic function) maps any real number to the range (0, 1):
σ(z) = 1 / (1 + e^(-z))
Key properties: σ(0) = 0.5, σ(+∞) → 1, σ(-∞) → 0. The S-shaped curve smoothly transitions from 0 to 1. When z is a large positive number (strong signal for class 1), the output approaches 1. When z is a large negative number (strong signal for class 0), the output approaches 0.
import numpy as np
import matplotlib.pyplot as plt
def sigmoid(z):
return 1 / (1 + np.exp(-z))
z = np.linspace(-10, 10, 200)
y = sigmoid(z)
plt.plot(z, y, 'b-', linewidth=2)
plt.axhline(y=0.5, color='red', linestyle='--', alpha=0.5, label='threshold=0.5')
plt.axvline(x=0, color='gray', linestyle='--', alpha=0.5)
plt.xlabel('z (linear score)')
plt.ylabel('sigmoid(z) = probability')
plt.title('The Sigmoid Function')
plt.legend()
plt.show()
print('sigmoid(0):', sigmoid(0)) # 0.5
print('sigmoid(5):', sigmoid(5)) # ~0.993
print('sigmoid(-5):', sigmoid(-5)) # ~0.007All lessons in this course
- From Regression to Classification: Threshold Decisions
- Logistic Regression and the Sigmoid Function
- The Confusion Matrix Explained
- Precision, Recall, and F1-Score in Practice