0Pricing
Machine Learning Academy · Pelajaran

Regresi Logistik dan Fungsi Sigmoid

Peserta didik akan menerapkan sigmoid untuk menghasilkan probabilitas, menafsirkan output sebagai keyakinan kelas, dan melatih model logistik dengan scikit-learn.

Regresi Logistik dan Fungsi Sigmoid adalah pelajaran Machine Learning Academy gratis di CoddyKit. Ini adalah pelajaran 2 dari 4. Kamu bisa membaca pelajaran lengkapnya di bawah secara gratis — lalu praktikkan langsung di browser dengan editor kode bawaan dan tutor AI 24/7. Ini adalah bagian dari jalur belajar Machine Learning Academy, dan progresmu tersinkronisasi di web dan aplikasi CoddyKit. Kursus Machine Learning Academy mencakup 4 pelajaran total.

Bagian dari pelajaran ini belum diterjemahkan dan ditampilkan dalam bahasa Inggris.

Logistic Regression: A Probabilistic Classifier

Logistic regression is the go-to algorithm for binary classification. Despite its name, it is a classifier, not a regressor. It extends the linear model by squashing the output through a special function called the sigmoid, ensuring predictions are always valid probabilities between 0 and 1.

Logistic regression is one of the most widely deployed ML models in production. It is fast to train, interpretable, and provides calibrated probability estimates. It is often the first classifier you should try on a new binary classification problem before reaching for more complex methods.

The Sigmoid Function

The sigmoid function (also called the logistic function) maps any real number to the range (0, 1):

σ(z) = 1 / (1 + e^(-z))

Key properties: σ(0) = 0.5, σ(+∞) → 1, σ(-∞) → 0. The S-shaped curve smoothly transitions from 0 to 1. When z is a large positive number (strong signal for class 1), the output approaches 1. When z is a large negative number (strong signal for class 0), the output approaches 0.

import numpy as np
import matplotlib.pyplot as plt

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

z = np.linspace(-10, 10, 200)
y = sigmoid(z)

plt.plot(z, y, 'b-', linewidth=2)
plt.axhline(y=0.5, color='red', linestyle='--', alpha=0.5, label='threshold=0.5')
plt.axvline(x=0, color='gray', linestyle='--', alpha=0.5)
plt.xlabel('z (linear score)')
plt.ylabel('sigmoid(z) = probability')
plt.title('The Sigmoid Function')
plt.legend()
plt.show()

print('sigmoid(0):', sigmoid(0))   # 0.5
print('sigmoid(5):', sigmoid(5))   # ~0.993
print('sigmoid(-5):', sigmoid(-5)) # ~0.007

How Logistic Regression Computes Probability

Logistic regression first computes the same linear combination as linear regression (z = w₁x₁ + w₂x₂ + ... + b), then passes it through the sigmoid to get a probability:

P(y=1|x) = σ(wᵀx + b) = 1 / (1 + e^(-(wᵀx+b)))

The model outputs a probability that the input belongs to class 1. If this probability exceeds 0.5, the model predicts class 1; otherwise class 0. The weights w are learned during training to maximise the likelihood of the observed labels — a process called maximum likelihood estimation.

import numpy as np

def logistic_predict_proba(X, weights, bias):
    z = X @ weights + bias  # linear score
    probability = 1 / (1 + np.exp(-z))  # sigmoid
    return probability

# Example: spam classification
# Features: word_count, exclamation_marks, capital_ratio
weights = np.array([0.05, 0.3, 2.0])
bias = -3.0

new_email = np.array([50, 5, 0.6])  # 50 words, 5 '!', 60% capitals
prob_spam = logistic_predict_proba(new_email, weights, bias)
print(f'P(spam): {prob_spam:.3f}')
print('Predicted label:', 1 if prob_spam >= 0.5 else 0)

Log Loss: The Right Cost Function

Logistic regression does not use MSE as its cost function. Instead, it uses Binary Cross-Entropy (log loss):

Loss = -(y × log(ŷ) + (1-y) × log(1-ŷ))

This penalises confident wrong predictions extremely harshly. If the true label is 1 and the model predicts probability 0.001, the log loss is -log(0.001) ≈ 6.9 — enormous. If the model predicts 0.99, the loss is -log(0.99) ≈ 0.01 — tiny. This asymmetric penalisation is exactly right for probability calibration.

import numpy as np

def log_loss_single(y_true, y_pred_proba, epsilon=1e-9):
    # Clip to avoid log(0)
    p = np.clip(y_pred_proba, epsilon, 1 - epsilon)
    return -(y_true * np.log(p) + (1 - y_true) * np.log(1 - p))

# True label = 1 (spam)
print('P(spam)=0.95, loss:', log_loss_single(1, 0.95).round(3))  # small
print('P(spam)=0.5,  loss:', log_loss_single(1, 0.50).round(3))  # moderate
print('P(spam)=0.05, loss:', log_loss_single(1, 0.05).round(3))  # large!
print('P(spam)=0.001,loss:', log_loss_single(1, 0.001).round(3)) # very large!

Training Logistic Regression with scikit-learn

Scikit-learn's LogisticRegression uses a gradient-based optimiser (LBFGS by default) to minimise log loss and find the optimal weights. The API is identical to LinearRegression: instantiate, fit, predict.

Important parameters include C (inverse of regularisation strength — smaller C means stronger regularisation) and max_iter (maximum number of optimisation steps). If you see a convergence warning, increase max_iter or scale your features first.

from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Scale features (important for logistic regression)
scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)

model = LogisticRegression(C=1.0, max_iter=1000)
model.fit(X_train_s, y_train)
print('Test accuracy:', model.score(X_test_s, y_test).round(3))

Probability Outputs and Confidence

Logistic regression is one of the few classifiers that produces well-calibrated probability estimates. A probability of 0.85 for class 1 means roughly 85% of predictions with that confidence level should actually be class 1. This calibration is valuable for risk management applications (credit scoring, medical diagnosis) where the probability itself matters, not just the binary label.

Use predict_proba() to retrieve the full probability vector. The two columns represent P(class 0) and P(class 1). They always sum to 1.0.

import numpy as np

# Get class probabilities
probas = model.predict_proba(X_test_s[:8])
labels = model.predict(X_test_s[:8])

print('Sample | P(benign) | P(malignant) | Predicted')
for i, (proba, label) in enumerate(zip(probas, labels)):
    print(f'  {i+1}    |   {proba[0]:.3f}   |    {proba[1]:.3f}     | {label}')

print('\nClass names:', model.classes_)  # [0, 1] or ['benign', 'malignant']

Interpreting Logistic Regression Coefficients

Logistic regression coefficients are not as directly interpretable as linear regression coefficients, but they still carry meaningful information. The coefficient w for a feature represents the change in the log-odds of the positive class for a one-unit increase in that feature:

log(P(y=1)/P(y=0)) = wᵀx + b

The odds ratio for a feature is exp(w). If exp(w) = 2, that feature doubles the odds of the positive class. Large positive coefficients indicate features that strongly predict class 1; large negative coefficients strongly predict class 0.

import numpy as np
import pandas as pd
from sklearn.datasets import load_breast_cancer

data = load_breast_cancer()
feature_names = data.feature_names

# Coefficient table with odds ratios
coef_df = pd.DataFrame({
    'Feature': feature_names,
    'Coefficient': model.coef_[0],
    'Odds_Ratio': np.exp(model.coef_[0])
}).sort_values('Coefficient', key=abs, ascending=False)

print(coef_df.head(5).to_string(index=False))
# Features with largest |coef| drive predictions most strongly

Regularisation in Logistic Regression

Logistic regression in scikit-learn applies L2 regularisation by default, controlled by the parameter C. Unlike most regularisation parameters, C is the inverse of regularisation strength: a smaller C means stronger regularisation (more shrinkage of weights), and a larger C means weaker regularisation.

Use penalty='l1' and solver 'liblinear' for L1 regularisation, which performs automatic feature selection by zeroing out irrelevant feature weights. L1 logistic regression is particularly useful when you have many features and suspect most are irrelevant.

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

# Compare regularisation strengths
for C in [0.01, 0.1, 1.0, 10.0, 100.0]:
    model_c = LogisticRegression(C=C, max_iter=1000)
    scores = cross_val_score(model_c, X_train_s, y_train, cv=5)
    print(f'C={C:6}: CV Accuracy = {scores.mean():.3f} (+/- {scores.std():.3f})')

Classification Report

A single accuracy number hides important information. Scikit-learn's classification_report() prints precision, recall, and F1-score for each class, plus macro and weighted averages. This is the standard way to report classification performance in research and industry.

Reading the report: the 'support' column shows how many examples of each class are in the test set. If supports are very unequal (class imbalance), look at per-class recall rather than overall accuracy to understand where the model fails.

from sklearn.metrics import classification_report

y_pred = model.predict(X_test_s)
print(classification_report(y_test, y_pred, target_names=['benign', 'malignant']))
# Shows for each class:
#   precision: of all predicted positive, how many were actually positive
#   recall: of all actual positive, how many did we catch
#   f1-score: harmonic mean of precision and recall
#   support: number of true instances of each class in test set

Logistic Regression vs Linear Regression Compared

A clear comparison of when to use each:

  • Target type: Linear regression → continuous number; Logistic regression → probability / binary label.
  • Output range: Linear → (-∞, +∞); Logistic → (0, 1).
  • Loss function: Linear → MSE; Logistic → Binary Cross-Entropy (log loss).
  • Evaluation: Linear → RMSE, R²; Logistic → Accuracy, F1, AUC-ROC.
  • Shared property: Both are linear models — the decision boundary is a hyperplane, and both benefit from feature scaling and regularisation.

Multi-Class Logistic Regression

For problems with more than two classes, logistic regression generalises to Softmax regression (also called multinomial logistic regression). Instead of one sigmoid, it computes one linear score per class and passes them all through the softmax function, producing a probability distribution over all classes.

In scikit-learn, set multi_class='multinomial' and use solver 'lbfgs' or 'saga'. The model outputs a probability for each class, and the predicted class is the one with the highest probability.

from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
X_train_s = StandardScaler().fit_transform(X_train)
X_test_s = StandardScaler().fit_transform(X_test)

# Multinomial for 3 classes
model = LogisticRegression(multi_class='multinomial', solver='lbfgs', max_iter=1000)
model.fit(X_train_s, y_train)
print('Test accuracy:', model.score(X_test_s, y_test).round(3))
print('Proba shape:', model.predict_proba(X_test_s[:1]).shape)  # (1, 3)

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: the sigmoid function squashes any linear score to a valid probability between 0 and 1, logistic regression uses binary cross-entropy loss which harshly penalises confident wrong predictions, and coefficients represent changes in log-odds with odds ratio exp(w) giving a more intuitive scale-invariant interpretation. Next up we build a confusion matrix from predictions and ground truth, providing a detailed breakdown of where the classifier succeeds and fails.

Pertanyaan yang Sering Diajukan

Apakah pelajaran “Regresi Logistik dan Fungsi Sigmoid” gratis?

Ya — teks lengkap “Regresi Logistik dan Fungsi Sigmoid” gratis dibaca di sini di web. Untuk praktiknya secara interaktif (editor kode bawaan dan tutor AI 24/7) dan buka sisa kursus Machine Learning Academy, upgrade ke CoddyKit PRO. Kursus Machine Learning Academy mencakup 4 pelajaran total.

Apa yang akan aku pelajari di “Regresi Logistik dan Fungsi Sigmoid”?

Peserta didik akan menerapkan sigmoid untuk menghasilkan probabilitas, menafsirkan output sebagai keyakinan kelas, dan melatih model logistik dengan scikit-learn. Kamu berlatih Machine Learning Academy dengan kode praktik yang langsung kamu jalankan di browser, dan tutor AI 24/7 menjawab pertanyaanmu saat kamu mengerjakan pelajaran ini.

Apakah aku perlu pengalaman untuk memulai Machine Learning Academy?

Tidak diperlukan pengalaman sebelumnya. Machine Learning Academy di CoddyKit dirancang untuk pemula hingga pelajar tingkat lanjut, jadi kamu bisa memulai di sini atau dari awal dan belajar sesuai kecepatan kamu sendiri. Ini adalah pelajaran 2 dari 4.

Berapa lama pelajaran “Regresi Logistik dan Fungsi Sigmoid” memakan waktu?

Sebagian besar pelajaran CoddyKit memakan waktu sekitar 5–10 menit. Setiap pelajaran ringkas dan interaktif, jadi kamu membuat kemajuan stabil dan melanjutkan dari tempat kamu tinggalkan di web dan aplikasi.

Bisakah aku menulis dan menjalankan kode dalam pelajaran Machine Learning Academy ini?

Ya. Setiap pelajaran Machine Learning Academy menyertakan editor kode bawaan, jadi kamu menulis dan menjalankan kode nyata langsung di browser dan mendapatkan umpan balik AI instan — tidak diperlukan penyiapan lokal.

Semua pelajaran dalam kursus ini

  1. Dari Regresi ke Klasifikasi: Keputusan Ambang
  2. Regresi Logistik dan Fungsi Sigmoid
  3. Penjelasan Matriks Kebingungan
  4. Presisi, Recall, dan F1-Score dalam Praktik
← Kembali ke Machine Learning Academy