Machine Learning Academy · Урок

Обучение линейной регрессии с scikit-learn

Вы используете sklearn.linear_model.LinearRegression для подгонки модели по обучающим данным, построения прогнозов и изучения выученных коэффициентов

«Обучение линейной регрессии с scikit-learn» — бесплатный урок Machine Learning Academy на CoddyKit. Это урок 3 из 4. Ты можешь прочитать весь урок бесплатно ниже — а потом практиковать его прямо в браузере с встроенным редактором кода и ИИ-репетитором 24/7. Это часть пути обучения Machine Learning Academy, и твой прогресс синхронизируется между веб-версией и приложением CoddyKit. Курс Machine Learning Academy содержит 4 уроков всего.

Части этого урока еще не переведены и отображаются на английском.

scikit-learn's LinearRegression API

Scikit-learn's LinearRegression class provides a clean, consistent API for fitting ordinary least-squares linear regression. Like all scikit-learn estimators, it follows the same pattern: instantiate, fit, predict.

Under the hood, it uses the Normal Equation (or a numerically stable SVD decomposition for large feature matrices) to find the exact optimal weights in a single computation — no iterative training loop is needed. This makes it extremely fast even on large datasets with many features.

from sklearn.linear_model import LinearRegression
import numpy as np

# Instantiate with optional parameters
model = LinearRegression(
    fit_intercept=True,   # default: add bias term
    copy_X=True,          # don't modify input arrays
    n_jobs=-1             # use all CPU cores
)

print(model)  # shows default parameters

Preparing the Data

Before training, your data must be in the correct shape. scikit-learn expects:

X — a 2D array of shape (n_samples, n_features)
y — a 1D array of shape (n_samples,)

The most common shape error is passing a 1D array for X when you have one feature. The fix is to reshape with .reshape(-1, 1). Always split into training and test sets before any fitting — evaluation on training data gives misleadingly perfect scores.

import numpy as np
from sklearn.model_selection import train_test_split

# Simulate housing data
np.random.seed(42)
sqft = np.random.uniform(500, 3000, 200)
price = 150 * sqft + 50000 + np.random.normal(0, 25000, 200)

# Reshape X to 2D (n_samples, n_features)
X = sqft.reshape(-1, 1)  # shape (200, 1)
y = price                  # shape (200,)

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)
print('Train size:', X_train.shape, '| Test size:', X_test.shape)

Fitting the Model

Calling model.fit(X_train, y_train) computes the optimal weights using the Normal Equation and stores them inside the model object. After fitting, two attributes become available:

model.coef_ — an array of weights, one per feature
model.intercept_ — the bias term (scalar)

The trailing underscore in attribute names is a scikit-learn convention indicating that the attribute was set during fitting — it does not exist before fit() is called.

from sklearn.linear_model import LinearRegression
import numpy as np
from sklearn.model_selection import train_test_split

np.random.seed(42)
X = np.random.uniform(500, 3000, 200).reshape(-1, 1)
y = 150 * X.ravel() + 50000 + np.random.normal(0, 25000, 200)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

model = LinearRegression()
model.fit(X_train, y_train)

print(f'Slope (coef_):     {model.coef_[0]:.2f}')   # should be ~150
print(f'Intercept:         {model.intercept_:.2f}')  # should be ~50000

Making Predictions

After fitting, call model.predict(X) to generate predictions. This computes the dot product of the feature matrix with the learned weights, plus the intercept. You can predict on training data, test data, or entirely new data.

Remember: evaluating on training data is not a valid performance estimate. Always evaluate on the held-out test set to get an honest estimate of how well the model will perform on unseen data.

# Predict on test set
y_pred = model.predict(X_test)

print('First 5 predictions vs actual:')
for actual, pred in zip(y_test[:5], y_pred[:5]):
    print(f'  Actual: ${actual:,.0f}  |  Predicted: ${pred:,.0f}  |  Error: ${actual-pred:,.0f}')

# Predict a single new house
new_house = np.array([[1750]])  # must be 2D
print(f'\nPrediction for 1750 sqft: ${model.predict(new_house)[0]:,.0f}')

Evaluating on the Test Set

Use scikit-learn's metrics functions to quantify test-set performance. The model.score(X_test, y_test) shortcut returns R² directly. For the full picture, also compute RMSE and MAE.

Comparing these metrics across different models (e.g., linear regression vs random forest) allows you to objectively select the best approach for your dataset. Always include a naive baseline (like always predicting the mean of y) to confirm your model is actually adding value.

from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
import numpy as np

y_pred = model.predict(X_test)

rmse = np.sqrt(mean_squared_error(y_test, y_pred))
mae  = mean_absolute_error(y_test, y_pred)
r2   = r2_score(y_test, y_pred)

print(f'Test RMSE: ${rmse:,.0f}')
print(f'Test MAE:  ${mae:,.0f}')
print(f'Test R2:   {r2:.3f}')

# Baseline: always predict the mean
baseline_rmse = np.sqrt(((y_test - y_train.mean()) ** 2).mean())
print(f'Baseline RMSE: ${baseline_rmse:,.0f}  (must beat this!)')

Visualising the Regression Line

After fitting, plot the learned regression line on top of the training data scatter plot. This visual check answers: does the line make sense? Is it in roughly the right place? Are there large groups of outliers that the model consistently misses?

For a single-feature model this is straightforward. For multi-feature models you will need to plot predictions vs actual values (a residual plot or a 'predicted vs actual' scatter plot) instead, since the decision boundary lives in high-dimensional space.

import matplotlib.pyplot as plt
import numpy as np

# Plot training data and the fitted line
plt.figure(figsize=(8, 5))
plt.scatter(X_train, y_train, alpha=0.4, label='Training data')
plt.scatter(X_test, y_test, alpha=0.4, color='orange', label='Test data')

# Draw the regression line across the full range
x_range = np.linspace(500, 3000, 100).reshape(-1, 1)
y_range = model.predict(x_range)
plt.plot(x_range, y_range, 'r-', linewidth=2, label='Regression line')

plt.xlabel('Square Footage')
plt.ylabel('Price ($)')
plt.legend()
plt.title('Linear Regression Fit')
plt.show()

Interpreting the Coefficients

The learned coef_ values carry direct business interpretation. For a house-price model with multiple features, each coefficient tells you how much the predicted price changes when that feature increases by one unit, holding all other features constant (the ceteris paribus interpretation).

However, this interpretation is only valid when features are on similar scales. If one feature is in metres and another is in kilometres, their coefficients are not directly comparable. Always scale features before interpreting coefficients in multi-feature models.

from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import StandardScaler
import numpy as np
import pandas as pd

# Multi-feature housing data
features = ['sqft', 'bedrooms', 'bathrooms', 'age']
np.random.seed(0)
X_data = pd.DataFrame({
    'sqft': np.random.randint(600, 3000, 100),
    'bedrooms': np.random.randint(1, 6, 100),
    'bathrooms': np.random.randint(1, 4, 100),
    'age': np.random.randint(1, 50, 100)
})
y_data = 150*X_data['sqft'] + 8000*X_data['bedrooms'] - 500*X_data['age'] + 30000

model = LinearRegression().fit(X_data, y_data)
for feat, coef in zip(features, model.coef_):
    print(f'{feat}: {coef:.1f}')

Residual Analysis

Plotting residuals against predicted values is a critical diagnostic step. For a well-fit model, residuals should be randomly scattered around zero with no discernible pattern. Systematic patterns indicate a problem:

A funnel shape (residuals spread more for larger predictions) indicates heteroscedasticity — the variance of errors is not constant.
A curved pattern indicates the true relationship is non-linear and a linear model is under-fitting.
Residuals that are systematically positive or negative for certain prediction ranges mean the model is biased in that region.

import matplotlib.pyplot as plt
import numpy as np

y_pred = model.predict(X_test)
residuals = y_test - y_pred

plt.figure(figsize=(8, 5))
plt.scatter(y_pred, residuals, alpha=0.5)
plt.axhline(y=0, color='red', linestyle='--', linewidth=1.5)
plt.xlabel('Predicted Values')
plt.ylabel('Residuals')
plt.title('Residual Plot (should be random scatter around 0)')
plt.show()

print(f'Mean residual: {residuals.mean():.2f}  (should be ~0)')
print(f'Std of residuals: {residuals.std():.2f}')

Regularised Linear Regression

Plain LinearRegression can overfit when you have many features relative to training examples, or when features are highly correlated (multicollinearity). Scikit-learn provides two regularised variants:

Ridge (L2 regularisation) — shrinks all coefficients towards zero proportionally, keeping all features but reducing their magnitude.
Lasso (L1 regularisation) — shrinks some coefficients all the way to zero, effectively performing feature selection.

The alpha parameter controls regularisation strength: higher alpha = more shrinkage = simpler model.

from sklearn.linear_model import LinearRegression, Ridge, Lasso
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score
import numpy as np

np.random.seed(42)
X = np.random.randn(100, 20)  # 20 features, many irrelevant
y = 3*X[:, 0] + 2*X[:, 1] + np.random.randn(100)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

for name, model in [('OLS', LinearRegression()),
                    ('Ridge', Ridge(alpha=1.0)),
                    ('Lasso', Lasso(alpha=0.1))]:
    model.fit(X_train, y_train)
    print(f'{name} R2: {r2_score(y_test, model.predict(X_test)):.3f}')

Cross-Validation Score

A single train-test split can give a misleadingly high or low estimate of model performance depending on which examples happen to land in the test set. Cross-validation gives a more reliable estimate by averaging performance across multiple folds.

Use cross_val_score for a quick k-fold evaluation without manually splitting data. The returned scores are one per fold; report the mean and standard deviation to convey both expected performance and variability.

from sklearn.linear_model import LinearRegression
from sklearn.model_selection import cross_val_score
import numpy as np

np.random.seed(42)
X = np.random.randn(200, 3)
y = 2*X[:, 0] - X[:, 1] + np.random.randn(200)*0.5

model = LinearRegression()
scores = cross_val_score(
    model, X, y,
    cv=5,          # 5-fold cross-validation
    scoring='r2'   # use R2 as the metric
)

print('CV R2 scores per fold:', scores.round(3))
print(f'Mean R2: {scores.mean():.3f} (+/- {scores.std():.3f})')

End-to-End Pipeline

In professional ML workflows, preprocessing and the model are always combined into a Pipeline. This prevents data leakage during cross-validation (the scaler is fitted on each training fold separately) and simplifies deployment (one object to save and load).

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import cross_val_score
import numpy as np

np.random.seed(42)
X = np.random.randn(200, 3)
y = 2*X[:, 0] - X[:, 1] + np.random.randn(200)*0.5

# Scale + Fit in one object
pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('model', LinearRegression())
])

scores = cross_val_score(pipeline, X, y, cv=5, scoring='r2')
print(f'Pipeline CV R2: {scores.mean():.3f} (+/- {scores.std():.3f})')

# Fit once for deployment
pipeline.fit(X, y)
print('Pipeline fitted and ready for deployment.')

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: fit() computes optimal weights stored in coef_ and intercept_, predict() applies the learned line to new data, and residual plots reveal whether the linear assumption holds or a more complex model is needed. Next up we extend linear regression to multiple features, learn how to interpret each coefficient as a measure of feature importance, and evaluate with R-squared.

Часто задаваемые вопросы

Урок «Обучение линейной регрессии с scikit-learn» бесплатный?

Да — полный текст урока «Обучение линейной регрессии с scikit-learn» бесплатно доступен здесь в веб-версии. Чтобы практиковать его интерактивно (встроенный редактор кода и ИИ-репетитор 24/7) и разблокировать остальной курс Machine Learning Academy, подпишись на CoddyKit PRO. Курс Machine Learning Academy содержит 4 уроков всего.

Чему я научусь в уроке «Обучение линейной регрессии с scikit-learn»?

Вы используете sklearn.linear_model.LinearRegression для подгонки модели по обучающим данным, построения прогнозов и изучения выученных коэффициентов Ты практикуешь Machine Learning Academy с помощью реального кода, который запускаешь прямо в браузере, и ИИ-репетитор 24/7 отвечает на твои вопросы во время урока.

Нужен ли мне опыт, чтобы начать Machine Learning Academy?

Предыдущий опыт не требуется. Machine Learning Academy на CoddyKit структурирован для всех уровней — от новичков до продвинутых, поэтому ты можешь начать отсюда или с самого начала и учиться в своем темпе. Это урок 3 из 4.

Сколько времени занимает урок «Обучение линейной регрессии с scikit-learn»?

Большинство уроков CoddyKit занимают около 5–10 минут. Каждый из них компактный и интерактивный, поэтому ты постоянно делаешь прогресс и продолжаешь с того же места в веб-версии и приложении.

Можно ли писать и запускать код в этом уроке Machine Learning Academy?

Да. Каждый урок Machine Learning Academy включает встроенный редактор кода, поэтому ты пишешь и запускаешь реальный код прямо в браузере и получаешь моментальную обратную связь от AI — локальная установка не требуется.

Все уроки этого курса

Уравнение прямой: наклон, пересечение с осью и прогнозы
Функции потерь и метод наименьших квадратов
Обучение линейной регрессии с scikit-learn
Множественная линейная регрессия и важность признаков

← Назад к Machine Learning Academy