Machine Learning Academy · 课时

使用 scikit-learn 训练线性回归

您将使用 sklearn.linear_model.LinearRegression 在训练数据上拟合模型、进行预测，并检查模型学习到的系数

使用 scikit-learn 训练线性回归 是 CoddyKit 上的免费 Machine Learning Academy 课时。这是第 3 节课，共 4 节。你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。这是 Machine Learning Academy 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Machine Learning Academy 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

scikit-learn's LinearRegression API

Scikit-learn's LinearRegression class provides a clean, consistent API for fitting ordinary least-squares linear regression. Like all scikit-learn estimators, it follows the same pattern: instantiate, fit, predict.

Under the hood, it uses the Normal Equation (or a numerically stable SVD decomposition for large feature matrices) to find the exact optimal weights in a single computation — no iterative training loop is needed. This makes it extremely fast even on large datasets with many features.

from sklearn.linear_model import LinearRegression
import numpy as np

# Instantiate with optional parameters
model = LinearRegression(
    fit_intercept=True,   # default: add bias term
    copy_X=True,          # don't modify input arrays
    n_jobs=-1             # use all CPU cores
)

print(model)  # shows default parameters

Preparing the Data

Before training, your data must be in the correct shape. scikit-learn expects:

X — a 2D array of shape (n_samples, n_features)
y — a 1D array of shape (n_samples,)

The most common shape error is passing a 1D array for X when you have one feature. The fix is to reshape with .reshape(-1, 1). Always split into training and test sets before any fitting — evaluation on training data gives misleadingly perfect scores.

import numpy as np
from sklearn.model_selection import train_test_split

# Simulate housing data
np.random.seed(42)
sqft = np.random.uniform(500, 3000, 200)
price = 150 * sqft + 50000 + np.random.normal(0, 25000, 200)

# Reshape X to 2D (n_samples, n_features)
X = sqft.reshape(-1, 1)  # shape (200, 1)
y = price                  # shape (200,)

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)
print('Train size:', X_train.shape, '| Test size:', X_test.shape)

Fitting the Model

Calling model.fit(X_train, y_train) computes the optimal weights using the Normal Equation and stores them inside the model object. After fitting, two attributes become available:

model.coef_ — an array of weights, one per feature
model.intercept_ — the bias term (scalar)

The trailing underscore in attribute names is a scikit-learn convention indicating that the attribute was set during fitting — it does not exist before fit() is called.

from sklearn.linear_model import LinearRegression
import numpy as np
from sklearn.model_selection import train_test_split

np.random.seed(42)
X = np.random.uniform(500, 3000, 200).reshape(-1, 1)
y = 150 * X.ravel() + 50000 + np.random.normal(0, 25000, 200)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

model = LinearRegression()
model.fit(X_train, y_train)

print(f'Slope (coef_):     {model.coef_[0]:.2f}')   # should be ~150
print(f'Intercept:         {model.intercept_:.2f}')  # should be ~50000

Making Predictions

After fitting, call model.predict(X) to generate predictions. This computes the dot product of the feature matrix with the learned weights, plus the intercept. You can predict on training data, test data, or entirely new data.

Remember: evaluating on training data is not a valid performance estimate. Always evaluate on the held-out test set to get an honest estimate of how well the model will perform on unseen data.

# Predict on test set
y_pred = model.predict(X_test)

print('First 5 predictions vs actual:')
for actual, pred in zip(y_test[:5], y_pred[:5]):
    print(f'  Actual: ${actual:,.0f}  |  Predicted: ${pred:,.0f}  |  Error: ${actual-pred:,.0f}')

# Predict a single new house
new_house = np.array([[1750]])  # must be 2D
print(f'\nPrediction for 1750 sqft: ${model.predict(new_house)[0]:,.0f}')

Evaluating on the Test Set

Use scikit-learn's metrics functions to quantify test-set performance. The model.score(X_test, y_test) shortcut returns R² directly. For the full picture, also compute RMSE and MAE.

Comparing these metrics across different models (e.g., linear regression vs random forest) allows you to objectively select the best approach for your dataset. Always include a naive baseline (like always predicting the mean of y) to confirm your model is actually adding value.

from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
import numpy as np

y_pred = model.predict(X_test)

rmse = np.sqrt(mean_squared_error(y_test, y_pred))
mae  = mean_absolute_error(y_test, y_pred)
r2   = r2_score(y_test, y_pred)

print(f'Test RMSE: ${rmse:,.0f}')
print(f'Test MAE:  ${mae:,.0f}')
print(f'Test R2:   {r2:.3f}')

# Baseline: always predict the mean
baseline_rmse = np.sqrt(((y_test - y_train.mean()) ** 2).mean())
print(f'Baseline RMSE: ${baseline_rmse:,.0f}  (must beat this!)')

Visualising the Regression Line

After fitting, plot the learned regression line on top of the training data scatter plot. This visual check answers: does the line make sense? Is it in roughly the right place? Are there large groups of outliers that the model consistently misses?

For a single-feature model this is straightforward. For multi-feature models you will need to plot predictions vs actual values (a residual plot or a 'predicted vs actual' scatter plot) instead, since the decision boundary lives in high-dimensional space.

import matplotlib.pyplot as plt
import numpy as np

# Plot training data and the fitted line
plt.figure(figsize=(8, 5))
plt.scatter(X_train, y_train, alpha=0.4, label='Training data')
plt.scatter(X_test, y_test, alpha=0.4, color='orange', label='Test data')

# Draw the regression line across the full range
x_range = np.linspace(500, 3000, 100).reshape(-1, 1)
y_range = model.predict(x_range)
plt.plot(x_range, y_range, 'r-', linewidth=2, label='Regression line')

plt.xlabel('Square Footage')
plt.ylabel('Price ($)')
plt.legend()
plt.title('Linear Regression Fit')
plt.show()

Interpreting the Coefficients

The learned coef_ values carry direct business interpretation. For a house-price model with multiple features, each coefficient tells you how much the predicted price changes when that feature increases by one unit, holding all other features constant (the ceteris paribus interpretation).

However, this interpretation is only valid when features are on similar scales. If one feature is in metres and another is in kilometres, their coefficients are not directly comparable. Always scale features before interpreting coefficients in multi-feature models.

from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import StandardScaler
import numpy as np
import pandas as pd

# Multi-feature housing data
features = ['sqft', 'bedrooms', 'bathrooms', 'age']
np.random.seed(0)
X_data = pd.DataFrame({
    'sqft': np.random.randint(600, 3000, 100),
    'bedrooms': np.random.randint(1, 6, 100),
    'bathrooms': np.random.randint(1, 4, 100),
    'age': np.random.randint(1, 50, 100)
})
y_data = 150*X_data['sqft'] + 8000*X_data['bedrooms'] - 500*X_data['age'] + 30000

model = LinearRegression().fit(X_data, y_data)
for feat, coef in zip(features, model.coef_):
    print(f'{feat}: {coef:.1f}')

Residual Analysis

Plotting residuals against predicted values is a critical diagnostic step. For a well-fit model, residuals should be randomly scattered around zero with no discernible pattern. Systematic patterns indicate a problem:

A funnel shape (residuals spread more for larger predictions) indicates heteroscedasticity — the variance of errors is not constant.
A curved pattern indicates the true relationship is non-linear and a linear model is under-fitting.
Residuals that are systematically positive or negative for certain prediction ranges mean the model is biased in that region.

import matplotlib.pyplot as plt
import numpy as np

y_pred = model.predict(X_test)
residuals = y_test - y_pred

plt.figure(figsize=(8, 5))
plt.scatter(y_pred, residuals, alpha=0.5)
plt.axhline(y=0, color='red', linestyle='--', linewidth=1.5)
plt.xlabel('Predicted Values')
plt.ylabel('Residuals')
plt.title('Residual Plot (should be random scatter around 0)')
plt.show()

print(f'Mean residual: {residuals.mean():.2f}  (should be ~0)')
print(f'Std of residuals: {residuals.std():.2f}')

Regularised Linear Regression

Plain LinearRegression can overfit when you have many features relative to training examples, or when features are highly correlated (multicollinearity). Scikit-learn provides two regularised variants:

Ridge (L2 regularisation) — shrinks all coefficients towards zero proportionally, keeping all features but reducing their magnitude.
Lasso (L1 regularisation) — shrinks some coefficients all the way to zero, effectively performing feature selection.

The alpha parameter controls regularisation strength: higher alpha = more shrinkage = simpler model.

from sklearn.linear_model import LinearRegression, Ridge, Lasso
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score
import numpy as np

np.random.seed(42)
X = np.random.randn(100, 20)  # 20 features, many irrelevant
y = 3*X[:, 0] + 2*X[:, 1] + np.random.randn(100)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

for name, model in [('OLS', LinearRegression()),
                    ('Ridge', Ridge(alpha=1.0)),
                    ('Lasso', Lasso(alpha=0.1))]:
    model.fit(X_train, y_train)
    print(f'{name} R2: {r2_score(y_test, model.predict(X_test)):.3f}')

Cross-Validation Score

A single train-test split can give a misleadingly high or low estimate of model performance depending on which examples happen to land in the test set. Cross-validation gives a more reliable estimate by averaging performance across multiple folds.

Use cross_val_score for a quick k-fold evaluation without manually splitting data. The returned scores are one per fold; report the mean and standard deviation to convey both expected performance and variability.

from sklearn.linear_model import LinearRegression
from sklearn.model_selection import cross_val_score
import numpy as np

np.random.seed(42)
X = np.random.randn(200, 3)
y = 2*X[:, 0] - X[:, 1] + np.random.randn(200)*0.5

model = LinearRegression()
scores = cross_val_score(
    model, X, y,
    cv=5,          # 5-fold cross-validation
    scoring='r2'   # use R2 as the metric
)

print('CV R2 scores per fold:', scores.round(3))
print(f'Mean R2: {scores.mean():.3f} (+/- {scores.std():.3f})')

End-to-End Pipeline

In professional ML workflows, preprocessing and the model are always combined into a Pipeline. This prevents data leakage during cross-validation (the scaler is fitted on each training fold separately) and simplifies deployment (one object to save and load).

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import cross_val_score
import numpy as np

np.random.seed(42)
X = np.random.randn(200, 3)
y = 2*X[:, 0] - X[:, 1] + np.random.randn(200)*0.5

# Scale + Fit in one object
pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('model', LinearRegression())
])

scores = cross_val_score(pipeline, X, y, cv=5, scoring='r2')
print(f'Pipeline CV R2: {scores.mean():.3f} (+/- {scores.std():.3f})')

# Fit once for deployment
pipeline.fit(X, y)
print('Pipeline fitted and ready for deployment.')

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: fit() computes optimal weights stored in coef_ and intercept_, predict() applies the learned line to new data, and residual plots reveal whether the linear assumption holds or a more complex model is needed. Next up we extend linear regression to multiple features, learn how to interpret each coefficient as a measure of feature importance, and evaluate with R-squared.

常见问题解答

「使用 scikit-learn 训练线性回归」课时是免费的吗？

是的 — 「使用 scikit-learn 训练线性回归」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Machine Learning Academy 课程的其余内容，请升级到 CoddyKit PRO。 Machine Learning Academy 课程共包含 4 节课。

「使用 scikit-learn 训练线性回归」这节课中我会学到什么？

您将使用 sklearn.linear_model.LinearRegression 在训练数据上拟合模型、进行预测，并检查模型学习到的系数你通过在浏览器中直接运行的动手代码来练习 Machine Learning Academy，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 Machine Learning Academy 需要有经验吗？

无需任何先前经验。CoddyKit 上的 Machine Learning Academy 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。这是第 3 节课，共 4 节。

「使用 scikit-learn 训练线性回归」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 Machine Learning Academy 课中编写并运行代码吗？

能。每节 Machine Learning Academy 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

← 返回 Machine Learning Academy