Linear Models: Regression and Classification
Train linear regression and logistic regression models.
Linear Models: Regression and Classification is a free Python Academy lesson on CoddyKit. This is lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Python Academy learning path, and your progress syncs across the web and the CoddyKit app. The Python Academy course includes 4 lessons in total.
Linear Regression
LinearRegression fits a straight line (or hyperplane) by minimising least-squares error.
from sklearn.linear_model import LinearRegression
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
X, y = make_regression(n_samples=200, n_features=5, noise=10, random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=0)
model = LinearRegression().fit(X_tr, y_tr)
print("RMSE:", mean_squared_error(y_te, model.predict(X_te))**0.5)Model Coefficients
After fitting, inspect coef_ (feature weights) and intercept_.
from sklearn.linear_model import LinearRegression
import numpy as np
X = np.array([[1],[2],[3]])
y = np.array([2, 4, 6])
model = LinearRegression().fit(X, y)
print("Coef:", model.coef_) # [2.]
print("Intercept:", model.intercept_) # ~0Ridge and Lasso Regression
Ridge (L2) and Lasso (L1) add regularisation to prevent overfitting. alpha controls the regularisation strength.
from sklearn.linear_model import Ridge, Lasso
from sklearn.datasets import make_regression
X, y = make_regression(n_features=20, noise=15, random_state=0)
ridge = Ridge(alpha=1.0).fit(X, y)
lasso = Lasso(alpha=0.1).fit(X, y)
print("Ridge coef[:5]:", ridge.coef_[:5])
print("Lasso coef[:5]:", lasso.coef_[:5]) # some are 0 (sparse)Logistic Regression
LogisticRegression is a linear classifier for binary or multi-class problems. Despite the name, it predicts class probabilities.
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
X, y = make_classification(random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=0)
model = LogisticRegression().fit(X_tr, y_tr)
print("Accuracy:", accuracy_score(y_te, model.predict(X_te)))Multi-class Logistic Regression
Set multi_class="multinomial" or use the default OvR (one-vs-rest) for 3+ classes.
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_iris
X, y = load_iris(return_X_y=True)
model = LogisticRegression(max_iter=200).fit(X, y)
print("Classes:", model.classes_) # [0 1 2]
print("Accuracy:", model.score(X, y))Feature Scaling for Linear Models
Always scale features for linear models, especially with regularisation. Unscaled features give misleadingly different penalties.
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
pipe = Pipeline([
("scale", StandardScaler()),
("clf", LogisticRegression())
])
pipe.fit(X_tr, y_tr)
print(pipe.score(X_te, y_te))Learning Curve
A learning curve shows how model performance improves with more training data — useful for diagnosing under/overfitting.
from sklearn.model_selection import learning_curve
import numpy as np
train_sizes, train_scores, val_scores = learning_curve(
LogisticRegression(), X, y, cv=5,
train_sizes=np.linspace(0.1, 1.0, 5)
)
print("Val mean:", val_scores.mean(axis=1))R² Score for Regression
r2_score measures the proportion of variance explained. 1.0 is perfect, 0 means the model is no better than predicting the mean.
from sklearn.metrics import r2_score
from sklearn.linear_model import LinearRegression
from sklearn.datasets import make_regression
X, y = make_regression(noise=20, random_state=0)
model = LinearRegression().fit(X, y)
print("R²:", r2_score(y, model.predict(X)))Polynomial Features
Use PolynomialFeatures to fit non-linear relationships with a linear model.
from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LinearRegression
import numpy as np
X = np.array([[1],[2],[3],[4],[5]])
y = np.array([1, 4, 9, 16, 25]) # y = x^2
pipe = Pipeline([
("poly", PolynomialFeatures(degree=2)),
("lr", LinearRegression())
])
pipe.fit(X, y)
print(pipe.predict([[6]])) # ~36SGDClassifier and SGDRegressor
Stochastic gradient descent scales to very large datasets where standard solvers are too slow.
from sklearn.linear_model import SGDClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import make_classification
X, y = make_classification(n_samples=100_000, random_state=0)
pipe = Pipeline([("sc", StandardScaler()), ("sgd", SGDClassifier())])
pipe.fit(X, y)
print(pipe.score(X, y))Regularisation Paths
Vary alpha over a grid with RidgeCV/LassoCV which auto-select the best alpha via cross-validation.
from sklearn.linear_model import RidgeCV, LassoCV
from sklearn.datasets import make_regression
X, y = make_regression(n_features=20, noise=10, random_state=0)
ridge = RidgeCV(alphas=[0.1, 1, 10]).fit(X, y)
print("Best alpha:", ridge.alpha_)Quick Check
What is the key difference between Ridge and Lasso regression?
Recap
Use LinearRegression for regression, LogisticRegression for classification. Add Ridge/Lasso for regularisation. Always scale features. Use PolynomialFeatures for non-linear problems. Evaluate with r2_score (regression) or accuracy_score (classification).
Learn Python with an AI tutor — free
Write and run real code in your browser, get instant help from a 24/7 AI tutor, and pick up where you left off on the web or in the app.
- Courses
- 76
- Lessons
- 320
Frequently Asked Questions
Is the “Linear Models: Regression and Classification” lesson free?
Yes — the full text of “Linear Models: Regression and Classification” is free to read here on the web. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Python Academy course, upgrade to CoddyKit PRO. The Python Academy course includes 4 lessons in total.
What will I learn in “Linear Models: Regression and Classification”?
Train linear regression and logistic regression models. You practise Python Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Python Academy?
No prior experience is required. Python Academy on CoddyKit is structured for beginners through advanced learners, so you can start here or from the beginning and move at your own pace. This is lesson 2 of 4.
How long does the “Linear Models: Regression and Classification” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Python Academy lesson?
Yes. Every Python Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- The scikit-learn API: fit, transform, predict
- Linear Models: Regression and Classification
- Tree-Based Models: Decision Trees and Random Forests
- Model Evaluation and Cross-Validation