Linear Models: Regression and Classification
Train linear regression and logistic regression models.
Linear Models: Regression and Classification is a free Python Academy lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Python Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Linear Regression
LinearRegression fits a straight line (or hyperplane) by minimising least-squares error.
from sklearn.linear_model import LinearRegression
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
X, y = make_regression(n_samples=200, n_features=5, noise=10, random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=0)
model = LinearRegression().fit(X_tr, y_tr)
print("RMSE:", mean_squared_error(y_te, model.predict(X_te))**0.5)Model Coefficients
After fitting, inspect coef_ (feature weights) and intercept_.
from sklearn.linear_model import LinearRegression
import numpy as np
X = np.array([[1],[2],[3]])
y = np.array([2, 4, 6])
model = LinearRegression().fit(X, y)
print("Coef:", model.coef_) # [2.]
print("Intercept:", model.intercept_) # ~0Ridge and Lasso Regression
Ridge (L2) and Lasso (L1) add regularisation to prevent overfitting. alpha controls the regularisation strength.
from sklearn.linear_model import Ridge, Lasso
from sklearn.datasets import make_regression
X, y = make_regression(n_features=20, noise=15, random_state=0)
ridge = Ridge(alpha=1.0).fit(X, y)
lasso = Lasso(alpha=0.1).fit(X, y)
print("Ridge coef[:5]:", ridge.coef_[:5])
print("Lasso coef[:5]:", lasso.coef_[:5]) # some are 0 (sparse)Logistic Regression
LogisticRegression is a linear classifier for binary or multi-class problems. Despite the name, it predicts class probabilities.
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
X, y = make_classification(random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=0)
model = LogisticRegression().fit(X_tr, y_tr)
print("Accuracy:", accuracy_score(y_te, model.predict(X_te)))Multi-class Logistic Regression
Set multi_class="multinomial" or use the default OvR (one-vs-rest) for 3+ classes.
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_iris
X, y = load_iris(return_X_y=True)
model = LogisticRegression(max_iter=200).fit(X, y)
print("Classes:", model.classes_) # [0 1 2]
print("Accuracy:", model.score(X, y))Feature Scaling for Linear Models
Always scale features for linear models, especially with regularisation. Unscaled features give misleadingly different penalties.
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
pipe = Pipeline([
("scale", StandardScaler()),
("clf", LogisticRegression())
])
pipe.fit(X_tr, y_tr)
print(pipe.score(X_te, y_te))Learning Curve
A learning curve shows how model performance improves with more training data — useful for diagnosing under/overfitting.
from sklearn.model_selection import learning_curve
import numpy as np
train_sizes, train_scores, val_scores = learning_curve(
LogisticRegression(), X, y, cv=5,
train_sizes=np.linspace(0.1, 1.0, 5)
)
print("Val mean:", val_scores.mean(axis=1))R² Score for Regression
r2_score measures the proportion of variance explained. 1.0 is perfect, 0 means the model is no better than predicting the mean.
from sklearn.metrics import r2_score
from sklearn.linear_model import LinearRegression
from sklearn.datasets import make_regression
X, y = make_regression(noise=20, random_state=0)
model = LinearRegression().fit(X, y)
print("R²:", r2_score(y, model.predict(X)))Polynomial Features
Use PolynomialFeatures to fit non-linear relationships with a linear model.
from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LinearRegression
import numpy as np
X = np.array([[1],[2],[3],[4],[5]])
y = np.array([1, 4, 9, 16, 25]) # y = x^2
pipe = Pipeline([
("poly", PolynomialFeatures(degree=2)),
("lr", LinearRegression())
])
pipe.fit(X, y)
print(pipe.predict([[6]])) # ~36SGDClassifier and SGDRegressor
Stochastic gradient descent scales to very large datasets where standard solvers are too slow.
from sklearn.linear_model import SGDClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import make_classification
X, y = make_classification(n_samples=100_000, random_state=0)
pipe = Pipeline([("sc", StandardScaler()), ("sgd", SGDClassifier())])
pipe.fit(X, y)
print(pipe.score(X, y))Regularisation Paths
Vary alpha over a grid with RidgeCV/LassoCV which auto-select the best alpha via cross-validation.
from sklearn.linear_model import RidgeCV, LassoCV
from sklearn.datasets import make_regression
X, y = make_regression(n_features=20, noise=10, random_state=0)
ridge = RidgeCV(alphas=[0.1, 1, 10]).fit(X, y)
print("Best alpha:", ridge.alpha_)Quick Check
What is the key difference between Ridge and Lasso regression?
Recap
Use LinearRegression for regression, LogisticRegression for classification. Add Ridge/Lasso for regularisation. Always scale features. Use PolynomialFeatures for non-linear problems. Evaluate with r2_score (regression) or accuracy_score (classification).
Frequently asked questions
Is the “Linear Models: Regression and Classification” lesson free?
Yes — the full text of “Linear Models: Regression and Classification” is free to read here on the web, and the Python Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Python Academy course, upgrade to CoddyKit PRO.
What will I learn in “Linear Models: Regression and Classification”?
Train linear regression and logistic regression models. You practise Python Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Python Academy?
No prior experience is required. Python Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Linear Models: Regression and Classification” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Python Academy lesson?
Yes. Every Python Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- The scikit-learn API: fit, transform, predict
- Linear Models: Regression and Classification
- Tree-Based Models: Decision Trees and Random Forests
- Model Evaluation and Cross-Validation