Scaling and Selecting Features
Keep the features that actually help.
Scaling and Selecting Features is a free NLP Academy lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the NLP Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Too Many Features Hurt
Text models can explode to tens of thousands of features. Many add only noise, so trimming the list often improves both speed and accuracy.
Why Scale at All
Some models compare feature magnitudes directly. If one feature ranges 0 to 1000, it can drown out the rest unless you scale them first.
Standardizing Numbers
StandardScaler shifts each feature to zero mean and unit variance. Now every numeric feature speaks on the same scale.
from sklearn.preprocessing import StandardScaler
scaled = StandardScaler().fit_transform(numeric_features)Careful With Sparse Data
Centering a sparse TF-IDF matrix fills it with nonzeros and wastes memory. Use MaxAbsScaler, which scales without destroying sparsity.
from sklearn.preprocessing import MaxAbsScaler
scaled = MaxAbsScaler().fit_transform(tfidf_matrix)Drop the Dead Weight
Features that barely vary tell the model nothing. A VarianceThreshold filter removes near-constant columns in one quick pass.
Pick the Most Useful
SelectKBest keeps the top features by a scoring test like chi-squared. You ask for the best k and it drops the rest.
from sklearn.feature_selection import SelectKBest, chi2
best = SelectKBest(chi2, k=2000).fit_transform(X, y)Let the Model Choose
An L1-penalized model pushes useless weights to zero on its own. This built-in selection is often called embedded feature selection.
Fit on Train Only
Always fit scalers and selectors on training data alone, then apply them to test data. Mixing them causes leakage and rosy fake scores.
Keep It in a Pipeline
Putting scaling and selection inside a Pipeline runs them in the right order every time. It also blocks accidental leakage during validation.
from sklearn.pipeline import Pipeline
pipe = Pipeline([("select", SelectKBest(chi2, k=2000)), ("clf", model)])Fewer Features, Faster Model
A leaner feature set trains faster, needs less memory, and is easier to explain. Smaller is often better once noise is gone. ⚡
Measure, Do Not Guess
Try a few values of k and compare validation scores. Let the numbers, not a hunch, decide how many features to keep.
Quick Check
Which scaler keeps a sparse matrix sparse?
Recap
Scale numeric features and select the useful ones with SelectKBest or L1. Use MaxAbsScaler for sparse data, and fit everything inside a pipeline. ✅
Frequently asked questions
Is the “Scaling and Selecting Features” lesson free?
Yes — the full text of “Scaling and Selecting Features” is free to read here on the web, and the NLP Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the NLP Academy course, upgrade to CoddyKit PRO.
What will I learn in “Scaling and Selecting Features”?
Keep the features that actually help. You practise NLP Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start NLP Academy?
No prior experience is required. NLP Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Scaling and Selecting Features” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this NLP Academy lesson?
Yes. Every NLP Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Beyond Bag-of-Words
- Character N-Grams for Robustness
- Combining Multiple Feature Types
- Scaling and Selecting Features