Why Nonlinearity Unlocks Real Power
Stacking linear layers stays linear without it.
Linear All the Way Down
A neural layer is just a weighted sum: it scales and shifts its inputs. On its own, that operation is perfectly linear. 📏
Stacking Doesn't Help
Here is the catch: stacking two linear layers just gives you another linear layer. No matter how many you stack, the result stays a single straight line.
All lessons in this course
- Why Nonlinearity Unlocks Real Power
- ReLU and Its Leaky & GELU Cousins
- Sigmoid & Tanh: Squashing to a Range
- Softmax for Probabilities