Pourquoi la non-linéarité libère toute la puissance

A neural layer is just a weighted sum: it scales and shifts its inputs. On its own, that operation is perfectly linear. 📏

Here is the catch: stacking two linear layers just gives you another linear layer. No matter how many you stack, the result stays a single straight line.

Two linear steps collapse into one. The combined weight is simply the product of the two weight matrices, so depth buys you nothing here.

An activation function bends the signal between layers. That small kink is what stops layers from collapsing into one.

Once you insert a nonlinearity, each layer can reshape the data differently. Now depth actually adds power instead of repeating the same map.

With nonlinearity, your network can carve out curved decision boundaries. A plain linear model can only ever draw a straight cut.

Enough neurons plus a nonlinearity can approximate almost any function. This is the famous universal approximation idea. ✨

You place the activation right after each linear layer, inside the forward pass. It transforms the layer's output before the next layer sees it.

Images, speech, and language are deeply nonlinear patterns. Only a network that can bend can hope to model them well.

Forget the activation and your fancy deep model quietly becomes a single linear regression in disguise. The depth is wasted.

You usually apply the same activation after every hidden layer, then choose a special one at the output to match your task.

Think about what happens without any activation function.

Stacked linear layers stay linear, so they cannot model curves. Inserting a nonlinearity unlocks depth and lets your network learn rich, real-world patterns. 🎯