In a nutshell
An activation function is the bend applied to each neuron's output. It matters because stacking plain linear layers just gives one big straight line; the bend is the non-linearity that lets a deep network fit curved patterns. ReLU keeps positive inputs and zeroes the rest — cheap and the modern default. Sigmoid squashes everything into (0, 1), and tanh into (−1, 1); both are smooth but their gradients fade for large inputs, which can slow learning.