In a nutshell
Backpropagation is how a network learns from a mistake. A forward pass makes a prediction; the loss measures how wrong it is. Backprop then works backward from the output, using the chain rule to hand every weight a gradient — the direction and size of the nudge that would shrink the loss. An optimizer applies those nudges, and that one round trip is a single training step. It costs about the same as a forward pass.