Backpropagation

The algorithm that works out how much each weight contributed to the error, so gradient descent can fix it.

Backpropagation runs the error backwards through the network, layer by layer, assigning blame to every weight using the chain rule. It is what makes training deep networks computationally feasible, and it is the reason deep learning works at all.

In practice: The output was wrong by 0.3 — backprop decides which of the billions of weights are responsible.