Backpropagation

Backpropagation is the method through which a normal neural network trains and minimizes loss. The goal of backpropagation is compute a corresponding update matrix (\(\Delta \mathbf{w}\)) for each weight matrix in the network such that the loss function decreases.

Backpropagation primarily relies upon the Multivariable Chain Rule and the Jacobian.