The unstable gradient problem is a fundamental problem that occurs to gradients in early layers of a neural network.