Adding more layers should give a neural network more capacity, but very deep networks can become harder to optimize. Residual connections, popularized by ResNet, solve an important part of this problem and allow hundreds of layers to be trained effectively.
The degradation problem
A deeper model does not always achieve lower training error than a shallower one. Gradients may weaken as they move backward, and layers can struggle to preserve useful information. The network then becomes difficult to optimize even when overfitting is not the main issue.
Learn a residual instead
A residual block adds its input directly to the output of several learned layers. Instead of forcing those layers to learn a complete transformation, the block learns the difference between the desired output and the original input. If no major change is needed, the block can stay close to an identity mapping.
Skip connections improve information flow
The direct path gives activations and gradients a shorter route through the network. Earlier layers receive a stronger learning signal, while useful representations can pass forward without being repeatedly reconstructed. This makes optimization more stable and supports much deeper architectures.
Dimensions must remain compatible
The input and learned output need matching shapes before they can be added. When the number of channels or spatial resolution changes, a projection layer such as a one-by-one convolution can align them. This is common between major stages of a ResNet.
Why the idea matters
Residual learning is now used beyond image classification in detection, segmentation, language models, and many scientific applications. The key lesson is simple: a network does not always need to replace its existing representation. Sometimes learning a small correction is the easier and more reliable task.