ArticleKaiming He, Xiangyu Zhang, Shaoqing Ren, Jian SunResearch
The paper tackles the degradation and optimization difficulties inherent in training very deep networks. Residual connections allow networks to scale up significantly in depth, forming a cornerstone for modern computer vision architectures.
By reformulating the objective of individual network layers to learn residual functions relative to their inputs, the authors bypassed the barrier that previously hindered the optimization of extremely deep models. ResNet demonstrated that depth directly improves representation quality across various vision tasks, achieving state-of-the-art results across major benchmarks without incurring proportional computational costs.
Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.
Similar in content, found by the AI search.
Article:Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun
The authors analyse the propagation mechanics behind residual networks, demonstrating that forward and backward signals pass directly between blocks when identity mappings serve as skip connections and after-addition activations. This leads to an improved residual unit design that eases training and boosts generalization across deep architectures.
Link:Xavier Glorot, Yoshua Bengio
This research paper by Xavier Glorot and Yoshua Bengio investigates why standard gradient descent with random initialization performs poorly in deep neural networks. The authors examine the role of activation functions and gradient behaviour across layers, proposing a new initialization scheme that significantly accelerates convergence.
Article:David E. Rumelhart, Geoffrey E. Hinton & Ronald J. Williams
The research paper by David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams introduces the backpropagation learning procedure for neural networks, adjusting connection weights to minimize output error.
Article:Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, Sepp Hochreiter
A research team led by Sepp Hochreiter introduces xLSTM, an extension of the original Long Short-Term Memory architecture designed for large-scale language modelling. The approach addresses historical scaling limits of LSTMs and demonstrates competitive performance against state-of-the-art Transformers and State Space Models.
Article:Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin
The authors introduce the Transformer, a network architecture that completely replaces recurrence and convolutions with attention mechanisms.
Article:Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio
In this paper, Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio propose an approach that resolves the fixed-length vector bottleneck in encoder-decoder translation models through soft search.
Parts of this content were created with the assistance of AI.
