ArticleKaiming He, Xiangyu Zhang, Shaoqing Ren, Jian SunResearch
The paper provides the structural foundation for ResNet v2, demonstrating that maintaining an unobstructed shortcut pathway allows stable optimization in networks exceeding 1,000 layers.
By routing activations purely through identity connections on the residual branch, gradients propagate unimpeded across the network. This insight shifted deep learning architecture design towards pre-activation blocks before weight layers.
Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.
Similar in content, found by the AI search.
Article:Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun
Kaiming He and colleagues introduce a residual learning framework designed to simplify the training of substantially deeper neural networks by reformulating layers to learn residual functions referencing layer inputs.
Link:Xavier Glorot, Yoshua Bengio
This research paper by Xavier Glorot and Yoshua Bengio investigates why standard gradient descent with random initialization performs poorly in deep neural networks. The authors examine the role of activation functions and gradient behaviour across layers, proposing a new initialization scheme that significantly accelerates convergence.
Article:Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, Sepp Hochreiter
A research team led by Sepp Hochreiter introduces xLSTM, an extension of the original Long Short-Term Memory architecture designed for large-scale language modelling. The approach addresses historical scaling limits of LSTMs and demonstrates competitive performance against state-of-the-art Transformers and State Space Models.
Repository:josch14/double-descent
A research project from the University of Ulm analyses the double descent phenomenon in two-layer fully connected neural networks (FCNNs) using the MNIST dataset. The work reproduces and expands upon Belkin et al. (2019), evaluating the impact of label noise and regularization techniques.
15starsPython
Article:David E. Rumelhart, Geoffrey E. Hinton & Ronald J. Williams
The research paper by David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams introduces the backpropagation learning procedure for neural networks, adjusting connection weights to minimize output error.
Article:Mikhail Belkin, Daniel Hsu, Siyuan Ma, Soumik Mandal
A research team led by Mikhail Belkin resolves the apparent contradiction between classical learning theory and modern over-parameterised models. The authors propose a unified 'double descent' performance curve explaining why model performance improves when capacity is increased beyond the interpolation threshold.
Parts of this content were created with the assistance of AI.
