ArticleAshish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia PolosukhinResearch
The Transformer architecture replaces sequential recurrent architectures, enabling massively parallel training on GPU hardware and delivering superior benchmark performance at lower compute cost.
By demonstrating that attention alone suffices for sequence transduction, the authors remove the sequential processing bottleneck of RNNs. This structural simplification allows for parallelised training across multiple GPUs, achieving state-of-the-art translation quality with significantly fewer training resources.
Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.
Similar in content, found by the AI search.
Article:Wikipedia contributors
This Wikipedia overview covers the mechanics and historical evolution of transformer models. Built upon multi-head attention, the architecture abandons recurrence, enabling highly parallel training on GPUs.
Article:Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio
In this paper, Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio propose an approach that resolves the fixed-length vector bottleneck in encoder-decoder translation models through soft search.
Article:Yonghui Wu, Mike Schuster, Zhifeng Chen, et al.
Google researchers present GNMT (Google's Neural Machine Translation system), addressing long-standing obstacles in neural translation such as high compute costs during training, slow inference, and difficulties handling rare words.
Article:Albert Gu, Tri Dao
Albert Gu and Tri Dao introduce Mamba, a sequence modeling architecture based on selective structured state space models (SSMs). It achieves linear scaling with sequence length, eliminates attention and MLP blocks, and delivers five times higher inference throughput than Transformers.
Article:Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, Sepp Hochreiter
A research team led by Sepp Hochreiter introduces xLSTM, an extension of the original Long Short-Term Memory architecture designed for large-scale language modelling. The approach addresses historical scaling limits of LSTMs and demonstrates competitive performance against state-of-the-art Transformers and State Space Models.
Article:Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean
A research team led by Tomas Mikolov introduces two novel model architectures designed to compute continuous vector representations of words from massive datasets. The approach delivers higher accuracy with substantially lower computational requirements than prior neural network methods.
Parts of this content were created with the assistance of AI.
