Before Q, K, and V: Reconstructing the Transformer

The article explores the fundamental reasons behind the Transformer architecture's design, explaining why keys, queries, and values are essential components. It argues that these elements arise naturally from design constraints and discusses how the feedforward MLP block can be seen as a key-value store, offering insights into why Transformers outperform recurrent neural networks.