How Transformers Work: Attention Is All You Need, Explained
The 2017 paper Attention Is All You Need changed NLP forever. Before it, sequence models meant RNNs or LSTMs — sequential, slow, and bad at long-range dependencies. The Transformer threw all of that out and replaced it with pure attention. This post walks through the key components with the actual math. The Problem with RNNs An RNN processes tokens one at a time. To compute the hidden state at step $t$, you need the state at $t-1$: ...