What is a transformer, actually?
Attention explained with a dinner-party analogy that finally makes it click. No equations until you ask for them.
Almost every AI system you have heard of in the last few years - ChatGPT, Claude, Gemini, the models behind modern translation, coding assistants and many image generators - is built on the same architecture: the transformer. It was introduced in 2017 by a team of eight researchers at Google in a paper with a deliberately bold title, Attention Is All You Need.
The paper set out to improve machine translation. It ended up replacing most of what came before it across language, vision, audio and biology. This article explains what a transformer does, why it worked so well, and what "attention" really means - first in plain language, then with the one equation that matters.
The problem transformers solved
Language is sequential, but meaning is not local. In the sentence "The trophy didn't fit in the suitcase because it was too big", the word "it" refers to the trophy - a word six positions earlier. Change "big" to "small" and "it" now refers to the suitcase. Understanding a word often requires looking at words far away from it.
Before 2017, the dominant approach for text was the recurrent neural network (RNN) and its improved variant, the LSTM. These read a sentence one word at a time, carrying a running summary - a hidden state - from each step to the next. That design had two serious costs.
Long-range information faded. By the time the network reached the end of a long paragraph, details from the beginning had been compressed through dozens of updates and were often lost.
Training was slow. Because step 50 depends on step 49, the computation could not be spread across thousands of GPU cores in parallel. Scaling up was painful.
The transformer dropped recurrence entirely. Instead of passing a summary along a chain, it lets every word look directly at every other word in a single step. That mechanism is called self-attention.
Attention, explained as a dinner party
Picture a long dinner table. Each guest is a word in the sentence. Every guest wants to understand their own role in the conversation better, so each one does three things at once.
They ask a question. Each guest has something they are looking for. The word "it" is essentially asking: "Which noun am I standing in for?" This is the query.
They wear a name tag. Each guest advertises what they can offer. "Trophy" has a tag that says something like "concrete object, singular, could be big." This is the key.
They have something to say. If someone decides to listen to them, each guest has information to share. This is the value.
Every guest compares their question with everyone else's name tag. Where a question and a tag match well, the guest pays close attention; where they match poorly, they mostly ignore that person. Each guest then builds an updated understanding of themselves as a weighted blend of what everyone said - mostly from the people they paid attention to.
After one round, "it" has listened closely to "trophy" and now carries trophy-related meaning. The crucial detail: every guest does this simultaneously. Nobody waits for their neighbour to finish. That is what makes transformers fast to train on modern hardware.
The key idea: Attention lets each word decide, for itself, which other words matter - and pull in information from them directly, no matter how far away they are.
Many conversations at once: multi-head attention
A single round of questions captures one kind of relationship. But language has many at the same time: grammar (which noun goes with which verb), reference (what "it" points to), tone, topic, and more.
So transformers run several attention mechanisms in parallel, called heads. Back at the dinner table, imagine the same guests holding several separate conversations at once - one about who did what, one about time and tense, one about mood. Researchers who inspect trained models do find heads that specialise, for example ones that track the previous word or that link pronouns to the nouns they refer to, although many heads do things that are harder to name.
The full architecture
Attention is the star, but it sits inside a larger pipeline. Here is how a modern language model processes your text, end to end.
A decoder-only transformer, the design used by most chatbots. The highlighted block repeats dozens of times.
1. Tokenisation and embeddings
Text is split into tokens - whole words, word pieces or punctuation. Each token is mapped to an embedding: a long list of numbers (often thousands) that represents its meaning in a way the network can compute with. Words with related meanings end up with similar embeddings.
2. Positional information
Because attention looks at all tokens at once, it has no built-in sense of order - "dog bites man" and "man bites dog" would look identical. So the model adds positional information to each token. The original paper used fixed sine and cosine patterns; most modern models use learned or rotary position schemes.
3. The transformer block
Each block has two main parts. Self-attention lets tokens exchange information. A feed-forward network then processes each token individually, which is where a large share of the model's stored knowledge is thought to live. Around both parts are residual connections and normalisation, engineering details that let gradients flow through very deep stacks during training.
This block is repeated many times. Early layers tend to capture surface patterns like syntax; later layers build more abstract representations. Large models stack dozens of these blocks.
4. Predicting the next token
At the end, the model converts the final representation of the last position into a score for every token in its vocabulary. Those scores become probabilities, one token is chosen, it is appended to the input, and the whole process runs again. Every answer a chatbot writes is produced this way, one token at a time.
The final step: a probability for each possible next token (illustrative numbers).
Encoders, decoders and why it matters
The original transformer had two halves: an encoder that reads the input sentence and a decoder that writes the translation. Later models specialised.
The three transformer families.
Chat assistants are decoder-only. A detail that matters: when generating, each position can only attend to the tokens before it - the model is not allowed to peek at words it has not written yet. This is called causal masking.
Now the equation
You can understand transformers well without the maths, but the core of attention fits on one line, and it maps directly onto the dinner party.
Q·Kᵀ compares every question with every name tag, producing a grid of match scores.
÷ √dₖ scales the scores down so they do not grow too large as vectors get longer, which keeps training stable.
softmax turns each row of scores into percentages that add up to 100% - how much attention each guest pays to each other guest.
· V blends everyone's values using those percentages. The result is each token's updated representation.
Q, K and V are not separate inputs. They are all produced from the same token embeddings by three learned weight matrices - which is why it is called self-attention.
Why transformers won
Parallelism. All tokens are processed at once during training, so transformers make full use of GPUs and TPUs. That made it practical to train on far larger datasets.
Direct long-range connections. Any token can attend to any other in a single step, so distant context does not fade the way it did in RNNs.
Predictable scaling. Researchers found that performance kept improving smoothly as models, data and compute grew - which encouraged the huge investments behind today's models.
Generality. The same architecture works on image patches, audio frames, protein sequences and more. Anything that can be expressed as a sequence of tokens can be fed in.
The trade-offs
Attention compares every token with every other token, so its cost grows with the square of the input length. Doubling the context roughly quadruples the attention work. This is why long context windows are expensive and why so much research goes into more efficient attention variants, caching and alternative architectures.
Transformers are also data-hungry and opaque. They learn from enormous text collections, and even their designers cannot fully explain how specific knowledge is stored inside them. The growing field of interpretability research is working on exactly that question.
Summary
A transformer processes all tokens at once, using self-attention to let each token pull in information from the others.
Each token produces a query, a key and a value; attention scores come from matching queries against keys.
Multi-head attention runs several of these comparisons in parallel to capture different relationships.
Blocks of attention + feed-forward layers are stacked many times, and the final layer predicts the next token.
The design won because it is parallel, handles long-range context and scales predictably - at the cost of quadratic attention.