AlgoPlus//machine learning / attention
Read the theory

Attention Mechanism

A query compares against keys, softmaxes the scores, and reads a weighted sum of values.

Phase
Output
Attends to
Legend
Attention weight
Top match
AI Tutor Workspace
In a nutshell
Attention lets a model decide, for each item, which other items matter right now. Every item produces a query, a key, and a value; the query is compared to all keys by dot product to get scores, a softmax turns those scores into attention weights that sum to 1, and the output is the weighted sum of the values. So instead of treating all inputs equally, the model dynamically pulls in exactly the context it needs — the core operation inside transformers.
Ready
Press play to begin the cinematic walkthrough.
A spotlight you can aim. Ask 'who's relevant to me?' (query vs keys), turn the answers into a focus distribution (softmax), and blend exactly those inputs (values).
Key terms
Go deeper in the lesson
Read the full theory, intuition & complexity for Attention Mechanism.