AlgoPlus//machine learning / positional-encoding
Read the theory

Positional Encoding

Inject order into an order-blind model with sinusoids of many frequencies.

Position
Dims
Legend
+1
−1
AI Tutor Workspace
In a nutshell
On its own, attention treats a sentence as an unordered bag of words. A positional encoding fixes that by adding a unique signature to each position, built from sine and cosine waves of different speeds. Because nearby positions get similar signatures, the model can also read how far apart two words are.
Ready
Press play to begin the cinematic walkthrough.
Attention is a bag of words with no sense of order, so hand each position a unique fingerprint made of layered sine waves — fast wiggles plus slow ones — and add it in.
Key terms
Go deeper in the lesson
Read the full theory, intuition & complexity for Positional Encoding.