AlgoPlus//machine learning / tokenization
Read the theory

Tokenization (BPE)

Learn subword tokens by repeatedly merging the most frequent adjacent pair.

Merges
Vocab
Your corpus (words; repeat a word for higher frequency)
Legend
Pair to merge
New token
AI Tutor Workspace
In a nutshell
Models don't read whole words or single letters but 'tokens' — subword chunks. Byte-pair encoding learns them by starting from characters and repeatedly merging the most frequent neighbouring pair into a new token. Common pieces like 'ing' end up as single tokens while rare words stay split, balancing vocabulary size against sequence length.
Ready
Press play to begin the cinematic walkthrough.
Neither letters nor words — meet in the middle. Glue the pairs that show up together most often, over and over, until 'ing' and 'low' are each one chunk but 'zyzzyva' still isn't.
Key terms
Go deeper in the lesson
Read the full theory, intuition & complexity for Tokenization (BPE).