LLMs
BPE (Byte-Pair Encoding)
Subword tokenisation that starts from bytes and iteratively merges the most frequent adjacent pair.
Why interviewers ask about it
Non-English text can cost 3–5× more tokens; digit tokenisation inconsistency hurts arithmetic; trailing whitespace changes completions. All downstream cost estimates should be in tokens.
This term is part of the free AI/ML Engineer interview preparation module - browse the full glossary for every definition.