Transformers
FlashAttention
An IO-aware exact attention algorithm that tiles Q/K/V into SRAM and uses online softmax so the n×n score matrix never reaches HBM.
Why interviewers ask about it
It is exact and does not reduce FLOPs - it eliminates memory traffic. Calling it an approximation is the classic mistake (that is Linformer/Performer).
Related terms
This term is part of the free AI/ML Engineer interview preparation module - browse the full glossary for every definition.