Transformers
Scaled Dot-Product Attention
softmax(QKᵀ/√d_k)V - a soft dictionary lookup where Q asks, K advertises and V delivers.
Why interviewers ask about it
The √d_k exists because Var(q·k) = d_k for unit-variance components; unscaled logits saturate softmax and its gradient vanishes.
Related terms
This term is part of the free AI/ML Engineer interview preparation module - browse the full glossary for every definition.