Grouped-Query Attention (GQA)

Transformers

Grouped-Query Attention (GQA)

Multiple query heads sharing one key/value head, typically 8 KV heads for 32–64 query heads.

Why interviewers ask about it

Not an accuracy improvement - a serving-economics decision. 8× smaller KV cache means ~8× more concurrent requests per GPU at near-identical quality.

KV Cache

Storing past keys and values during autoregressive decoding so each step is O(n) instead of O(n²).

This term is part of the free AI/ML Engineer interview preparation module - browse the full glossary for every definition.