Deep Learning
Learning Rate Warmup
Ramping the learning rate from ~0 over the first few hundred to few thousand steps.
Why interviewers ask about it
Adam's second-moment estimate is effectively a one-sample estimate at step 1; a small v produces an enormous update that can permanently damage an untrained network.
Related terms
This term is part of the free AI/ML Engineer interview preparation module - browse the full glossary for every definition.