StackScholar

Deep Learning

Gradient Checkpointing

Discarding intermediate activations during the forward pass and recomputing them during backward.

Why interviewers ask about it

Trades roughly 30% extra compute for a large activation-memory reduction. The trade works because recompute is compute-bound and storage is memory-bound.

Related terms

Backpropagation

Computing gradients by applying the chain rule in reverse topological order over the computation graph, caching intermediates.

Mixed Precision Training

Running matmuls in 16-bit while keeping master weights and optimiser state in fp32.

This term is part of the free AI/ML Engineer interview preparation module - browse the full glossary for every definition.