Deep Learning
He Initialization
Weight variance 2/fan_in, compensating for ReLU zeroing roughly half its inputs.
Why interviewers ask about it
Xavier (2/(fan_in+fan_out)) is for symmetric activations like tanh. Using Xavier with ReLU in a deep network causes slow signal decay.
Related terms
This term is part of the free AI/ML Engineer interview preparation module - browse the full glossary for every definition.