Bohan Wang

Large Catapults in Momentum Gradient Descent with Warmup: An Empirical Study

Although gradient descent with momentum is widely used in modern deep learning, a concrete understanding of its effects on the training …

Prin Phunyaphibarn, Junghyun Lee, Bohan Wang, Huishuai Zhang, Chulhee Yun

Deep Learning Theory - Optimization

specific topics tbd

Junghyun Lee