Deep Learning Theory - Optimization
Part of MSRA project at OptiML Lab.
Project #1. Gradient Descent with Polyak’s Momentum Finds Flatter Minima via Large Catapults
Updated version accepted to ICML 2024 - 2nd Workshop on High-dimensional Learning Dynamics (HiLD): The Emergence of Structure and Reasoning.
Preliminary version (under different title “Large Catapults in Momentum Gradient Descent with Warmup: An Empirical Study”) accepted to NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning (M3L) as oral.
Joint work with Prin Phunyaphibarn (KAIST Math, intern, equal contributions), Chulhee Yun (KAIST AI), Bohan Wang (USTC), and Huishuai Zhang (Microsoft Research Asia - Theory Centre).
Project #2. A Statistical Analysis of Stochastic Gradient Noises for GNNs
Preliminary work accepted to KCC 2022. Joint work with Minchan Jeong, Namgyu Ho, and Se-Young Yun (KAIST AI).
Junghyun Lee
PhD Student
PhD student at GSAI, KAIST, jointly advised by Prof. Se-Young Yun and Prof. Chulhee Yun. Interested in mathematical and theoretical AI, i.e., essentially any machine learning challenges necessitating mathematical analysis. Recently focused on statistical problems arising from RLHF, including interactive machine learning and low-dimensional structure recovery.
