Junghyun Lee
Junghyun Lee
Home
Experiences
Publications
Projects
Posts
Seminars
Contact
Light
Dark
Automatic
DL Theory
Conference Week (NeurIPS 2020)
Event OSI Lab Conference Week (NeurIPS 2020) Short summary In this seminar titled Topics not covered by anyone… (kind of), I will talk about 4 NeurIPS 2020 papers (all of which are theory-heavy) that I thought were interesting, whose topics were not covered by anyone in OSI Lab.
Jan 7, 2021
Heavy-tail behaviour of SGD - Part 2
Event Weekly OSI Lab Seminar Short summary This seminar continues on from Part 1, focusing more on the implication of such heavy-tailed theories of SGD to generalization capability of neural nets, and the origin of the heavy-tailedness.
Nov 6, 2020
Heavy-tail behaviour of SGD - Part 1
Event Weekly OSI Lab Seminar Short summary In this seminar, I will talk about a recent line of works that propose to analyze SGD under heavy-tail noise assumptions. Abstract One of the popular ways of analyzing the behavior of SGD and SGDm(SGD with momentum) is by considering it as a discretization of Langevin-type SDE.
Aug 14, 2020
Large Catapults in Momentum Gradient Descent with Warmup: An Empirical Study
Although gradient descent with momentum is widely used in modern deep learning, a concrete understanding of its effects on the training …
Prin Phunyaphibarn
,
Junghyun Lee
,
Bohan Wang
,
Huishuai Zhang
,
Chulhee Yun
PDF
Cite
Project
Poster
Slides
Deep Learning Theory - Optimization
specific topics tbd
Junghyun Lee
«
Cite
×