DL Theory

Event OSI Lab Conference Week (NeurIPS 2020) Short summary In this seminar titled Topics not covered by anyone… (kind of), I will talk about 4 NeurIPS 2020 papers (all of which are theory-heavy) that I thought were interesting, whose topics were not covered by anyone in OSI Lab.

Event Weekly OSI Lab Seminar Short summary This seminar continues on from Part 1, focusing more on the implication of such heavy-tailed theories of SGD to generalization capability of neural nets, and the origin of the heavy-tailedness.

Event Weekly OSI Lab Seminar Short summary In this seminar, I will talk about a recent line of works that propose to analyze SGD under heavy-tail noise assumptions. Abstract One of the popular ways of analyzing the behavior of SGD and SGDm(SGD with momentum) is by considering it as a discretization of Langevin-type SDE.

Although gradient descent with Polyak’s momentum is widely used in modern machine and deep learning, a concrete understanding of …

specific topics tbd