DL Theory

Event Weekly OSI Lab Seminar Short summary In this seminar, I will about two recent papers on the high-dimensional limit of SGD and their statistical theories, which were recently described by the so-called homogeneized-SGD (HSGD).

Event Weekly OptiML Lab Group Meeting Short summary In this seminar, I will talk about the paper “Implicit Regularization or Implicit Conditioning? Exact Risk Trajectories of SGD in High Dimensions” (Paquette et al.

Event Weekly OSI Lab Seminar Short summary In this seminar, I will talk about a line of works that propose a new definition of “flatness” for loss landscapes in terms of entropy, and new variants of (S)GD based on the new definitions.

Event Weekly DL Theory & Stat Phy Seminar Short summary In this seminar, I will talk about the paper “Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalization” (Bach and Chizat, arXiv 2021) and the references therein.

Event Weekly DL Theory & Stat Phy Seminar Short summary In this seminar, I will talk about a line of works that tries to explain the phase transition behavior of the loss landscape (and the training dynamics as well) as the number of datas and parameters vary from a statistical physics point of view, namely, the jamming transition.

Event OSI Lab Conference Week (AISTATS & ICML 2022) Short summary This seminar covers some recent progress in optimization theory and deep learning theory. Specifically, for the optimization theory, I cover continuous-time analysis of accelerated optimization algorithms, and for the deep learning theory, I cover the so-called “edge of stability”.

Event Weekly DL Theory & Stat Phy Seminar Short summary In this seminar, I will talk about a line of works that propose a new definition of “flatness” for loss landscapes in terms of entropy, and new variants of (S)GD based on the new definitions.

Event Weekly DL Theory & Stat Phy Seminar Short summary In this seminar, I will talk about a recent line of works that propose to analyze SGD under heavy-tail noise assumptions using techniques from Levy-driven SDE theory and metastability analysis from statistical physics.

Event Weekly OSI Lab Seminar Short summary In this seminar, I will talk about the paper “Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalization” (Bach and Chizat, arXiv 2021) and the references therein.

Event Weekly OSI Lab Seminar Short summary In this seminar, I will talk about the paper “Landscape and training regimes in deep learning” (Geiger et al., 2021), accepted for publication in the Physics Report.