Event
Weekly DL Theory & Stat Phy Seminar
Short summary
In this seminar, I will talk about a line of works that propose a new definition of “flatness” for loss landscapes in terms of entropy, and new variants of (S)GD based on the new definitions.
Papers
Papers discussed in the seminar:
- Pratik Chaudhari, Anna Choromanska, Stefano Soatto, Yann LeCun, Carlo Baldassi, Christian Borgs, Jennifer Chayes, Levent Sagun, and Riccardo Zecchina. Entropy-SGD: biasing gradient descent into wide valleys. In Journal of Statistical Mechanics: Theory and Experiment 2019(12):124018, 2019.
- Fabrizio Pittorino, Carlo Lucibello, Christoph Feinauer, Gabriele Perugini, Carlo Baldassi, Elizaveta Demyanenko, and Riccardo Zecchina. Entropic gradient descent algorithms and wide flat minima. In Journal of Statistical Mechanics: Theory and Experiment 2021(12):124015, 2021.