Entropic variants of SGD

Event

Weekly OSI Lab Seminar

Short summary

In this seminar, I will talk about a line of works that propose a new definition of “flatness” for loss landscapes in terms of entropy, and new variants of (S)GD based on the new definitions.

Papers

Papers discussed in the seminar:

  • Pratik Chaudhari, Anna Choromanska, Stefano Soatto, Yann LeCun, Carlo Baldassi, Christian Borgs, Jennifer Chayes, Levent Sagun, and Riccardo Zecchina. Entropy-SGD: biasing gradient descent into wide valleys. In Journal of Statistical Mechanics: Theory and Experiment 2019(12):124018, 2019.
  • Fabrizio Pittorino, Carlo Lucibello, Christoph Feinauer, Gabriele Perugini, Carlo Baldassi, Elizaveta Demyanenko, and Riccardo Zecchina. Entropic gradient descent algorithms and wide flat minima. In Journal of Statistical Mechanics: Theory and Experiment 2021(12):124015, 2021.
Previous
Next