Optimization

Event OSI Lab Conference Week (NeurIPS 2020) Short summary In this seminar titled Topics not covered by anyone… (kind of), I will talk about 4 NeurIPS 2020 papers (all of which are theory-heavy) that I thought were interesting, whose topics were not covered by anyone in OSI Lab.

Event Weekly OSI Lab Seminar Short summary This seminar continues on from Part 1, focusing more on the implication of such heavy-tailed theories of SGD to generalization capability of neural nets, and the origin of the heavy-tailedness.

Event Weekly OSI Lab Seminar Short summary In this seminar, I will talk about a recent line of works that propose to analyze SGD under heavy-tail noise assumptions. Abstract One of the popular ways of analyzing the behavior of SGD and SGDm(SGD with momentum) is by considering it as a discretization of Langevin-type SDE.

Although gradient descent with Polyak’s momentum is widely used in modern machine and deep learning, a concrete understanding of …

Proposes a framework for performing fair PCA in memory limited, streaming setting. Sample complexity results and empirical discussions show the superiority of our approach compared to the existing approaches.

Proposes a new MMD-based definition of fairness for PCA, then formulate fair PCA as an optimization over the Stiefel manifold. Various theoretical and empirical discussions show the superiority of our approach compared to the existing approach (Olfat & Aswani, AAAI'19).

specific topics tbd

Part of fair representation learning. Develop a theory of fairness in dimensionality reduction: new definition, new (efficient) algorithm, new theoretical results. Currently focused on PCA.