Event
BIMAG Journal Club
Short summary
In this journal club session, I will cover the basics of variational inference and and some recent advances in its application to parameter inference of coupled non-linear ODEs.
Abstract
The problem of approximating the posterior distribution (or density estimation in general) is a crucial problem in Bayesian statistics, in which intractable integrals often become the computational bottleneck. MCMC sampling is the most widely used family of algorithms for approximating posteriors. However, if the underlying graphical model is too complex or the data is in very high dimensions, then such sampling-based methodologies run into several problems. Variational inference (Jordan et al., 1999; Wainwright and Jordan, 2008) is a family of machine learning methodologies that transforms the problem of approximating posterior densities to an optimization, which lets us circumvent all such problems.
In the first part, I will introduce the general framework of variational inference and some underlying theory. In the second part, I will introduce some recent works on applying variational inference to parameter inference of coupled non-linear ODEs arising in various biological contexts.
Papers
References for Variational Inference:
- Blei, D. M., Kucukelbir, A., McAuliffe, J. D. (2017), “Variational Inference: A Review for Statisticians,” Journal of the American Statistical Association, 518, 859-877.
- Hoffman, M. D., Blei, D. M., Wang, C., Paisley, J. (2013), “Stochastic Variational Inference,” Journal of Machine Learning Research, 14, 1303-1347.
- Blei, D. M., Ng, A. Y., Jordan, M. I. (2003), “Latent Diriclet Allocation,” Journal of Machine Learning Research, 3, 993-1022.
- Wainwright, M. J., Jordan, M. I. (2008), “Graphical Models, Exponential Families, and Variational Inference,” Foundations and Trends in Machine Learning, 1(1-2), 1-305.
- Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., Saul, L. K. (1999), “An Introduction to Variational Methods for Graphical Models,” Machine Learning, 37, 183-233.
References for V.I.-based ODE Parameter Inference:
- Ghosh, S., Birrell, P. J., Angelis, D. D. (2021), “Variational inference for nonlinear ordinary differential equations,” Proceedings of the 24th International Conference on Artificial Intelligence and Statistics (AISTATS).
- Gorbach, N. S., Bauer, S., Buhmann, J. M. (2017), “Scalable Variational Inference for Dynamical Systems,” Advances in Neural Information Processing Systems (NIPS), 30.