Introduction to Bayesian ML/DL, with Application to Parameter Inference of Coupled Non-linear ODEs - Part 1

Apr 29, 2021 · 2 min read
seminars

Event

BIMAG Journal Club

Short summary

In this journal club session, I will cover the basics of Gaussian process(GP) and some recent advances in its application to parameter inference of coupled non-linear ODEs.

Abstract

Gaussian process(GP) is a stochastic process such that the joint distribution of arbitrary finite subset of the random variables is a multivariate normal. It plays a fundamental role in Bayesian machine learning as it can be interpreted as a prior over functions (Rasmussen and Williams, 2006), hence providing a nonparametric approach to various tasks.

In the first part, I will introduce the general framework of GP and some underlying theory, accompanied by an illustrative example of GP regression, also known as Kringing. In the second part, I will introduce some recent works on applying GP to parameter inference of coupled non-linear ODEs arising in various biological contexts.

Papers

References for GP:

  • Gaussian Processes for Machine Learning (C. E. Rasmussen and C. K. I. Williams, the MIT press, 2006)

References for GP-based ODE Parameter Inference:

  • Yang, S., Wong, S. W. K., Kou, S.C. (2021), “Inference of dynamic systems from noisy and sparse data via manifold-constrained Gaussian processes,” Proceedings of the National Academy of Sciences of the United States of America (PNAS), 118(15).
  • Wenk, P., Gotovos, Al., Bauer, S., Gorbach, N. S., Krause, A., Buhmann, J. M. (2019), “Fast Gaussian process based gradient matching for parameter identification in systems of nonlinear ODEs,” Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), 89, 1351-1360.
  • Gorbach, N. S., Bauer, S., Buhmann, J. M. (2017), “Scalable Variational Inference for Dynamical Systems,” Advances in Neural Information Processing Systems (NIPS), 30.
  • Macdonald, B., Higham, C., Husmeier, D. (2015), “Controversy in mechanistic modelling with Gaussian processes,” Proceedings of the 32nd International Conference on Machine Learning (ICML).
  • Wang, Y., Barber, D. (2014), “Gaussian Processes for Bayesian Estimation in Ordinary Differential Equations,” Proceedings of the 31st International Conference on Machine Learning (ICML).
  • Dondelinger, F., Filippone, M., Rogers, S., Husmeler, D. (2013), “ODE parameter inference using adaptive gradient matching with Gaussian processes,” Proceedings of the 16th International Conference on Artificial Intelligence and Statistics (AISTATS).
  • Calderhead, B., Girolami, M., Lawrence, N. (2008), “Accelerating Bayesian Inference over Nonlinear Differential Equations with Gaussian Processes,” Advances in Neural Information Processing Systems (NIPS), 21.
Junghyun Lee
Authors
Junghyun Lee
PhD Candidate in AI
PhD candidate at KAIST AI, jointly advised by Se-Young Yun and Chulhee Yun. I work on interactive machine learning, theoretical aspects of LLMs, learning/optimization theory, and statistical analysis of large networks.