Gaussian density parametrization flow: Particle and stochastic approaches

Bayesian inference is intractable for most practical problems and requires approximation schemes with several trade-offs. Variational inference provides one of such approximations which, while powerful, has thus far seen limited use in high-dimensional applications due to its complexity and computational cost. This paper introduces a scalable, theoretically grounded, and simple-to-implement algorithm for approximate inference with a variational Gaussian distribution. Specifically, we establish a practical particle-based algorithm to perform variational Gaussian inference that scales linearly in the problem dimensionality. We show that our approach performs on par with the state of the art on a set of challenging high-dimensional problems.