Bayesian parameter estimation using amortized variational inference.

Abstract

Through the use of models, it is possible to express information about a process or data being analyzed. These models seek to explain or predict the process of interest. Models with a known and fixed number of parameters, known as parametric models, let us interpret more easily the phenomena being studied by incorporating assumptions directly in the modeling part through the parameters. The parameters of the model must be estimated to accomplish a specific task related to the data being used. One way of finding these parameters is by using the Bayesian approach, which provides information about the uncertainty of the model being used as well as including prior information into the learning process. Methods commonly used for Bayesian inference, such as Markov Chain Monte Carlo or the faster but approximate Variational Inference methods are not suited when the model is used multiple times on different sets of data. This is because they work on one set of observations at a time, which means that the same procedure must be repeated from scratch to find the parameters of the model that work best for the new data. Instead, we provide experiments on synthetic data that show how Amortized Variational Inference can be used to obtain results comparable to more precise methods while reducing the inference time for new observations by making use of global information learned via an inference network. Our results show that our approximate inference procedure can provide results similar to classic methods even in the presence of noise for simple models.