Talk abstract:

Variational Methods for Inference

Tommi Jaakkola
Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology

Variational methods have a long history as principled approximations in physics, statistics, and other fields. Techniques such as mean field approximation and finite element methods are naturally viewed as variational methods. The basic idea underlying these methods is a transformation from the problem of interest such as computation of marginal probabilities in factor graphs to a manageable optimization problem. The objective function used in the resulting optimization problem relates (monotonically) to the estimation accuracy of the desired quantities (marginal probabilities) yielding e.g. upper and lower bounds. The purpose of this tutorial talk is to introduce a class of variational methods and demonstrate their use in probabilistic inference calculations in factor graphs. We show in particular how these methods can be readily combined with exact inference algorithms to maximally exploit any feasible substructures in the graphs. Numerical examples come from a large scale inference problem in medical diagnosis.

Back to Codes, Systems and Graphical Models