Introduction to Gaussian Process Regression

Wednesday, July 29, 2015 - 8:30am - 12:00pm
DLR Room 131
Ilias Bilionis (Purdue University)
Research groups, national laboratories, and R&D corporate divisions spend years of development of sophisticated software in order to simulate realistically important physical phenomena. However, carrying out tasks like uncertainty quantification, model calibration or design using the fully-fledged model is -in all but the simplest cases- a daunting task, since a single simulation might take days or even weeks to complete, even with state-of-the-art modern computing systems. One, then, has to resort to computationally inexpensive surrogates of the computer code. The idea is to run the solver on a small, well-selected set of inputs and then use these data to learn the response surface. The surrogate surface may be subsequently used to carry out any of the computationally intensive engineering tasks.

In this part of the workshop, we provide an introduction to a very successful surrogate building tool known as Gaussian process regression (GPR). The two key characteristics of GPR are that it is non-parametric and Bayesian. GPR’s Bayesian nature allows for the quantification of the epistemic uncertainty induced by limited simulations, i.e., it can provide credibility intervals for its predictions. This uncertainty can be subsequently exploited in order to drive the selection of new simulations in an informative manner. The non-parametric nature provides extreme representational flexibility which can be further enhanced by exploiting prior knowledge about the simulator, e.g., smoothness, invariance principles, availability of coarse models, etc. The lecture assumes only a minimal understanding of the multivariate Gaussian distribution and basic probability theory. The topics we cover are: Gaussian process as a generalization of the multivariate Gaussian distribution, covariance functions as representations of prior knowledge, the predictive distribution of GPR, training the parameters of a covariance function via likelihood maximization, training the parameters of a covariance function via Markov-Chain-Monte-Carlo (MCMC) sampling of the posterior, dealing with many outputs, dealing with the curse of dimensionality, on the informative selection of the simulations, applications to uncertainty quantification.