How priors of initial hyperparameters affect Gaussian process regression models

Zexun Chen*, Bo Wang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

The hyperparameters in Gaussian process regression (GPR) model with a specified kernel are often estimated from the data via the maximum marginal likelihood. Due to the non-convexity of marginal likelihood with respect to the hyperparameters, the optimisation may not converge to the global maxima. A common approach to tackle this issue is to use multiple starting points randomly selected from a specific prior distribution. As a result the choice of prior distribution may play a vital role in the predictability of this approach. However, there exists little research in the literature to study the impact of the prior distributions on the hyperparameter estimation and the performance of GPR. In this paper, we provide the first empirical study on this problem using simulated and real data experiments. We consider different types of priors for the initial values of hyperparameters for some commonly used kernels and investigate the influence of the priors on the predictability of GPR models. The results reveal that, once a kernel is chosen, different priors for the initial hyperparameters have no significant impact on the performance of GPR prediction, despite that the estimates of the hyperparameters are very different to the true values in some cases.

Original languageEnglish
Pages (from-to)1702-1710
Number of pages9
JournalNeurocomputing
Volume275
Early online date1 Nov 2017
DOIs
Publication statusPublished - 31 Jan 2018

Keywords / Materials (for Non-textual outputs)

  • Gaussian process regression
  • Hyperparameters
  • Kernel selection
  • Maximum marginal likelihood
  • Prior distribution

Fingerprint

Dive into the research topics of 'How priors of initial hyperparameters affect Gaussian process regression models'. Together they form a unique fingerprint.

Cite this