Doctoral thesis

Contributions to scalable gaussian processes

Show more…
  • 2022

PhD: Università della Svizzera italiana

English This thesis provides novel contributions to scalable Gaussian processes (GPs), which constitute an important tool in machine learning and statistics with applications ranging from social and natural science through engineering. GPs are powerful probabilistic methods with many benefits, such as their modeling flexibility, the robustness to overfitting and the availability of well-calibrated predictive uncertainty estimates. However, off-the-shelf GP inference procedures are limited to datasets with several thousand training data points because of their cubic computational complexity. This thesis presents new methodologies and novel algorithms for scaling GP regression to larger datasets by employing sequential and local methods. In particular, the first contribution of this work is a unifying GP approximation method based on a recursive formulation, which enables to train analytically a range of existing GP models in an online and distributed way. In this formulation, the so-called hyperparameters, which refer to a few parameters determining the GP, are assumed to be known. On the other hand, those can be learned by the second major contribution consisting of two novel algorithms for sequential hyperparameters estimation. These allow to scale the training of GPs up to millions of training samples. The last contribution involves a novel unifying GP approximation model exploiting sparsity and locality. Specifically, a method based on local GPs, which can share common information with a flexible correlation structure, is proposed. Thereby, this new model unifies several existing local and global GP approximation approaches. All the proposed methods in this thesis are theoretically supported and empirically tested on synthetic as well as real-world datasets with up to millions of training samples. Thereby, these new methods outperform the state-of-the-art in several tasks. This demonstrates the effectiveness of the novel GP approximations proposed in this thesis, which can achieve high-scalability without sacrifying the performance of original GPs. Therefore, this work substantially contributes to overcome the computational complexity barrier for the large-scale adoption of GPs.
Collections
Language
  • English
Classification
Computer science
License
License undefined
Open access status
green
Identifiers
  • NDP-USI 2022INF007
Persistent URL
https://susi.usi.ch/usi/documents/321105
Statistics

Document views: 8 File downloads:
  • 2022INF007.pdf: 3