Scalable Gaussian Processes with Kronecker Methods and Random Fourier Features

Abstract

Gaussian processes (GPs) are a flexible class of methods with state of the art performance on spatial statistics applications. However, GPs require O(n^3) computations and O(n^2) storage, and popular GP kernels are typically limited to smoothing and interpolation. To address these difficulties, Kronecker methods have been used to exploit structure in the GP covariance matrix for scalability, while allowing for expressive kernel learning (Wilson et al., 2014). However, fast Kronecker methods have been confined to Gaussian likelihoods. We propose new scalable Kronecker methods for Gaussian processes with non-Gaussian likelihoods, using a Laplace approximation which involves linear conjugate gradients for inference, and a lower bound on the GP marginal likelihood for kernel learning. Our approach has near linear scaling, requiring O(D n^((D+1)/D) ) operations and O(D n^(2 / D) ) storage, for n training data-points on a dense D > 1 dimensional grid. Moreover, we introduce a log Gaussian Cox process, with highly expressive kernels, for modelling spatiotemporal count processes, and apply it to a point pattern (n = 233,088) of a decade of crime events in Chicago. Using our model, we discover spatially varying multiscale seasonal trends and produce highly accurate long-range local area forecasts. Time permitting, I will discuss in-progress extensions to this work, using random Fourier features, with the goal of forecasting a marked point process dataset consisting of crimes of different types. Link: http://jmlr.org/proceedings/papers/v37/flaxman15.pdf

Short Bio

Seth Flaxman (www.sethrf.com) is a postdoc with Yee Whye Teh at Oxford in the computational statistics and machine learning group in the Department of Statistics. His research is on scalable methods for spatiotemporal statistics and Bayesian machine learning, applied to public policy / social science areas including crime, emotion, and public health. Seth completed his BA in computer science and mathematics at Harvard in 2008 and his PhD in machine learning andpublic policy at Carnegie Mellon University in 2015, advised by Daniel Neill and Alex Smola.

Venue

Large Conference Room, O'Reilly Institute