Smoothing for exponential family dynamical systems

TL;DR — We develop approximate Bayesian smoothing algorithms for dynamical systems with exponential family distribution stochastic transitions.

Abstract

State-space modeling is a powerful technique for the analysis of spatiotemporal structures of time series. However, when assumptions about linearity or Gaussianity are violated, statistical inference about the latent process is challenging. While variational inference can be used to approximate the posterior in these nonlinear or non-Gaussian settings, it is desirable to preserve the temporal structure of the true posterior in the variational approximation, while ensuring inference scales linearly in sequence length. We propose a new structured variational approximation that satisfies these desiderata. Furthermore, by generalizing to *exponential family dynamical systems*, we are able to develop decoupled second order inference algorithms that have simple updates, without increased computational complexity. Then, we extend our insights and develop the *auto-encoding backward factorized smoother*, making it easy to leverage modern deep learning tools. For settings where a sequential inference algorithm may be more appropriate, we also present the *variational Bryson-Frazier* algorithm, by developing a new backward smoothing objective. We compare against various inference algorithms for state-space models, and validate the theory presented through numerical experiments.