AeroAstro-CSE PhD Thesis Defense: Joanna Zou

Abstract:

Stochastic differential equations (SDEs) are canonical models for many stochastic dynamical systems, describing processes ranging from particle diffusion in physics to Langevin algorithms for sampling. Numerical simulations of SDEs are widely used to predict complex phenomena, yet many such phenomena occur at scales beyond what can be feasibly simulated, due to both the high cost of evaluating high-fidelity, first-principles models and the intrinsic stochasticity of the process. To address this limitation, there is considerable interest in developing data-driven surrogate models for SDEs which emulate the high-fidelity process at substantially reduced cost.

This thesis introduces novel methodologies for learning data-driven models of nonlinear stochastic dynamical systems governed by SDEs. Our methods are goal-oriented in the sense that they target accuracy in a specific observable of the system that is otherwise cost-prohibitive to quantify with high fidelity. We focus on path-space observables that take the form of statistics of functionals of the stochastic process, corresponding to transient or non-equilibrium properties of the system.

First, for the core problem of learning SDE models from data, we derive goal-oriented loss functions that provide error guarantees for path-space observables. We show that goal-oriented learning can be a cost-effective means of predicting observables with controlled accuracy even though the true value of the observable is unknown. Our approach is to utilize an information-theoretic error bound on the observable for variational learning of the drift function of the SDE, where minimizing the tractable upper bound is a proxy for minimizing the intractable error in the observable.

Next, we develop an active learning algorithm which adaptively acquires training data for refining models of ergodic stochastic dynamical systems. To accelerate the sampling of underrepresented regions of state space, we utilize an interacting particle system whose mean field limit corresponds to a gradient flow of the Kullback–Leibler divergence in the Stein geometry induced by a chosen kernel. We introduce an adaptive stopping criterion which enables online data acquisition by using Stein’s identity as an indicator of information gain.

Finally, we introduce a technique for the goal-oriented dimension reduction of SDE drift functions defined in infinite-dimensional Hilbert spaces, which enables tractable uncertainty quantification for path-space observables and suggests effective low-dimensional parameterizations of the SDE model. We show that the active subspace, derived from a sensitivity operator based on the Fréchet derivative of the observable, identifies principal features of the drift function that provide more accurate low-dimensional approximations of the observable compared to standard dimension reduction techniques.

Together, the contributions of this thesis comprise a theoretically principled framework to learning models of stochastic dynamical systems which capture observable properties of interest and enable computational simulations at larger scales and complexities.

Thesis Committee Members:

• Professor Youssef M. Marzouk, Breene M. Kerr (1951) Professor, Department of Aeronautics & Astronautics, MIT (Chair)
• Professor Han Cheng Lie, Junior Professor, Institute of Mathematics, University of Potsdam
• Professor Themistoklis Sapsis, William I. Koch Professor, Department of Mechanical Engineering, MIT
• Professor Markos Katsoulakis, Director of the Center of Applied Mathematics, Department of Mathematics and Statistics, University of Massachusetts Amherst (Reader)
• Dr. Dallas Foster, Senior Deep Learning and Software Engineer, NVIDIA (Reader)