AM Seminar: Science in the Age of Foundation Models

Presenter: Dr. Danielle Robinson, AWS AI

Description: In this talk, I will discuss the large impact of foundation models within the sciences with a particular focus on the importance of physical constraints and uncertainty quantification. First, I will detail our novel ProbConserv framework for enforcing hard constraints within black-box deep learning models. ProbConserv provides uncertainty quantification, and can be used to enforce conservation law constraints as well as other nonlinear constraints. Next, I will discuss its extensions to ensembles of Neural Operators and out-of-distribution (OOD) estimations, as well as how it can be used in constrained generative modeling of PDEs. I will then show applications of our work in computational fluid dynamics (CFD), including weather forecasting, aerodynamics and chaotic systems. Lastly, I will conclude with a forward-looking view of the next steps for designing a physics foundation model that can be applied across various types of flows, geometries and boundary conditions, and what is needed for such a model to be developed.

Bio: Danielle Maddix Robinson is a Senior Applied Scientist in the Machine Learning Forecasting Group within AWS AI. She graduated with her PhD in Computational and Mathematical Engineering from the Institute of Computational and Mathematical Engineering (ICME) at Stanford University. She was advised by Professor Margot Gerritsen and developed robust numerical methods to remove spurious temporal oscillations in the degenerate nonlinear Generalized Porous Medium Equation. She is passionate about the underlying numerical analysis, linear algebra and optimization methods behind numerical PDEs and applying these techniques to deep learning. During her PhD, she also did an internship at NVIDIA with Joe Eaton and Alex Fender, and implemented an efficient and load-balanced sparse matrix vector multiplication (spmv) in cuSPARSE and nvGRAPH libraries. She is excited to be back at NVIDIA today. After graduating, Danielle joined AWS in 2018, and has been working on developing statistical and deep learning foundation models for time series forecasting including Chronos. Over the last several years, she has been leading the research initiative on developing models for physics-constrained machine learning for scientific computing on the DeepEarth team. In particular, she has researched how to apply ideas from numerical methods, e.g., finite volume schemes, to improve the accuracy of black-box ML models for PDEs with applications to ocean and climate models, aerodynamics and chaotic systems.

Link: https://ucsc.zoom.us/j/96136632376?pwd=yb27lop8mnhnsairAPgezmVJZzFb74.1.