Statistics Seminar: Evaluating Predictive Algorithms Under Missing Data

Presenter: Amanda Coston, Assistant Professor, University of California Berkeley

Description: Performance evaluation plays a central role in decisions about whether and how predictive algorithms should be deployed in high-stakes settings. Yet, in many real-world domains, evaluation is fundamentally difficult: the data available for assessment are often biased, incomplete, or noisy, and the act of deploying a model can itself alter which outcomes are observed. As a result, standard evaluation practices may substantially misrepresent both overall model performance and disparities across groups. In this talk, we examine several common threats to valid evaluation—including measurement error, selection bias, and distribution shift—and present principled evaluation methods that enable valid performance assessment under these challenges when appropriate conditions are met.

Bio: From UC Berkeley website: Amanda Coston is an assistant professor of statistics at UC Berkeley. Her research addresses real-world data problems that challenge the validity, reliability, and equity of algorithmic decision support systems and data-driven policy-making. Her work draws on techniques from causal inference, machine learning, and nonparametric statistics. She earned her PhD in machine learning and public policy at Carnegie Mellon University and subsequently completed a postdoc at Microsoft Research on the Machine Learning and Statistics Team. She also holds a Bachelor of Science in Engineering from Princeton in computer science and a certificate in the Princeton School of Public and International Affairs.

Hosted by: Statistics Department