CSE Colloquium: Mitigating Data Scarcity via Simulation by Roozbeh Mottaghi

Presenter: Roozbeh Mottaghi, University of Washington

Abstract: Data has revolutionized progress across AI fields like natural language processing and computer vision. Yet, in robotics, data collection remains a significant challenge: robots must interact with complex, dynamic environments, making the process slow, costly, and difficult to scale. In this talk, I will discuss how simulation is transforming the landscape of robotics research by addressing these data bottlenecks. I will introduce Habitat 3.0, a 3D simulator designed for training and evaluating robotic agents in dynamic environments that include human interactions. Focusing on collaborative human-robot tasks, I will present PARTNR, a simulation benchmark designed to rigorously evaluate planning and reasoning in interactive settings. I will share key insights from this benchmark, revealing both the impressive capabilities of current LLMs and the significant challenges they encounter when faced with the complexities of real-world environments.

Bio: Roozbeh Mottaghi is a Senior Research Scientist Manager at FAIR and an Affiliate Associate Professor in Paul G. Allen School of Computer Science and Engineering at the University of Washington. Prior to joining FAIR, he was the Research Manager of the Perceptual Reasoning and Interaction Research (PRIOR) group at the Allen Institute for AI (AI2). He obtained his PhD in Computer Science in 2013 from the University of California, Los Angeles. After PhD, he joined the Computer Science Department at Stanford University as a post-doctoral researcher. His research mainly focuses on embodied AI, reasoning via perception, and learning via interaction, and his work on large-scale Embodied AI received the Outstanding Paper Award at NeurIPS 2022.