CSE Colloquium: Incentivized Alignment for Strategic Agents (Human and Otherwise)

Presenter: Grant Schoenebeck, University of Michigan

Abstract: Advances in machine learning enable new forms of human-AI collaboration, but collaborative settings typically involve agents with divergent objectives and private information. This will become increasingly critical in the emerging world of agentic AI, where ML-powered agents act on behalf of individuals or institutions with conflicting goals. I use the term incentivized alignment to describe the approach of combining both machine learning and incentive design to achieve alignment of system outcomes despite misaligned agents. This talk presents two case studies of incentivized alignment showing how machine learning can make mechanism design scalable and practical, and how mechanism design can make machine learning strategically robust. First, I examine the use of LLMs as judges for rating subjective responses. While LLMs perform well on existing datasets, they are highly susceptible to manipulation. I propose adapting peer-prediction mechanisms to create strategically-robust scoring mechanisms that incentivize honest reporting. Beyond ensuring high-quality inputs to AI systems, these mechanisms can potentially eliminate reward hacking in ML training pipelines. Second, I consider collective decision-making where agents hold different objectives and private information. The goal is to design mechanisms that incentivize strategic agents to select outcomes that would be optimal under full information sharing, according to certain criteria. Both case studies demonstrate solutions for incentivized alignment in multi-agent systems employing the combination of incentive design and machine learning, a theme likely to be central to the future of collaborative AI.

Bio: Grant Schoenebeck is an associate professor at the University of Michigan in the School of Information. His work has recently focused on developing and analyzing systems for eliciting and aggregating information from a diverse group of agents with varying information, interests, and abilities by combining ideas from machine learning and economics (e.g. game theory, mechanism design, and information design). More generally, his recent work has been about incentives and (machine) learning in a variety of contexts. His research is supported by multiple NSF grants including a CAREER award and spans publications in top venues including NeurIPS, ICLR, EC, WINE, the Web Conference, STOC, and FOCS. His former PhD students and postdocs now hold tenure-track positions at the University of Illinois Urbana-Champaign, Peking University, George Mason University, and Shanghai Jiao Tong University. He recently served as Program Committee Co-chair for WINE, Theory Track Co-chair for EC, and Economics and Computation Track co-chair at the Web Conference. Grant received his PhD at UC Berkeley, studied theology at Oxford University, and received his BA in mathematics and computer science from Harvard.

Hosted by: Professor Nikos Tziavelis

Location: Engineering 2, Room E2-180

*Light refreshments such as coffee, pastries, and fruit will be available.

Zoom: https://ucsc.zoom.us/j/93445911992?pwd=YkJ2TQtF79h0PcNXbEcpZLbpK0coiY.1&jst=3