Siu, V. (CS) – Diagnosing and Improving LLM Agents

Language model agents act on behalf of users and can fail with significant consequences. The proposed dissertation studies two sources of information that control an agent’s decisions: the context that an agent receives and the model’s internal representations. First, authorization context determines whether an agent should act, and current models enforce it unreliably. Given full authorization context, frontier models still refuse more than a third of authorized requests, a failure that only the model can fix. Second, compact directions in activation space control agent behavior. A refusal direction found in chat models removes refusal from agents, and a second direction controls how often agents call tools. The proposed work applies both results to oversight between agents, and it uses probes on the policy’s activations as critics in reinforcement learning for agents.

Event Host: Vincent Siu, PhD Student, Computer Science

Advisor: Chenguang Wang

Zoom: https://ucsc.zoom.us/j/97764742933?pwd=aYcn5Z5jFzfTWPfayaHyQqfPA1ovXw.1&jst=3