Burbano, L. (CS) – Security of autonomous decision-making agents: From control systems to embodied AI

Due to their increasing complexity, autonomous decision-making agents rely on increasingly advanced algorithms, from classical control theory to reinforcement learning (RL) and, more recently, large vision-language models. While these algorithms help automate the decision-making in complex systems, they bring newer attack vulnerabilities that an adversary can exploit. In this dissertation, we study the security of autonomous decision agents that use control systems, RL, and AI. We focus on the security of cyber-physical and autonomous cyber-defense systems. In particular, we study how an attacker can compromise decision-making agents.

For control systems, this dissertation studies the existence of backdoor attacks against control systems that rely on data and proposes a defense strategy against the sensors of control systems.

For reinforcement learning, we study the security of autonomous cyber-defense (ACD)) agents that automatically respond to attackers’ actions in a network. While previous works focus on creating agents, we study an adversary who compromises the agent’s own infrastructure, manipulating the information it observes to steer the network toward an attacker-chosen state. We also propose a defense strategy that focuses on determining if an attacker is compromising the ACD.

Finally, we study the security of embodied AI, where CPS rely on large vision-language models (LVLMs) for decision-making. We propose a novel attack that can cause an agent to make unsafe decisions by presenting a well-designed textual sign via the visual modality. While previous attacks against neural network-based algorithms rely on creating adversarial patches without semantic meaning, in this work, we exploit the fact that LVLMs can understand text.

Event Host: Luis Burbano, Ph.D. Candidate, Computer Science 

Advisor: Alvaro Cardenas

Zoom: https://ucsc.zoom.us/j/92373119649?pwd=BLFQMrGkOxJVXnjrJhXqudN1iciZAn.1

Passcode: 160434