Zhang, R. (CSE) – Towards next-generation infrastructure for agentic AI

LLM workloads are shifting from chat to agents: one task becomes tens of model calls, dispatched along a graph the harness holds and repeating the same instructions. This execution is structured and predictable, yet the serving stack discards that structure at every boundary it crosses. I argue that agent infrastructure should treat execution semantics and state as first-class abstractions. My completed work recovers them between harness and engine: a policy-driven runtime layer and CacheSage, which predicts when each agent returns, cutting time to first token by up to 45%. Proposed work extends this across machines, shared retrieval, and reinforcement learning.

Event Host: Rui Zhang, PhD Student, Computer Science & Engineering 

Advisor: Liting Hu 

Zoom: https://ucsc.zoom.us/j/4646537931?pwd=aJHaERqBaST4DTj2SMJg22HgFeEjuN.1#success