Zhu, R. (ECE) – From Neuromorphic Principles to Efficient Neural Language Architectures

While Large Language Models exhibit remarkable capabilities, their reliance on the standard Transformer architecture imposes prohibitive computational costs and quadratic memory complexity. To bridge the gap between biological efficiency and high-performance AI, we have established foundational work in linearizing attention and maximizing hardware utilization through architectures such as RWKV and MatMul-Free networks. Addressing the remaining bottlenecks in long-term memory consolidation and optimization stability, we propose a research roadmap focused on “In-Place Test-Time Training” (TTT) to enable compositional memory via dynamic weight updates, and the Muon optimizer to stabilize deep reasoning through orthogonal gradient updates. Ultimately, this work aims to unify neuromorphic principles with scalable deep learning to enable robust performance in resource-efficient environments.

Event Host: Ridger Zhu, Ph.D. Student, Electrical and Computer Engineering 

Advisor: Jason Eshraghian

Zoom- https://ucsc.zoom.us/j/95241268060?pwd=WDMgDWhhSyXNh8NZpBDvgpbcMVbvUz.1

Passcode- 256794