Zhu, R. (ECE) – From Neuromorphic Principles to Efficient Neural Language Architectures
Hybrid Event
While Large Language Models exhibit remarkable capabilities, their reliance on the standard Transformer architecture imposes prohibitive computational costs and quadratic memory complexity. To bridge the gap between biological efficiency and high-performance AI, we have established foundational work in linearizing attention and maximizing hardware utilization through architectures such as RWKV and MatMul-Free networks. Addressing the remaining […]