Kembay, A. (ECE) – Sparse and Continual Foundations for Adaptive General Intelligence

While the human brain learns continually, mastering new tasks without forgetting
the old and adapting to unfamiliar ones from context alone, modern neural networks
still lack both. To bridge the gap between biological adaptivity and modern AI, we
have established foundational work on sparsity as a computational principle at three
levels of neural computation, through salient feature masking that distills only the most
informative knowledge from a teacher, quantized spiking neural networks whose sparse
activations mitigate catastrophic forgetting by updating weights only when new learn-
ing requires it, and complex-pole value-path dynamics that give Transformer attention
a resonant, positionally selective memory. Addressing the remaining bottleneck, that
these sparse structures are fixed in advance rather than adapted to the task at hand,
we propose a research roadmap centered on in-context meta-learning with sparse atten-
tion priors, enabling models to ‘learn to be sparse’ by inferring task-relevant structure
from context alone, without any weight update. Taken together, this research seeks
to unify brain-inspired sparsity with continual and in-context learning as a foundation
for adaptive general intelligence.

Event Host: Assel Kembay, Ph.D. Student, Electrical & Computer Engineering

Advisor: Jason Eshraghian

Zoom: https://ucsc.zoom.us/j/92202931005?pwd=peVIc4e03fUPwFqlGa6yWx6ZlL33lI.1

Passcode: 742766