Gholami, K. (ECE) – Efficient Language Model Construction and Inference via Sparsity
While large language models can match or exceed human performance, they do so with memory and energy costs orders of magnitude greater than biological cognition. We investigate sparsity as a brain-inspired computational principle to address both. We first establish a framework for evaluating small language model construction methods, using the next-token logit distribution as a […]