BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Events - ECPv6.16.5.1//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-ORIGINAL-URL:https://events.ucsc.edu
X-WR-CALDESC:Events for Events
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20250309T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20251102T090000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20260308T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20261101T090000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20270314T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20271107T090000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/Los_Angeles:20260724T140000
DTEND;TZID=America/Los_Angeles:20260724T160000
DTSTAMP:20260716T222234Z
CREATED:20260716T222234Z
LAST-MODIFIED:20260716T222234Z
UID:10015099-1784901600-1784908800@events.ucsc.edu
SUMMARY:Gholami\, K. (ECE) - Efficient Language Model Construction and Inference via Sparsity
DESCRIPTION:While large language models can match or exceed human performance\, they do so with memory and energy costs orders of magnitude greater than biological cognition. We investigate sparsity as a brain-inspired computational principle to address both. We first establish a framework for evaluating small language model construction methods\, using the next-token logit distribution as a behavioral fingerprint. Then\, we introduce a semi-structured correlation-aware weight sparsity (CWS) method that uses the full activation covariance to identify and prune correlated weights whose combined removal cost is lower than any individual score predicts. CWS\, improves perplexity over existing criteria up to 70% sparsity. To extend this gain to extreme sparsity\, we propose a hierarchical ADMM framework that optimizes pruning directly against cross-entropy and distillation loss\, first layer-wise for efficiency and then globally for cross-layer coordination. This research establishes brain-inspired principles as a foundation for efficient language models that remain accurate even under extreme compression. \nEvent Host: Kimia Gholami\, Ph.D. Student\, Electrical & Computer Engineering \nAdvisor: Jason Eshraghian \nZoom: https://ucsc.zoom.us/j/9827512398?pwd=SGpDWGtVVG81dkgyTHhjbG81dEVUZz09&omn=98349793611 \nPasscode: 8398
URL:https://events.ucsc.edu/event/gholami-k-ece-efficient-language-model-construction-and-inference-via-sparsity/
LOCATION:Engineering 2\, Engineering 2 1156 High Street\, Santa Cruz\, CA\, 95064
CATEGORIES:Ph.D. Presentations
ATTACH;FMTTYPE=image/png:https://events.ucsc.edu/wp-content/uploads/2026/04/ph.d.-presentation-graphic-option-3.png
GEO:37.0009723;-122.0632371
X-APPLE-STRUCTURED-LOCATION;VALUE=URI;X-ADDRESS=Engineering 2 Engineering 2 1156 High Street Santa Cruz CA 95064;X-APPLE-RADIUS=500;X-TITLE=Engineering 2 1156 High Street:geo:-122.0632371,37.0009723
END:VEVENT
END:VCALENDAR