Chen, Z. (CSE) – GPU Subgroup Semantics for Portable High-Performance Kernels

Modern high-performance GPU kernels increasingly rely on subgroup-level execution, including subgroup-level communication, subgroup operations, and matrix operations. These features are essential for workloads such as matrix multiplication and FlashAttention, but their language-level guarantees remain difficult to reason about. Existing programming models often leave unclear which threads participate in subgroup operations, when subgroup threads are required to execute together, and what synchronization is implied by subgroup-level operations. This ambiguity becomes especially important in portable GPU programming, where the same kernel may run across devices with different subgroup sizes, compiler stacks, browser backends, and hardware execution behavior.

My research studies how precise subgroup semantics can support portable and correct high-performance GPU kernels. SIMT-Step, my main completed work, develops a formal and flexible operational semantics for GPU subgroup execution. It introduces dynamic blocks to specify converged subgroup execution and subgroup-operation participation, classifies instructions as independent, synchronous, or collective to express a spectrum of candidate subgroup semantics, and validates these models through a TLA+ implementation and an empirical fuzzing study across real GPUs. My systems work studies how subgroup-dependent kernels behave in practice, including WebGPU FlashAttention kernels for LLM inference, tunable WebGPU kernels for performance portability, and Vulkan-based execution for heterogeneous SoCs. Building on these foundations, my proposed verification work develops data-race-free checking techniques for ML kernels that rely on subgroup operations and matrix operations. Together, these projects aim to clarify the execution guarantees that optimized GPU kernels can rely on and to support portable GPU programming systems whose performance and correctness can be reasoned about across diverse hardware.

Event Host: Zheyuan Chen, Ph.D. Student, Computer Science & Engineering

Advisor: Tyler Sorensen

Zoom: https://ucsc.zoom.us/j/92175288480?pwd=jGajtqerVbKuW1FPNr3awqOYoxATsp.1&jst=3

Passcode: 693354