CSE Colloquium: Enabling scalable GPU computing via efficient virtual memory systems

Presenter: Hyeran Jeon, UC Merced

Title: Enabling scalable GPU computing via efficient virtual memory systems

Abstract: GPUs have become one of the most important accelerators of various emerging workloads. While the massive parallelism makes the GPUs one of the most favorable compute engines, the limited on-device memory capacity hinders their wider adoption. Virtual memory systems have been helpful in transparently expanding memory space to host CPU memory and peer GPU memories, as GPUs can access data through simple pointer sharing regardless of the physical location of that data. However, as GPU architecture evolves and the volume of data increases significantly, the virtual memory systems themselves become one of the critical performance bottlenecks. This talk will explore the perils and opportunities of virtual memory systems under emerging GPU architectures.
The first part of the talk will show that the limited IOMMU bandwidth is one of the primary performance limiters in multi-chip-module GPUs (MCM-GPUs) due to the increasing concurrency of multiple GPU chiplets. To fundamentally reduce the translation demands, the talk will introduce a new way of page mapping that effectively replaces page-table-based virtual memory translations with simple calculation-based translations. The second part of the talk will focus on GPUs that have internal GMMUs. The talk will first show that naive GMMU translation bandwidth increase will incur significant area overhead. Then, we show that one of the existing but highly underutilized accelerators integrated in the GPU die, the ray tracing cores, can be repurposed for virtual memory translation.

Bio: Hyeran Jeon is an associate professor at the University of California, Merced. Hyeran's main research interests lie in high-performance, energy-efficient, and robust computer architecture and systems. Her research lab (MoCA Lab) has been sponsored by the government and industry, including National Science Foundation, California Energy Commission, Xilinx, NVIDIA, and Ampere Computing. She earned her Ph.D. at the University of Southern California. She has industry experience as an intern at IBM T.J. Watson Research Center and AMD Research, and as a systems software engineer at Samsung Electronics. She is a recipient of the NSF CAREER award and the USC Viterbi dean’s doctoral fellowship.

Hosted by: Professor Mohsen Lesani

Location: Engineering 2, E2-180
*Refreshments such as coffee and pastries will be provided.