Holmes, J. (CM) – Towards a Multi-dimensional Model of User Load

Games user researchers (GURs) use various methods to understand when a game is overloading its players. In games research where data-driven multimodal approaches are necessary to drive insights, the currently available tools to measure user load are coarse, one-dimensional, and often aggregated. The more dominant instruments, such as the Cognitive Load Scale (CLS) and the NASA-TLX, rely on player reflections of mental effort, primarily at the end of the playtest session, to distinguish different cognitive load types. This makes it difficult to: (1) understand where specifically players are struggling and experiencing high load, especially at the non-reflective subconscious level, (2) identify where that load is primarily coming from (e.g., perceptual clutter or difficulty/skill imbalance), and (3) examine user overload at scale, a crucial component of designing a game with large player bases. Telemetry is the behavioral record of what players are doing from moment to moment, in varying degrees of granularity. Telemetry has served as a powerful tool to understand player behaviors at scale, yet is rarely used to measure user load, especially through a validated multidimensional framework. This dissertation proposes that behavioral signatures of specific load constructs are observable in game telemetry, and a model built and validated on such telemetry can measure the distinct components of each load at the moment-to-moment granularity of individual play, as opposed to aggregated magnitude. This dissertation consists of three parts: (1) Validation through construct manipulation and reference measurement. Specifically, manipulating theoretically grounded load constructs and confirming that the proposed telemetry features respond as predicted, relative to the established measurements collected alongside them (e.g., NASA-TLX, pupillometry, secondary-task). (2) Individual-level validation through rigorous longitudinal examination of the same players repeatedly across many sessions such that load constructs can be tracked at the within-person granularity. This is necessary to establish that the measure works for an individual player and not just for population averages. (3) Test the user load model by applying it to naturalistic game telemetry. Additionally, this phase will entail the development of an insight-oriented measurement tool for GURs based on our validated user load model. The overarching contribution is a behavioral, telemetry-based method for measuring multidimensional user load in games, validated to measure load within each person (individual-level). This gives GURs a scalable tool and replicable process for detecting user load in commercial game telemetry.

Event Host: Jonattan Holmes, Ph.D. Student, Computational Media

Advisor: Magy Seif El-Nasr

Zoom: https://ucsc.zoom.us/j/98245962806?pwd=HnkwPMFSamQJFrE5aihbZbKDBbt4s9.1

Passcode: 347521