An Autoethnography on Visualization Literacy: A Wicked Measurement Problem

Abstract
We contribute an autoethnographic reflection on the complexity of defining and measuring visualization literacy (i.e., the ability to interpret and construct visualizations) to expose our tacit thoughts that often exist in-between polished works and remain unreported in individual research papers. Our work is inspired by the growing number of empirical studies in visualization research that rely on visualization literacy as a basis for developing effective data representations or educational interventions. Researchers have already made various efforts to assess this construct, yet it is often hard to pinpoint either what we want to measure or what we are effectively measuring. In this autoethnography, we gather insights from 14 internal interviews with researchers who are users or designers of visualization literacy tests. We aim to identify what makes visualization literacy assessment a ``wicked'' problem. We further reflect on the fluidity of visualization literacy and discuss how this property may lead to misalignment between what the construct is and how measurements of it are used or designed. We also examine potential threats to measurement validity from conceptual, operational, and methodological perspectives. Based on our experiences and reflections, we propose several calls to action aimed at tackling the wicked problem of visualization literacy measurement, such as by broadening test scopes and modalities, improving test ecological validity, making it easier to use tests, seeking interdisciplinary collaboration, and drawing from continued dialogue on visualization literacy to expect and be more comfortable with its fluidity.
Materials
PDF | Preprint | Supplement | BibTeX
Authors
Lily W. Ge
Anne-Flore Cabouat
Karen Bonilla
Yuan Cui
Yiren Ding
Noëlle Rakotondravony
Jasmine Tan Otto
Maryam Hedayati
Bum Chul Kwon
Angela Locoro
Petra Isenberg
Matthew Kay
Citation
Thumbnail image for publication titled: An Autoethnography on Visualization Literacy: A Wicked Measurement Problem
An Autoethnography on Visualization Literacy: A Wicked Measurement Problem

Lily W. Ge, Anne-Flore Cabouat, Karen Bonilla, Yuan Cui, Yiren Ding, Noëlle Rakotondravony, Mackenzie Michael Creamer, Jasmine Tan Otto, Maryam Hedayati, Bum Chul Kwon, Angela Locoro, Lane Harrison, Petra Isenberg, Michael Correll, and Matthew Kay. IEEE Transactions on Visualization and Computer Graphics—VIS/TVCG. 2025.

PDF | Preprint | Supplement | BibTeX


Khoury Vis Lab — Northeastern University
* West Village H, Room 302, 440 Huntington Ave, Boston, MA 02115, USA
* 100 Fore Street, Portland, ME 04101, USA
* Carnegie Hall, 201, 5000 MacArthur Blvd, Oakland, CA 94613, USA