MIND: Empowering Mental Health Clinicians with Multimodal Data Insights through a Narrative Dashboard

A comparison between the proposed narrative dashboard MIND (A) with a data collection dashboard design (B), showing the data from the same hypothetical patient. MIND presents computationally generated multimodal insights in a narrative way ❶, allowing clinicians to better understand multimodal patient data. Clinicians can also easily navigate through timeline ❷, filter by insight category ❸, and expand details on demand ❹.
Abstract
Advances in data collection enable the capture of rich patient-generated data: from passive sensing (e.g., wearables and smartphones) to active self-reports (e.g., cross-sectional surveys and ecological momentary assessments). Although prior research has demonstrated the utility of patient-generated data in mental healthcare, significant challenges remain in effectively presenting these data streams along with clinical data (e.g., clinical notes) for clinical decision-making. Through co-design sessions with five clinicians, we propose MIND, a large language model-powered dashboard designed to present clinically relevant multimodal data insights for mental healthcare. MIND presents multimodal insights through narrative text, complemented by charts communicating underlying data. Our user study (N=16) demonstrates that clinicians perceive MIND as a significant improvement over baseline methods, reporting improved performance to reveal hidden and clinically relevant data insights (p<.001) and support their decision-making (p=.004). Grounded in the study results, we discuss future research opportunities to integrate data narratives in broader clinical practices.
Materials
PDF | Preprint | DOI | BibTeX | CHI Best Paper Honorable Mention
Authors
Ruishi Zou
Shiyu Xu
Margaret E Morris
Jihan Ryu
Timothy D. Becker
Nicholas Allen
Anne Marie Albano
Randy Auerbach
Dan Adler
Varun Mishra
Dakuo Wang
Ryan Sultan
Xuhai Xu
Citation
Thumbnail image for publication titled: MIND: Empowering Mental Health Clinicians with Multimodal Data Insights through a Narrative Dashboard
MIND: Empowering Mental Health Clinicians with Multimodal Data Insights through a Narrative Dashboard

Ruishi Zou, Shiyu Xu, Margaret E Morris, Jihan Ryu, Timothy D. Becker, Nicholas Allen, Anne Marie Albano, Randy Auerbach, Dan Adler, Varun Mishra, Lace M. Padilla, Dakuo Wang, Ryan Sultan, and Xuhai Xu. Proc. CHI Conference on Human Factors in Computing Systems—CHI. 2026. DOI: 10.1145/3772318.3790529

PDF | Preprint | DOI | BibTeX | CHI Best Paper Honorable Mention


Khoury Vis Lab — Northeastern University
* West Village H, Room 302, 440 Huntington Ave, Boston, MA 02115, USA
* 100 Fore Street, Portland, ME 04101, USA
* Carnegie Hall, 201, 5000 MacArthur Blvd, Oakland, CA 94613, USA