MatplotAlt: A Python Library for Adding Alt Text to Matplotlib Figures in Computational Notebooks

A screenshot of a Jupyter notebook with annotations. At the top is a Python code cell with the annotation "Matplotlib code to generate a barplot". Below the Matplotlib code is the line show_with_alt(desc_level=3, methods=["markdown", "img_file"]). An annotation indicates this line is "calling MatplotAlt's show_with_alt function to generate and surface alt text using the 'markdown' and 'img_file' methods". Below the code cell is alt text for the generated bar plot in markdown, with the annotation "Heuristic-based alt text is displayed in markdown output". Finally, at the bottom of the figure is the generated barplot displaying average yearly hours of sunshine for each month. The last annotation indicates that this "Matplotlib figure is displayed and saved as an image file with embedded alt text".
MatplotAlt can be used to generate and surface Matplotlib alt text in a single line of code. In this example, we call MatplotAlt's show_with_alt function after creating a Matplotlib bar chart to display heuristic-based alt text in markdown and embedded in a saved version of the figure. desc_level=3 indicates that the description includes encodings, statistics, and trends. MatplotAlt also provides options to embed alt text directly in Jupyter figures, and generate descriptions using vision language models.
Abstract
We present MatplotAlt, an open-source Python package for easily adding alternative text to Matplotlib figures. MatplotAlt equips Jupyter notebook authors to automatically generate and surface chart descriptions with a single line of code or command, and supports a range of options that allow users to customize the generation and display of captions based on their preferences and accessibility needs. Our evaluation indicates that MatplotAlt’s heuristic and LLM-based methods to generate alt text can create accurate long-form descriptions of both simple univariate and complex Matplotlib figures. We find that state-of-the-art LLMs still struggle with factual errors when describing charts, and improve the accuracy of our descriptions by prompting GPT4-turbo with heuristic-based alt text or data tables parsed from the Matplotlib figure.
Materials
PDF | Preprint | DOI | Code | Homepage | BibTeX
Authors
Jennifer Mankoff
Venkatesh Potluri
Citation

Khoury Vis Lab — Northeastern University
* West Village H, Room 302, 440 Huntington Ave, Boston, MA 02115, USA
* 100 Fore Street, Portland, ME 04101, USA
* Carnegie Hall, 201, 5000 MacArthur Blvd, Oakland, CA 94613, USA