Advancing AI Incidents Classification: Leveraging LLMs with Strategic Prompting

A hierarchical tree diagram titled 'Irresponsible AI Taxonomy' branching from a single root node into categories: discrimination (splitting into data bias with subcategories gender, race, sexual orientation, economic; and algorithmic bias with subcategories interaction, feedback loop, optimization function), human incompetence, pseudoscience (facial, other), environmental impact, disinformation (copyright violation, mental health, textual, image, video, audio).
Figure: Irresponsible AI Taxonomy — a tree visualization classifying types of irresponsible AI behavior used as the classification schema for AI incident labeling with LLMs.
Abstract
This paper presents an approach to classifying AI incidents using large language models (LLMs) with strategic prompting. The work demonstrates how carefully designed prompts can improve the accuracy and consistency of AI incident classification, supporting safer AI deployment.
Materials
DOI | Homepage | BibTeX
Authors
Yian Chen
Lana Do
Liheng Yi
Ricardo Baeza-Yates
John Alexis Guerra Gomez
Citation
Thumbnail image for publication titled: Advancing AI Incidents Classification: Leveraging LLMs with Strategic Prompting
Advancing AI Incidents Classification: Leveraging LLMs with Strategic Prompting

Yian Chen, Lana Do, Liheng Yi, Ricardo Baeza-Yates, and John Alexis Guerra-Gomez. Communications in Computer and Information Science—CCIS. 2024. DOI: 10.1007/978-3-031-91328-0_4

DOI | Homepage | BibTeX

Khoury Vis Lab — Northeastern University
* West Village H, Room 302, 440 Huntington Ave, Boston, MA 02115, USA
* 100 Fore Street, Portland, ME 04101, USA
* Carnegie Hall, 201, 5000 MacArthur Blvd, Oakland, CA 94613, USA