Why We Need to Know More: Exploring the State of AI Incident Documentation Practices

Catherine Aiken. 2021. Classifying AI Systems. Technical Report. Center for Security and Emerging Technology. https://doi.org/10.51593/20200025

Anonymous. 2016. Incident Number 20. AI Incident Database (2016). https://incidentdatabase.ai/cite/20

Anonymous. 2016. Incident Number 40. AI Incident Database (2016). https://incidentdatabase.ai/cite/40

Daniel Atherton. 2023. Incident Number 541. AI Incident Database (2023). https://incidentdatabase.ai/cite/541

Hollen Barmer, Rachel Dzombak, Matt Gaston, Eric Heim, Jay Palat, Frank Redner, Tanisha Smith, and Nathan VanHoudnos. 2021. Robust and Secure AI. https://resources.sei.cmu.edu/asset_files/WhitePaper/2021_019_001_735346.pdf

Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang. 2022. On the Opportunities and Risks of Foundation Models. https://doi.org/10.48550/arXiv.2108.07258 arXiv:2108.07258 [cs].

Catherine Olsson [@catherineols]. 2019. @paul_scharre I keep a list of (mostly unintended) AI failures, and can share some that are in a grey area, because the harm is not *the goal*, but seems to be fairly closely tied to how the goal was achieved, and the relevant actors knew what was happening:. https://twitter.com/catherineols/status/1105561165646585857

Paul Cichonski, Tom Millar, Tim Grance, and Karen Scarfone. 2012. Computer Security Incident Handling Guide : Recommendations of the National Institute of Standards and Technology. Technical Report NIST SP 800-61r2. National Institute of Standards and Technology. NIST SP 800–61r2 pages. https://doi.org/10.6028/NIST.SP.800-61r2

Responsible AI Collaborative. 2023. The Artificial Intelligence Incident Database. Retrieved July 5, 2023 from https://incidentdatabase.ai/

Richard Cook, David Woods, and Charlotte Miller. 1998. A Tale of Two Stories: Contrasting Views of Patient Safety. Technical Report. National Health Care Safety Council of the National Patient Safety Foundation at the AMA.

Jeffrey Cooper, Ronald Newbower, Charlene Long, and Bucknam McPeek. 1978. Preventable Anesthesia Mishaps: A Study of Human Factors. Anesthesiology 49, 309 (1978).

David Dao. 2023. Awful AI. https://github.com/daviddao/awful-ai original-date: 2018-03-27T15:30:34Z.

Cornelia Dean. 2006. Engineering a Safer, More Beautiful World, One Failure at a Time. The New York Times (May 2006). https://www.nytimes.com/2006/05/02/science/02prof.html

Government of India Department of Personnel & Training. 2021. Right to Information Act 2005.

Divij Joshi and Mozilla Foundation. 2020. AI Observatory. https://ai-observatory.in/

Electronic Frontier Foundation. 2023. Atlas of Surveillance. https://atlasofsurveillance.org/

Eticas Foundation. 2023. Observatory of Algorithms with Social Impact (OASI) Register. https://airtable.com/shrsAN2oTf68kM6O9/tblG2604tSoMOcwWX?backgroundC+olor=teal&viewControls=on

EU AI Watch. 2023. AI Public Services Explorer. https://ai-watch.github.io/AI-watch-T6-X/

Federal Aviation Administration. 2023. FAA Aviation Safety Information Analysis and Sharing. https://www.asias.faa.gov/apex/f?p=100:1

AI Global. 2021. The Artificial Intelligence Incident Database. Retrieved July 5, 2023 from https://map.ai-global.org/

GobLab UAI. 2023. Algoritmos Públicos. https://www.algoritmospublicos.cl/

Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé, Miro Dudik, and Hanna Wallach. 2019. Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–16. https://doi.org/10.1145/3290605.3300830

National Artificial Intelligence Initiative. 2020. Agency inventories of AI use cases. https://www.ai.gov/ai-use-case-inventories/

Internet Freedom Foundation. 2023. Panoptic Tracker. https://panoptic.in

Machine Learning Failures [@mlfailures]. 2021. Machine Learning Learning Failures - Daylight Lab at UC Berkeley CLTC. https://twitter.com/mlfailures/status/1357100619182403584

Sean McGregor. 2016. Incident Number 51. AI Incident Database (2016). https://incidentdatabase.ai/cite/51

Sean McGregor. 2020. Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. https://doi.org/10.48550/arXiv.2011.08512 arXiv:2011.08512 [cs].

Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A Survey on Bias and Fairness in Machine Learning. Comput. Surveys 54, 6 (July 2021), 115:1–115:35. https://doi.org/10.1145/3457607

MITRE. 2023. Common Attack Pattern Enumeration and Classification. https://capec.mitre.org/

MITRE. 2023. Common Vulnerabilities and Exposures. https://cve.mitre.org/

MITRE. 2023. Common Weakness Enumeration. https://cwe.mitre.org/

NASA. 2023. ASRS - Aviation Safety Reporting System. https://asrs.arc.nasa.gov/

NTSB. 2023. Case Analysis and Reporting Online (CAROL). https://data.ntsb.gov/carol-main-public/landing-page

NTSB. 2023. The Investigative Process. https://www.ntsb.gov/investigations/process/Pages/default.aspx

City of Amsterdam. 2023. City of Amsterdam Algorithm Register. https://algoritmeregister.amsterdam.nl/en/ai-register/

Code of Federal Regulations. 2023. 49 CFR 830.2 - Definitions. https://www.ecfr.gov/current/title-49/subtitle-B/chapter-VIII/part-830/subpart-A/section-830.2

City of Helsinki. 2023. City of Helsinki AI Register. https://ai.hel.fi/en/ai-register/

Clinton V. Oster, John S. Strong, and C. Kurt Zorn. 2013. Analyzing aviation safety: Problems, challenges, opportunities. Research in Transportation Economics 43, 1 (July 2013), 148–164. https://doi.org/10.1016/j.retrec.2012.12.001

ph_. 2023. awesome-machine-learning-interpretability. https://github.com/jphall663/awesome-machine-learning-interpretability original-date: 2018-06-21T14:26:51Z.

Charlie Pownall and CPC & Associates. 2023. AIAAIC. https://www.aiaaic.org/

Coding Rights and Paz Peña. 2022. AI Projects in the Public Sector in Latin America. https://notmy.ai/mapping-of-projects/

AI Risk and Vulnerability Analysis. 2023. AI Vulnerability Database. https://avidml.org/

AI Risk and Vulnerability Analysis. 2023. AI Vulnerability Database Taxonomy. https://avidml.org/taxonomy/

Drew Roselli, Jeanna Matthews, and Nisha Talagala. 2019. Managing Bias in AI. In Companion Proceedings of The 2019 World Wide Web Conference(WWW ’19). Association for Computing Machinery, New York, NY, USA, 539–544. https://doi.org/10.1145/3308560.3317590

Jonathan M. Spring, April Galyardt, Allen D. Householder, and Nathan VanHoudnos. 2021. On managing vulnerabilities in AI/ML systems. In New Security Paradigms Workshop 2020(NSPW ’20). Association for Computing Machinery, New York, NY, USA, 111–126. https://doi.org/10.1145/3442167.3442177

Ramya Srinivasan and Ajay Chander. 2021. Biases in AI Systems: A survey for practitioners. Queue 19, 2 (May 2021), Pages 10:45–Pages 10:64. https://doi.org/10.1145/3466132.3466134

Charles Vincent. 2007. Incident reporting and patient safety. BMJ 334, 7584 (2007), 51–51. https://doi.org/10.1136/bmj.39071.441609.80 arXiv:https://www.bmj.com/content/334/7584/51.full.pdf

Angelina Wang, Alexander Liu, Ryan Zhang, Anat Kleiman, Leslie Kim, Dora Zhao, Iroha Shirai, Arvind Narayanan, and Olga Russakovsky. 2022. REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets. International Journal of Computer Vision 130, 7 (July 2022), 1790–1810. https://doi.org/10.1007/s11263-022-01625-5