Abstract
Artificial intelligence (AI) technologies have become the key driver of innovation in society. However, numerous vulnerabilities of AI systems can lead to negative consequences for society, such as biases encoded in the training data and algorithms and lack of transparency. This calls for AI systems to be audited to ensure that the impact on society is understood and mitigated. To enable AI audits, auditability measures need to be implemented. This study provides a systematic review of academic work and regulatory work on AI audits and AI auditability. Results reveal the current understanding of the AI audit scope, audit challenges, and auditability measures. We identify and categorize AI auditability measures for each audit area and specific process to be audited and the party responsible for each process to be audited. Our findings will guide existing efforts to make AI systems auditable across the lifecycle of AI systems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability Statements
Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.
References
AI HLEG. (2020). Assessment list for trustworthy artificial intelligence (altai) for self-assessment. Retrieved July 20, 2023, from https://digital-strategy.ec.europa.eu/en/library/assessment-list-trustworthy-artificial-intelligence-altai-self-assessment
AI HLEG. (2019). Ethics Guidelines for Trustworthy AI. Retrieved July 20, 2023, from https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai
Adler, P., Falk, C., Friedler, S. A., Nix, T., Rybeck, G., Scheidegger, C., & Venkatasubramanian, S. (2018). Auditing black-box models for indirect influence. Knowledge and Information Systems, 54, 95–122. https://doi.org/10.1007/s10115-017-1116-3
Albahri, A. S., Duhaim, A. M., Fadhel, M. A., Alnoor, A., Baqer, N. S., Alzubaidi, L., & Deveci, M. (2023). A systematic review of trustworthy and explainable artificial intelligence in healthcare: assessment of quality, bias risk, and data fusion. Information Fusion, 96, 156–191. https://doi.org/10.1016/j.inffus.2023.03.008
Amend, J. J., & Spurlock, S. (2021). Improving machine learning fairness with sampling and adversarial learning. Journal of Computing Sciences in Colleges, 36, 14–23.
Amri, M. M., Kumar, V., Khattak, W. A., Pandey, D., & Kundu, A. (2021). Personalized healthcare in the age of AI: A comprehensive overview of its applications and limitations. International Journal of Intelligent Automation and Computing, 4(1), 20–34. https://research.tensorgate.org/index.php/IJIAC/article/view/7.
Arnold, M., Bellamy, R. K., Hind, M., Houde, S., Mehta, S., Mojsilović, A., & Varshney, K. R. (2019). FactSheets: Increasing trust in AI services through supplier’s declarations of conformity. IBM Journal of Research and Development, 63(4/5), 6–1. https://doi.org/10.1147/JRD.2019.2942288
Ashmore, R., Calinescu, R., & Paterson, C. (2021). Assuring the machine learning lifecycle: Desiderata, methods, and challenges. ACM Computing Surveys (CSUR), 54(5), 1–39. https://doi.org/10.1145/3453444
Asif, R., Hassan, S. R., & Parr, G. (2023). Integrating a blockchain-based governance framework for responsible AI. Future Internet, 15(3), 97. https://doi.org/10.3390/fi15030097
Ayling, J., & Chapman, A. (2022). Putting AI ethics to work: Are the tools fit for purpose? AI and Ethics, 2(3), 405–429. https://doi.org/10.1007/s43681-021-00084-x
Balahur, A., Jenet, A., Hupont Torres, I., Charisi, V., Ganesh, A., Griesinger, C. B., Maurer, P., Mian, L., Salvi, M., Scalzo, S., Soler Garrido, J., Taucer, F., & Tolan, S. (2022). Data quality requirements for inclusive, non-biased and trustworthy AI - Putting science into standards. In JRC Conference and Workshop. https://doi.org/10.2760/365479
Bankins, S., Formosa, P., Griep, Y., & Richards, D. (2022). AI decision making with dignity? Contrasting workers’ justice perceptions of human and AI decision making in a human resource management context. Information Systems Frontiers, 24, 1–19. https://doi.org/10.1007/s10796-021-10223-8
Barocas, S., Guo, A., Kamar, E., Krones, J., Morris, M. R., Vaughan, J. W., ... & Wallach, H. (2021). Designing disaggregated evaluations of ai systems: Choices, considerations, and tradeoffs. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (pp. 368–378). https://doi.org/10.1145/3461702.3462610
Bell, A., Solano-Kamaiko, I., Nov, O., & Stoyanovich, J. (2022). It’s just not that simple: an empirical study of the accuracy-explainability trade-off in machine learning for public policy. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 248–266). https://doi.org/10.1145/3531146.3533090
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big?. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency (pp. 610–623). https://doi.org/10.1145/3442188.3445922
Bender, E. M., & Friedman, B. (2018). Data statements for NLP: Toward mitigating system bias and enabling better science. Transactions of the Association for Computational Linguistics, 6, 587–604. https://doi.org/10.1162/tacl_a_00041
Berghoff, C., Böddinghaus14, J., Danos, V., Davelaar13, G., Doms, T., Ehrich, H., ... & Fliehe, M. (2021). Towards Auditable AI Systems (p. 100). Technical report. Federal Office for Information Security.
Bertino, E., Kundu, A., & Sura, Z. (2019). Data transparency with blockchain and AI ethics. Journal of Data and Information Quality (JDIQ), 11(4), 1–8. https://doi.org/10.1145/3312750
Bhatt, U., Xiang, A., Sharma, S., Weller, A., Taly, A., Jia, Y., ... & Eckersley, P. (2020). Explainable machine learning in deployment. In Proceedings of the 2020 conference on fairness, accountability, and transparency (pp. 648–657). https://doi.org/10.1145/3351095.3375624
Birkstedt, T., Minkkinen, M., Tandon, A., & Mäntymäki, M. (2023). AI governance: Themes, knowledge gaps and future agendas. Internet Research, 33(7), 133–167. https://doi.org/10.1108/INTR-01-2022-0042
Bourgais, A., & Ibnouhsein, I. (2021). Ethics-by-design: The next frontier of industrialization. AI and Ethics, 2, 317–324. https://doi.org/10.1007/s43681-021-00057-0
Boyd, K. L. (2021). Datasheets for datasets help ML engineers notice and understand ethical issues in training data. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2), 1–27. https://doi.org/10.1145/3479582
Boyd, K. (2022). Designing up with value-sensitive design: Building a field guide for ethical ML development. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 2069–2082). https://doi.org/10.1145/3531146.3534626
Cai, W., Encarnacion, R., Chern, B., Corbett-Davies, S., Bogen, M., Bergman, S., & Goel, S. (2022). Adaptive sampling strategies to construct equitable training datasets. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 1467–1478). https://doi.org/10.1145/3531146.3533203
Calzarossa, M. C., De Lotto, I., & Rogerson, S. (2010). Ethics and information systems - guest editors introduction. Information Systems Frontiers, 12(4), 357–359. https://doi.org/10.1007/s10796-009-9198-4
Chmielinski, K. S., Newman, S., Taylor, M., Joseph, J., Thomas, K., Yurkofsky, J., & Qiu, Y. C. (2022). The dataset nutrition label (2nd Gen): Leveraging context to mitigate harms in artificial intelligence. arXiv preprint arXiv:2201.03954. https://doi.org/10.48550/arXiv.2201.03954
Christoforaki, M., & Beyan, O. (2022). Ai ethics—a bird’s eye view. Applied Sciences, 12(9), 4130. https://doi.org/10.3390/app12094130
Chui, M., Hazan, E., Roberts, R., Singla, A., & Smaje, K. (2023). The economic potential of generative AI. McKinsey & Company. Retrieved May 11, 2024, from http://dln.jaipuria.ac.in:8080/jspui/bitstream/123456789/14313/1/The-economic-potential-of-generative-ai-the-next-productivity-frontier.pdf
Clarke, R. (2019). Why the world wants controls over Artificial Intelligence. Computer Law & Security Review, 35(4), 423–433. https://doi.org/10.1016/j.clsr.2019.04.006
Cobbe, J., Lee, M. S. A., & Singh, J. (2021). Reviewable automated decision-making: A framework for accountable algorithmic systems. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency (pp. 598–609). https://doi.org/10.1145/3442188.3445921
Cooper, A. F., Moss, E., Laufer, B., & Nissenbaum, H. (2022). Accountability in an algorithmic society: relationality, responsibility, and robustness in machine learning. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 864–876). https://doi.org/10.1145/3531146.3533150
Corrêa, N. K., Galvão, C., Santos, J. W., Del Pino, C., Pinto, E. P., Barbosa, C., ... & Terem, E. (2022). Worldwide AI Ethics: a review of 200 guidelines and recommendations for AI governance. arXiv preprint arXiv:2206.11922. https://doi.org/10.48550/arXiv.2206.11922
Coston, A., Guha, N., Ouyang, D., Lu, L., Chouldechova, A., & Ho, D. E. (2021). Leveraging administrative data for bias audits: Assessing disparate coverage with mobility data for COVID-19 policy. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 173–184). https://doi.org/10.1145/3442188.3445881
Crisan, A., Drouhard, M., Vig, J., & Rajani, N. (2022). Interactive model cards: A human-centered approach to model documentation. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 427–439). https://doi.org/10.1145/3531146.3533108
CSSF. (2018). Artificial Intelligence: opportunities, risks and recommendations for the financial sector. Retrieved July 20, 2023, from https://www.cssf.lu/wp-content/uploads/files/Publications/Rapports_ponctuels/CSSF_White_Paper_Artificial_Intelligence_201218.pdf
D’Antonoli, T. A. (2020). Ethical considerations for artificial intelligence: An overview of the current radiology landscape. Diagnostic and Interventional Radiology, 26(5), 504. https://doi.org/10.5152/dir.2020.19279
Dai, J., Fazelpour, S., & Lipton, Z. (2021). Fair machine learning under partial compliance. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (pp. 55–65). https://doi.org/10.1145/3461702.3462521
Datta, A., Sen, S., & Zick, Y. (2016). Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. In 2016 IEEE symposium on security and privacy (SP) (pp. 598–617). IEEE. https://doi.org/10.1109/SP.2016.42
Dennehy, D., Griva, A., Pouloudi, N., Dwivedi, Y. K., Mäntymäki, M., & Pappas, I. O. (2023). Artificial intelligence (AI) and information systems: Perspectives to responsible AI. Information Systems Frontiers, 25(1), 1–7. https://doi.org/10.1007/s10796-022-10365-3
DRCF. (2022). Auditing algorithms: the existing landscape, role of regulators and future outlook. Retrieved July 20, 2023, from https://www.gov.uk/government/publications/findings-from-the-drcf-algorithmic-processing-workstream-spring-2022/auditing-algorithms-the-existing-landscape-role-of-regulators-and-future-outlook
Dvorak, R., Liao, H., Schibel, S., & Tribelhorn, B. (2021). Towards evaluating ethical accountability and trustworthiness in AI systems. Journal of Computing Sciences in Colleges, 37(2), 11–22.
EC. (2018). Communication on artificial intelligence for Europe. Retrieved July 20, 2023, from https://ec.europa.eu/digital-single-market/en/news/communication-artificial-intelligence-europe
Ellul, J., Pace, G., McCarthy, S., Sammut, T., Brockdorff, J., & Scerri, M. (2021). Regulating artificial intelligence: a technology regulator's perspective. In Proceedings of the eighteenth international conference on artificial intelligence and law (pp. 190–194). https://doi.org/10.1145/3462757.3466093
Enholm, I. M., Papagiannidis, E., Mikalef, P., & Krogstie, J. (2022). Artificial Intelligence and Business Value: a Literature Review. Information System Frontiers, 24, 1709–1734. https://doi.org/10.1007/s10796-021-10186-w
EPRS. (2020). The ethics of artificial intelligence: Issues and initiatives. Retrieved July 20, 2023, from https://policycommons.net/artifacts/1337278/the-ethics-of-artificial-intelligence/1944981/
EU. (2022). The AI Act. The EU AI Act Newsletter. Retrieved July 20, 2023, from https://artificialintelligenceact.eu/
European Group on Ethics in Science and New Technologies. (2018). Statement on artificial intelligence, robotics and ‘autonomous’ systems. Retrieved September 18, 2018, from https://data.europa.eu/doi/https://doi.org/10.2777/531856
Executive Office of the President. (2020). Promoting the Use of Trustworthy Artificial Intelligence in the Federal Government, Pub. L. No. 2020–27065. Retrieved July 20, 2023, from https://www.federalregister.gov/documents/2020/12/08/2020-27065/promoting-the-use-of-trustworthy-artificial-intelligence-in-the-federal-government
Falco, G., Shneiderman, B., Badger, J., Carrier, R., Dahbura, A., Danks, D., & Yeong, Z. K. (2021). Governing AI safety through independent audits. Nature Machine Intelligence, 3(7), 566–571. https://doi.org/10.1038/s42256-021-00370-7
Fan, W., Zhao, X., Chen, X., Su, J., Gao, J., Wang, L., ... & Li, Q. (2022). A comprehensive survey on trustworthy recommender systems. arXiv preprint arXiv:2209.10117. https://doi.org/10.48550/arXiv.2209.10117
Faßbender, J. (2021). Particles of a Whole: Design Patterns for Transparent and Auditable AI-Systems. In Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers (pp. 272–275). https://doi.org/10.1145/3460418.3479345
Federal Reserve and Office of the Comptroller of the Currency. (2011). SR 11–7: Guidance on Model Risk Management. Retrieved July 20, 2023, from https://www.federalreserve.gov/supervisionreg/srletters/sr1107.htm
Felzmann, H., Fosch-Villaronga, E., Lutz, C., & Tamò-Larrieux, A. (2020). Towards transparency by design for artificial intelligence. Science and Engineering Ethics, 26(6), 3333–3361. https://doi.org/10.1007/s11948-020-00276-4
Ferrario, A., & Loi, M. (2022). How explainability contributes to trust in AI. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 1457–1466). https://doi.org/10.1145/3531146.3533202
Feuerriegel, S., Dolata, M., & Schwabe, G. (2020). Fair AI: Challenges and opportunities. Business & Information Systems Engineering, 62, 379–384. https://doi.org/10.1007/s12599-020-00650-3
Floridi, L., Holweg, M., Taddeo, M., Amaya Silva, J., Mökander, J., & Wen, Y. (2022). CapAI-A procedure for conducting conformity assessment of AI systems in line with the EU artificial intelligence act. Available at SSRN 4064091. https://doi.org/10.2139/ssrn.4064091
Franklin, M., Ashton, H., Awad, E., & Lagnado, D. (2022). Causal framework of artificial autonomous agent responsibility. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society (pp. 276–284). https://doi.org/10.1145/3514094.3534140
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Iii, H. D., & Crawford, K. (2021). Datasheets for datasets. Communications of the ACM, 64(12), 86–92. https://doi.org/10.1145/3458723
Gerke, S., Minssen, T., & Cohen, G. (2020). Ethical and legal challenges of artificial intelligence-driven healthcare. In Artificial intelligence in healthcare (pp. 295–336). Academic Press. https://doi.org/10.1016/B978-0-12-818438-7.00012-5
Government of Canada. (2018). Responsible use of artificial intelligence (AI). Digital Government Innovations. Retrieved July 20, 2023, from https://www.canada.ca/en/government/system/digital-government/digital-government-innovations/responsible-use-ai.html
Goyal, P., Soriano, A. R., Hazirbas, C., Sagun, L., & Usunier, N. (2022). Fairness indicators for systematic assessments of visual feature extractors. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 70–88). https://doi.org/10.1145/3531146.3533074
Guembe, B., Azeta, A., Misra, S., Osamor, V. C., Fernandez-Sanz, L., & Pospelova, V. (2022). The emerging threat of ai-driven cyber attacks: A review. Applied Artificial Intelligence, 36(1), 2037254. https://doi.org/10.1080/08839514.2022.2037254
Gupta, M., Parra, C., & Dennehy, D. (2021). Questioning racial and gender Bias in AI recommendations: Do Individual-Level Cultural values Matter? Information Systems Frontiers, 24, 1465–1481. https://doi.org/10.1007/s10796-021-10156-2
Havelka, D., & Merhout, J. W. (2013). Internal information technology audit process quality: Theory development using structured group processes. International Journal of Accounting Information Systems, 14(3), 165–192. https://doi.org/10.1016/j.accinf.2012.12.001
Holland, S., Hosny, A., Newman, S., Joseph, J., & Chmielinski, K. (2020). The dataset nutrition label. Data Protection and Privacy, 12(12), 1.
Hong, S. R., Hullman, J., & Bertini, E. (2020). Human factors in model interpretability: Industry practices, challenges, and needs. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW1), 1–26. https://doi.org/10.1145/3392878
Hopkins, A., & Booth, S. (2021). Machine learning practices outside big tech: How resource constraints challenge responsible development. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (pp. 134–145). https://doi.org/10.1145/3461702.3462527
Hutchinson, B., Smart, A., Hanna, A., Denton, E., Greer, C., Kjartansson, O., ... & Mitchell, M. (2021). Towards accountability for machine learning datasets: Practices from software engineering and infrastructure. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 560–575). https://doi.org/10.1145/3442188.3445918
Hutchinson, B., Rostamzadeh, N., Greer, C., Heller, K., & Prabhakaran, V. (2022). Evaluation gaps in machine learning practice. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 1859–1876). https://doi.org/10.1145/3531146.3533233
ICO. (2020). Guidance on the AI auditing framework: Draft guidance for consultation. Retrieved July 20, 2023, from https://ico.org.uk/media/about-the-ico/consultations/2617219/guidance-on-the-ai-auditing-framework-draft-for-consultation.pdf
IIA. (2017). Global Perspectives and Insights: The IIA’s Artificial Intelligence Auditing Framework. Retrieved July 20, 2023, from https://www.theiia.org/en/content/articles/global-perspectives-and-insights/2017/the-iias-artificial-intelligence-auditing-framework-practical-applications-part-ii/
IIA. (2020). The IIA’s Three Lines Model. Retrieved July 20, 2023, from https://www.theiia.org/globalassets/documents/resources/the-iias-three-lines-model-an-update-of-the-three-lines-of-defense-july-2020/three-lines-model-updated-english.pdf
Irion, K. (2022). Algorithms Off-limits? If digital trade law restricts access to source code of software then accountability will suffer. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 1561–1570). https://doi.org/10.1145/3531146.3533212
Jacovi, A., Marasović, A., Miller, T., & Goldberg, Y. (2021). Formalizing trust in artificial intelligence: Prerequisites, causes and goals of human trust in AI. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency (pp. 624–635). https://doi.org/10.1145/3442188.3445923
Janssen, M., Brous, P., Estevez, E., Barbosa, L. S., & Janowski, T. (2020). Data governance: Organizing data for trustworthy Artificial Intelligence. Government Information Quarterly, 37(3), 101493. https://doi.org/10.1016/j.giq.2020.101493
Kaplan, A., & Haenlein, M. (2020). Rulers of the world, unite! The challenges and opportunities of artificial intelligence. Business Horizons, 63(1), 37–50. https://doi.org/10.1016/j.bushor.2019.09.003
Kaur, D., Uslu, S., Rittichier, K. J., & Durresi, A. (2022). Trustworthy artificial intelligence: A review. ACM Computing Surveys (CSUR), 55(2), 1–38. https://doi.org/10.1145/3491209
Kazim, E., & Koshiyama, A. (2020). Explaining decisions made with AI: a review of the co-badged guidance by the ICO and the Turing Institute. Available at SSRN 3656269. https://doi.org/10.2139/ssrn.3656269
Knowles, B., & Richards, J. T. (2021). The sanction of authority: Promoting public trust in AI. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 262–271). https://doi.org/10.1145/3442188.3445890
Krafft, P. M., Young, M., Katell, M., Lee, J. E., Narayan, S., Epstein, M., ... & Barghouti, B. (2021). An action-oriented AI policy toolkit for technology audits by community advocates and activists. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency (pp. 772–781). https://doi.org/10.1145/3442188.3445938
Kroll, J. A. (2021). Outlining traceability: A principle for operationalizing accountability in computing systems. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 758–771). https://doi.org/10.1145/3442188.3445937
LaBrie, R. C., & Steinke, G. (2019). Towards a framework for ethical audits of AI algorithms. In Twenty-fifth Americas Conference on Information Systems. Cancun.
Laine, J., Minkkinen, M., & Mäntymäki, M. (2024) Ethics-based AI auditing: A systematic literature review on conceptualizations of ethical principles and knowledge contributions to stakeholders. Information & Management, 103969. https://doi.org/10.1016/j.im.2024.103969
Larson, J., Mattu, S., Kirchner, L., & Angwin, J. (2016). How we analyzed the COMPAS Recidivism Algorithm. ProPublica. Retrieved July 20, 2023, from https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm
Lee, M. S. A., & Singh, J. (2021). Risk identification questionnaire for detecting unintended bias in the machine learning development lifecycle. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (pp. 704–714). https://doi.org/10.1145/3461702.3462572
Leslie, D., Burr, C., Aitken, M., Cowls, J., Katell, M., & Briggs, M. (2021). Artificial intelligence, human rights, democracy, and the rule of law. The Council of Europe’s Ad Hoc Committee on Artificial Intelligence. Retrieved July 20, 2023, from https://rm.coe.int/primer-en-new-cover-pages-coe-english-compressed-2754-7186-0228-v-1/1680a2fd4a
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., & Zhou, B. (2023). Trustworthy ai: From principles to practices. ACM Computing Surveys, 55(9), 1–46. https://doi.org/10.1145/3555803
Li, Y., & Goel, S. (in press). Artificial intelligence auditability and auditors’ readiness for auditing AI systems. International Journal of Accounting Information Systems, Special Issue: Advanced Technologies and Decision Support for Audit.
Liang, T. P., Robert, L., Sarker, S., Cheung, C. M. K., Matt, C., Trenz, M., & Turel, O. (2021). Artificial intelligence and robots in individuals’ lives: how to align technological possibilities and ethical issues. Internet Research, 31(1). https://doi.org/10.1108/INTR-11-2020-0668
Lima, G., Grgić-Hlača, N., Jeong, J. K., & Cha, M. (2022). The conflict between explainable and accountable decision-making algorithms. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 2103–2113). https://doi.org/10.1145/3531146.3534628
Liu, H., Wang, Y., Fan, W., Liu, X., Li, Y., Jain, S., Liu, Y., Jain, A. K., & Tang, J. (2022). Trustworthy AI: A computational perspective. Association for Computing Machinery, 1(1), 1–55. http://arxiv.org/abs/2107.06641.
Loi, M., & Spielkamp, M. (2021). Towards accountability in the use of artificial intelligence for public administrations. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (pp. 757–766). https://doi.org/10.1145/3461702.3462631
Macrae, C. (2019). Governing the safety of artificial intelligence in healthcare. BMJ Quality & Safety, 28(6), 495–498. https://doi.org/10.1136/bmjqs-2019-009484
Mäntymäki, M., Minkkinen, M., Birkstedt, T., & Viljanen, M. (2022). Defining organizational AI governance. AI and Ethics, 2(4), 603–609. https://doi.org/10.1007/s43681-022-00143-x
Mei, Q., Xie, Y., Yuan, W., & Jackson, M. O. (2024). A turing test of whether AI chatbots are behaviorally similar to humans. In Proceedings of the National Academy of Sciences, 121(9), e2313925121. https://doi.org/10.1073/pnas.2313925121
Merhi, M. I. (2023). An Assessment of the Barriers Impacting Responsible Artificial Intelligence. Information System Frontiers, 25(3), 1147–1160. https://doi.org/10.1007/s10796-022-10276-3
Merhout, J. W., & Havelka, D. (2008). Information technology auditing: A value-added IT governance partnership between IT management and audit. Communications of the Association for Information Systems, 23(1), 26. https://doi.org/10.17705/1CAIS.02326
Meske, C., Bunde, E., Schneider, J., & Gersch, M. (2020). Explainable artificial intelligence: Objectives, stakeholders, and future research opportunities. Information Systems Management, 39(1), 53–63. https://doi.org/10.1080/10580530.2020.1849465
Miceli, M., Yang, T., Naudts, L., Schuessler, M., Serbanescu, D., & Hanna, A. (2021). Documenting computer vision datasets: An invitation to reflexive data practices. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 161–172). https://doi.org/10.1145/3442188.3445880
Miller, G. J. (2021). Artificial Intelligence Project Success Factors: Moral Decision-Making with Algorithms. In 2021 16th Conference on Computer Science and Intelligence Systems (FedCSIS) (pp. 379–390). IEEE. https://doi.org/10.15439/2021F26
Mingers, J., & Walsham, G. (2010). Toward ethical information systems: The contribution of discourse ethics. MIS Quarterly, 34(4), 833–854. https://doi.org/10.2307/25750707
Minkkinen, M., & Mäntymäki, M. (2023). Discerning between the “easy” and “hard” problems of AI governance. IEEE Transactions on Technology and Society, 4(2), 188–194. https://doi.org/10.1109/TTS.2023.3267382
Minkkinen, M., Laine, J., & Mäntymäki, M. (2022). Continuous auditing of artificial intelligence: A conceptualization and assessment of tools and frameworks. Digital Society, 1, 21. https://doi.org/10.1007/s44206-022-00022-2
Minkkinen, M., Niukkanen, A., & Mäntymäki, M. (2024). What about investors? ESG analyses as tools for ethics-based AI auditing. AI & Society, 39(1), 329–343. https://doi.org/10.1007/s00146-022-01415-0
Mirbabaie, M., Brendel, A. B., & Hofeditz, L. (2022). Ethics and AI in information systems research. Communications of the Association for Information Systems, 50(1), 38. https://doi.org/10.17705/1CAIS.05034
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., ... & Gebru, T. (2019). Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency (pp. 220–229). https://doi.org/10.1145/3287560.3287596
Mökander, J., & Floridi, L. (2023). Operationalising AI governance through ethics-based auditing: An industry case study. AI and Ethics, 3(2), 451–468. https://doi.org/10.1007/s43681-022-00171-7
Niederman, F., & Baker, E. W. (2023). Ethics and AI issues: Old Container with New Wine? Information Systems Frontiers, 25(1). https://doi.org/10.1007/s10796-022-10305-1
NIST. (2022). AI Risk Management Framework: Second Draft. Retrieved July 20, 2023, from https://www.nist.gov/system/files/documents/2022/08/18/AI_RMF_2nd_draft.pdf
Noriega-Campero, A., Garcia-Bulle, B., Cantu, L. F., Bakker, M. A., Tejerina, L., & Pentland, A. (2020). Algorithmic targeting of social policies: fairness, accuracy, and distributed governance. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (pp. 241–251). https://doi.org/10.1145/3351095.3375784
OECD. (2019). Recommendations of the Council on Artificial Intelligence. Retrieved July 20, 2023, from https://legalinstruments.oecd.org/en/instruments/oecd-legal-0449
Office of U.S. Senator Ron Wyden. (2022). Algorithmic Accountability Act of 2022. 117th Congress 2D Session. https://doi.org/10.1016/S0140-6736(02)37657-8
Olteanu, A., Castillo, C., Diaz, F., & Kıcıman, E. (2019). Social data: Biases, methodological pitfalls, and ethical boundaries. Frontiers in Big Data, 2, 13. https://doi.org/10.3389/fdata.2019.00013
Passi, S., & Jackson, S. J. (2018). Trust in data science: Collaboration, translation, and accountability in corporate data science projects. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW), 1–28. https://doi.org/10.1145/3274405
PDPC. (2020). Model artificial intelligence governance framework second edition. Retrieved July 20, 2023, from https://www.pdpc.gov.sg/-/media/files/pdpc/pdf-files/resource-for-organisation/ai/sgmodelaigovframework2.pdf
Pery, A., Rafiei, M., Simon, M., & van der Aalst, W. M. (2021). Trustworthy artificial intelligence and process mining: challenges and opportunities. In International Conference on Process Mining (pp. 395–407). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-98581-3_29
Procter, R., Tolmie, P., & Rouncefield, M. (2023). Holding AI to account: Challenges for the delivery of trustworthy AI in healthcare. ACM Transactions on Computer-Human Interaction, 30(2), 1–34. https://doi.org/10.1145/3577009
PwC. (2023). PwC's Responsible AI. Retrieved June 24, 2023, from https://www.pwc.com/gx/en/issues/data-and-analytics/artificial-intelligence/what-is-responsible-ai.html
Qadir, J., Islam, M. Q., & Al-Fuqaha, A. (2022). Toward accountable human-centered AI: Rationale and promising directions. Journal of Information, Communication and Ethics in Society, 20(2), 329–342. https://doi.org/10.1680/udap.2010.163
Raaijmakers, S. (2019). Artificial intelligence for law enforcement: Challenges and opportunities. IEEE Security & Privacy, 17(5), 74–77. https://doi.org/10.1109/MSEC.2019.2925649
Raja, A. K., & Zhou, J. (2023). AI accountability: Approaches, affecting factors, and challenges. Computer, 56(4), 61–70. https://doi.org/10.1109/MC.2023.3238390
Rajabi, E., & Etminani, K. (2022). Knowledge-graph-based explainable AI: A systematic review. Journal of Information Science, 01655515221112844. https://doi.org/10.1177/0165551522111284
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., ... & Barnes, P. (2020). Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In Proceedings of the 2020 conference on fairness, accountability, and transparency (pp. 33–44). https://doi.org/10.1145/3351095.3372873
Raji, I. D., Kumar, I. E., Horowitz, A., & Selbst, A. (2022). The fallacy of AI functionality. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 959–972). https://doi.org/10.1145/3531146.3533158
Raji, I. D., Xu, P., Honigsberg, C., & Ho, D. (2022). Outsider oversight: Designing a third party audit ecosystem for ai governance. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society (pp. 557–571). https://doi.org/10.1145/3514094.3534181
Rhea, A., Markey, K., D'Arinzo, L., Schellmann, H., Sloane, M., Squires, P., & Stoyanovich, J. (2022). Resume Format, LinkedIn URLs and Other Unexpected Influences on AI Personality Prediction in Hiring: Results of an Audit. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society (pp. 572–587). https://doi.org/10.1145/3514094.3534189
Rich, E. (1985). Artificial intelligence and the humanities. Computers and the Humanities, 19(2), 117–122. https://www.jstor.org/stable/30204398.
Robert, L. P., Pierce, C., Marquis, L., Kim, S., & Alahmad, R. (2020). Designing fair AI for managing employees in organizations: a review, critique, and design agenda. Human-Computer Interaction, 35(5–6). https://doi.org/10.1080/07370024.2020.1735391
SAI. (2020). Auditing machine learning algorithms: A white paper for public auditors. Retrieved July 20, 2023, from https://www.auditingalgorithms.net/
SAI. (2023). Auditing machine learning algorithms: A white paper for public auditors. Retrieved July 20, 2023, from https://www.auditingalgorithms.net/
Saleiro, P., Kuester, B., Hinkson, L., London, J., Stevens, A., Anisfeld, A., ... & Ghani, R. (2018). Aequitas: A bias and fairness audit toolkit. arXiv preprint arXiv:1811.05577. https://arxiv.org/abs/1811.05577.
Salvi del Pero, A., Wyckoff, P., & Vourc’H, A. (2022). Using Artificial Intelligence in the workplace: What are the main ethical risks?. http://hdl.voced.edu.au/10707/616557
Sandvig, C., Hamilton, K., Karahalios, K., & Langbort, C. (2014). Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and Discrimination: Converting Critical Concerns into Productive Inquiry, 22(2014), 4349–4357.
Scheuerman, M. K., Weathington, K., Mugunthan, T., Denton, E., & Fiesler, C. (2023). From Human to Data to Dataset: Mapping the Traceability of Human Subjects in Computer Vision Datasets. Proceedings of the ACM on Human-Computer Interaction, 7(CSCW1), 1–33. https://doi.org/10.1145/3579488
Shahriari, K., & Shahriari, M. (2017). IEEE standard review — Ethically aligned design: A vision for prioritizing human wellbeing with artificial intelligence and autonomous systems. In 2017 IEEE Canada International Humanitarian Technology Conference (IHTC) (pp.197–201). https://doi.org/10.1109/IHTC.2017.805818
Schneider, J., Abraham, R., Meske, C., & Vom Brocke, J. (2023). Artificial intelligence governance for businesses. Information Systems Management, 40(3), 229–249. https://doi.org/10.1080/10580530.2022.2085825
Schopmans, H. R. (2022). From Coded Bias to Existential Threat: Expert Frames and the Epistemic Politics of AI Governance. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society (pp. 627–640). https://doi.org/10.1145/3514094.3534161
Schöppl, N., Taddeo, M., & Floridi, L. (2022). Ethics auditing: Lessons from business ethics for ethics auditing of AI. In: Mökander, J., Ziosi, M. (eds) The 2021 Yearbook of the Digital Ethics Lab. Digital Ethics Lab Yearbook. Springer, Cham. https://doi.org/10.1007/978-3-031-09846-8_13
Shen, M., Gu, A., Kang, J., Tang, X., Lin, X., Zhu, L., & Niyato, D. (2023). Blockchains for artificial intelligence of things: A comprehensive survey. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2023.3268705
Shneiderman, B. (2020). Bridging the gap between ethics and practice: Guidelines for reliable, safe, and trustworthy human-centered AI systems. ACM Transactions on Interactive Intelligent Systems (TiiS), 10(4), 1–31. https://doi.org/10.1145/3419764
Sloane, M., Moss, E., & Chowdhury, R. (2022a). A Silicon Valley love triangle: Hiring algorithms, pseudo-science, and the quest for auditability. Patterns, 3(2), 100425. https://doi.org/10.1016/j.patter.2021.100425
Sloane, M., & Zakrzewski, J. (2022). German AI Start-Ups and “AI Ethics”: Using A Social Practice Lens for Assessing and Implementing Socio-Technical Innovation. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 935–947). https://doi.org/10.1145/3531146.3533156
Solans, D., Fabbri, F., Calsamiglia, C., Castillo, C., & Bonchi, F. (2021). Comparing Equity and Effectiveness of Different Algorithms in an Application for the Room Rental Market. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (pp. 978–988). https://doi.org/10.1145/3461702.3462600
Stoel, M. D., & Havelka, D. (2021). Information technology audit quality: An investigation of the impact of individual and organizational factors. Journal of Information Systems, 35(1), 135–154. https://doi.org/10.2308/isys-18-043
Stoel, D., Havelka, D., & Merhout, J. W. (2012). An analysis of attributes that impact information technology audit quality: A study of IT and financial audit practitioners. International Journal of Accounting Information Systems, 13(1), 60–79. https://doi.org/10.1016/j.accinf.2011.11.001
Toader, A. (2019). Auditability of AI systems – Brake or acceleration to innovation?. Available at SSRN 3526222. https://doi.org/10.2139/ssrn.3526222
Toreini, E., Aitken, M., Coopamootoo, K., Elliott, K., Zelaya, C. G., & Van Moorsel, A. (2020). The relationship between trust in AI and trustworthy machine learning technologies. In Proceedings of the 2020 conference on fairness, accountability, and transparency (pp. 272–283). https://doi.org/10.1145/3351095.3372834
Turing, A. (1950). Computing Machinery and Intelligence. In: Copeland, B. J. (Eds). The Essential Turing. Clarendon Press Oxford.
UNESCO. (2021). Recommendation on the Ethics of Artificial Intelligence. Retrieved July 20, 2023, from https://unesdoc.unesco.org/ark:/48223/pf0000381137
Unruh, C. F., Haid, C., Johannes, F., & Büthe, T. (2022). Human autonomy in algorithmic management. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society (pp. 753–762). https://doi.org/10.1145/3514094.3534168
Villani, C., Bonnet, Y., & Rondepierre, B. (2018). For a meaningful artificial intelligence: Towards a French and European strategy. Conseil national du numérique. Retrieved June 23, 2023, from https://www.aiforhumanity.fr/pdfs/MissionVillani_Report_ENG-VF.pdf
Weber, M., Engert, M., Schaffer, N., et al. (2023). Organizational Capabilities for AI Implementation—Coping with Inscrutability and Data Dependency in AI. Information Systems Frontiers, 25(4), 1549–1569. https://doi.org/10.1007/s10796-022-10297-y
Wieringa, M. (2020). What to account for when accounting for algorithms: a systematic literature review on algorithmic accountability. In Proceedings of the 2020 conference on fairness, accountability, and transparency (pp. 1–18). https://doi.org/10.1145/3351095.3372833
Wong, R. Y., Madaio, M. A., & Merrill, N. (2023). Seeing like a toolkit: How toolkits envision the work of AI ethics. Proceedings of the ACM on Human-Computer Interaction, 7(CSCW1), 1–27. https://doi.org/10.1145/3579621
Yurrita, M., Murray-Rust, D., Balayn, A., & Bozzon, A. (2022). Towards a multi-stakeholder value-based assessment framework for algorithmic systems. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 535–563). https://doi.org/10.1145/3531146.3533118
Zinda, N. (2022). Ethics auditing framework for trustworthy AI: Lessons from the IT audit literature. In: Mökander, J., Ziosi, M. (eds) The 2021 Yearbook of the Digital Ethics Lab. Digital Ethics Lab Yearbook. Springer, Cham. https://doi.org/10.1007/978-3-031-09846-8_12
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Declarations of interest
None.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Li, Y., Goel, S. Making It Possible for the Auditing of AI: A Systematic Review of AI Audits and AI Auditability. Inf Syst Front (2024). https://doi.org/10.1007/s10796-024-10508-8
Accepted:
Published:
DOI: https://doi.org/10.1007/s10796-024-10508-8