More Web Proxy on the site http://driver.im/

research-article

Open access

The Model Card Authoring Toolkit: Toward Community-centered, Deliberation-driven AI Design

Authors:

Wesley H. Deng,

Ronald Velgersdijk,

Haiyi ZhuAuthors Info & Claims

FAccT '22: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency

Pages 440 - 451

https://doi.org/10.1145/3531146.3533110

Published: 20 June 2022 Publication History

All formats PDF

Abstract

There have been increasing calls for centering impacted communities – both online and offline – in the design of the AI systems that will be deployed in their communities. However, the complicated nature of a community’s goals and needs, as well as the complexity of AI’s development procedures, outputs, and potential impacts, often prevents effective participation. In this paper, we present the Model Card Authoring Toolkit, a toolkit that supports community members to understand, navigate and negotiate a spectrum of machine learning models via deliberation and pick the ones that best align with their collective values. Through a series of workshops, we conduct an empirical investigation of the initial effectiveness of our approach in two online communities – English and Dutch Wikipedia, and document how our participants collectively set the threshold for a machine learning based quality prediction system used in their communities’ content moderation applications. Our results suggest that the use of the Model Card Authoring Toolkit helps improve the understanding of the trade-offs across multiple community goals on AI design, engage community members to discuss and negotiate the trade-offs, and facilitate collective and informed decision-making in their own community contexts. Finally, we discuss the challenges for a community-centered, deliberation-driven approach for AI design as well as potential design implications.

References

[1]

Julia Abelson, Pierre-Gerlier Forest, John Eyles, Patricia Smith, Elisabeth Martin, and Francois-Pierre Gauvin. 2003. Deliberations about deliberative methods: issues in the design and evaluation of public participation processes. Social Science & Medicine 57, 2 (2003), 239–251.

[2]

Muhammad Ali, Piotr Sapiezynski, Miranda Bogen, Aleksandra Korolova, Alan Mislove, and Aaron Rieke. 2019. Discrimination through optimization: How Facebook’s Ad delivery can lead to biased outcomes. Proceedings of the ACM on Human-Computer Interaction 3, CSCW(2019), 1–30.

Digital Library

[3]

Rico Angell, Brittany Johnson, Yuriy Brun, and Alexandra Meliou. 2018. Themis: Automatically testing software for discrimination. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 871–875.

Digital Library

[4]

Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias. ProPublica (May 2016). https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

[5]

Ofer Arazy, Oded Nov, and Felipe Ortega. 2014. The [Wikipedia] world is not flat: On the organizational structure of online production communities. In Proceedings of the Twenty Second European Conference on Information Systems. 1–15.

[6]

Patti Bao, Brent Hecht, Samuel Carton, Mahmood Quaderi, Michael Horn, and Darren Gergle. 2012. Omnipedia: Bridging the wikipedia language gap. In Proceedings of the 2012 CHI Conference on Human Factors in Computing Systems. 1075–1084.

Digital Library

[7]

Solon Barocas and Andrew D Selbst. 2016. Big data’s disparate impact. Calif. L. Rev. 104(2016), 671–732.

[8]

Rachel KE Bellamy, Kuntal Dey, Michael Hind, Samuel C Hoffman, Stephanie Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, A Mojsilović, 2019. AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM Journal of Research and Development 63, 4/5 (2019), 4–1.

[9]

Yochai Benkler. 2008. The wealth of networks. Yale University Press.

[10]

Dimitris Bertsimas, Arthur Delarue, William Eger, John Hanlon, and Sebastien Martin. 2020. Bus routing optimization helps Boston public schools design better policies. INFORMS Journal on Applied Analytics 50, 1 (2020), 37–49.

Digital Library

[11]

Patrick Biernacki and Dan Waldorf. 1981. Snowball sampling: Problems and techniques of chain referral sampling. Sociological methods & research 10, 2 (1981), 141–163.

[12]

Sarah Bird, Miro Dudík, Richard Edgar, Brandon Horn, Roman Lutz, Vanessa Milan, Mehrnoosh Sameki, Hanna Wallach, and Kathleen Walker. 2020. Fairlearn: A toolkit for assessing and improving fairness in AI. Microsoft, Tech. Rep. MSR-TR-2020-32(2020).

[13]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (2006), 77–101.

[14]

Anna Brown, Alexandra Chouldechova, Emily Putnam-Hornstein, Andrew Tobin, and Rhema Vaithianathan. 2019. Toward algorithmic accountability in public services: A qualitative study of affected community perspectives on algorithmic decision-making in child welfare services. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[15]

Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency. 77–91.

[16]

Jenna Burrell. 2016. How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big Data & Society 3, 1 (2016), 1–12.

[17]

Robert J Cavalier. 2011. Approaching deliberative democracy: Theory and practice. Carnegie Mellon University Press.

[18]

Kathy Charmaz. 2014. Constructing grounded theory. Sage.

[19]

Irene Chen, Fredrik D Johansson, and David Sontag. 2018. Why is my classifier discriminatory?. In NIPS’18: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 3539–3550.

[20]

Hao-Fei Cheng, Logan Stapleton, Ruiqi Wang, Paige Bullock, Alexandra Chouldechova, Zhiwei Steven Steven Wu, and Haiyi Zhu. 2021. Soliciting Stakeholders’ Fairness Notions in Child Maltreatment Predictive Systems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–17.

Digital Library

[21]

Alexandra Chouldechova, Diana Benavides-Prado, Oleksandr Fialko, and Rhema Vaithianathan. 2018. A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency. 134–148.

[22]

Sarah Dean, Sarah Rich, and Benjamin Recht. 2020. Recommendations and user agency: The reachability of collaboratively-filtered information. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 436–445.

Digital Library

[23]

Virginia Eubanks. 2018. Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin’s Press.

Digital Library

[24]

James D. Fearon. 1998. Deliberation as discussion. In Deliberative democracy, J. Elster (Ed.). Cambridge University Press, Cambridge, MA, 44–68.

[25]

Nancy Fraser. 1992. Rethinking the public sphere: A contribution to the critique of acutally existing democracy. In Habermas and the public sphere, C. Calhoun (Ed.). The MIT Press, Cambridge, MA, 109–142.

[26]

Sainyam Galhotra, Yuriy Brun, and Alexandra Meliou. 2017. Fairness testing: Testing software for discrimination. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. 498–510.

Digital Library

[27]

Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford. 2021. Datasheets for datasets. Commun. ACM 64, 12 (2021), 86–92.

Digital Library

[28]

Ben Green. 2021. Data science as political action: Grounding data science in a politics of justice. Journal of Social Computing 2, 3 (2021), 249–265.

[29]

Aaron Halfaker and R Stuart Geiger. 2020. Ores: Lowering barriers with participatory machine learning in wikipedia. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2(2020), 1–37.

Digital Library

[30]

Aaron Halfaker and John Riedl. 2012. Bots and cyborgs: Wikipedia’s immune system. Computer 45, 03 (2012), 79–82.

Digital Library

[31]

Christina Harrington, Sheena Erete, and Anne Marie Piper. 2019. Deconstructing community-based collaborative design: Towards more equitable participatory design engagements. Proceedings of the ACM on Human-Computer Interaction 3, CSCW(2019), 1–25.

Digital Library

[32]

Brent Hecht and Darren Gergle. 2010. The tower of Babel meets web 2.0: user-generated content and its applications in a multilingual context. In Proceedings of the 2010 CHI Conference on Human Factors in Computing Systems. 291–300.

Digital Library

[33]

Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miro Dudik, and Hanna Wallach. 2019. Improving fairness in machine learning systems: What do industry practitioners need?. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–16.

Digital Library

[34]

Yen-Chia Hsu, Ting-Hao ‘Kenneth’ Huang, Himanshu Verma, Andrea Mauri, Illah Nourbakhsh, and Alessandro Bozzon. 2022. Empowering local communities using artificial intelligence. Patterns 3, 3 (2022), 100449.

[35]

Jane Im, Amy X. Zhang, Christopher J. Schilling, and David Karger. 2018. Deliberation and resolution on Wikipedia: A Case Study of Requests for Comments. Proceedings of the ACM on Human-Computer Interaction 2, CSCW(2018), 1–24.

Digital Library

[36]

Anson Kahng, Min Kyung Lee, Ritesh Noothigattu, Ariel Procaccia, and Christos-Alexandros Psomas. 2019. Statistical foundations of virtual democracy. In Proceedings of the 36th International Conference on Machine Learning. PMLR 97:3173–3182.

[37]

Michael Katell, Meg Young, Dharma Dailey, Bernease Herman, Vivian Guetler, Aaron Tam, Corinne Bintz, Daniella Raz, and PM Krafft. 2020. Toward situated interventions for algorithmic equity: Lessons from the field. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 45–55.

Digital Library

[38]

Jon Kleinberg. 2018. Inherent trade-offs in algorithmic fairness. In Abstracts of the 2018 ACM International Conference on Measurement and Modeling of Computer Systems. 40.

Digital Library

[39]

Jon Kleinberg, Jens Ludwig, Sendhil Mullainathan, and Ashesh Rambachan. 2018. Algorithmic fairness. In Aea papers and proceedings, Vol. 108. 22–27.

[40]

PM Krafft, Meg Young, Michael Katell, Jennifer E Lee, Shankar Narayan, Micah Epstein, Dharma Dailey, Bernease Herman, Aaron Tam, Vivian Guetler, 2021. An action-oriented AI policy toolkit for technology audits by community advocates and activists. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 772–781.

Digital Library

[41]

Bogdan Kulynych, David Madras, Smitha Milli, Inioluwa Deborah Raji, Angela Zhou, and Richard Zemel. 2020. Participatory approaches to machine learning. In International Conference on Machine Learning Workshop.

[42]

Bogdan Kulynych, Rebekah Overdorf, Carmela Troncoso, and Seda Gürses. 2020. POTs: Protective optimization technologies. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 177–188.

Digital Library

[43]

Min Kyung Lee, Daniel Kusbit, Anson Kahng, Ji Tae Kim, Xinran Yuan, Allissa Chan, Daniel See, Ritesh Noothigattu, Siheon Lee, Alexandros Psomas, 2019. WeBuildAI: Participatory framework for algorithmic governance. Proceedings of the ACM on Human-Computer Interaction 3, CSCW(2019), 1–35.

Digital Library

[44]

Michael Madaio, Lisa Egede, Hariharan Subramonyam, Jennifer Wortman Vaughan, and Hanna Wallach. 2022. Assessing the Fairness of AI Systems: AI Practitioners’ Processes, Challenges, and Needs for Support. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1(2022), 1–26.

Digital Library

[45]

Michael A Madaio, Luke Stark, Jennifer Wortman Vaughan, and Hanna Wallach. 2020. Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.

Digital Library

[46]

MediaWiki. 2021. ORES – MediaWiki. https://www.mediawiki.org/w/index.php?title=ORES&oldid=4901890

[47]

Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 220–229.

Digital Library

[48]

Chantal Mouffe. 1999. Deliberative democracy or agonistic pluralism?Social research (1999), 745–758.

[49]

Otter.ai. 2020. Otter.ai, Inc.https://otter.ai

[50]

Jennifer Pierre, Roderic Crooks, Morgan Currie, Britt Paris, and Irene Pasquetto. 2021. Getting Ourselves Together: Data-centered participatory design research & epistemic burden. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–11.

Digital Library

[51]

Inioluwa Deborah Raji, Andrew Smart, Rebecca N White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020. Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 33–44.

Digital Library

[52]

Brianna Richardson, Jean Garcia-Gathright, Samuel F Way, Jennifer Thom, and Henriette Cramer. 2021. Towards fairness in practice: A practitioner-oriented rubric for evaluating Fair ML Toolkits. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[53]

Samantha Robertson, Tonya Nguyen, and Niloufar Salehi. 2021. Modeling assumptions clash with the real world: Transparency, equity, and community challenges for student assignment algorithms. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14.

Digital Library

[54]

Yda Schreuder. 2001. The Polder model in Dutch economic and environmental planning. Bulletin of Science, Technology & Society 21, 4 (2001), 237–245.

[55]

Hetan Shah. 2018. Algorithmic accountability. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 376, 2128 (2018), 20170362.

[56]

Hong Shen, Wesley H Deng, Aditi Chattopadhyay, Zhiwei Steven Wu, Xu Wang, and Haiyi Zhu. 2021. Value Cards: An Educational Toolkit for Teaching Social Impacts of Machine Learning through Deliberation. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 850–861.

Digital Library

[57]

Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2(2021), 1–29.

Digital Library

[58]

Hong Shen, Ángel Alexander Cabrera, Adam Perer, and Jason Hong. 2020. “Public(s)-in-the-Loop”: Facilitating Deliberation of Algorithmic Decisions in Contentious Public Policy Domains. In Fair & Responsible AI Workshop CHI2020.

[59]

Mona Sloane, Emanuel Moss, Olaitan Awomolo, and Laura Forlano. 2020. Participation is not a design fix for machine learning. In Proceedings of the International Conference on Machine Learning. 1–7.

[60]

C Estelle Smith, Bowen Yu, Anjali Srivastava, Aaron Halfaker, Loren Terveen, and Haiyi Zhu. 2020. Keeping community in the loop: Understanding wikipedia stakeholder values for machine learning-based systems. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.

Digital Library

[61]

Megha Srivastava, Hoda Heidari, and Andreas Krause. 2019. Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining(2019).

Digital Library

[62]

Laura Stein. 2013. Policy and participation on social media: The cases of YouTube, Facebook, and Wikipedia. Communication, Culture & Critique 6, 3 (2013), 353–371.

[63]

Berk Ustun, Alexander Spangher, and Yang Liu. 2019. Actionable recourse in linear classification. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 10–19.

Digital Library

[64]

Michael Veale, Max Van Kleek, and Reuben Binns. 2018. Fairness and accountability design needs for algorithmic support in high-stakes public sector decision-making. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–14.

Digital Library

[65]

Miaomiao Wen, Keith Maki, Steven Dow, James D Herbsleb, and Carolyn Rose. 2017. Supporting virtual team formation through community-wide deliberation. Proceedings of the ACM on Human-Computer Interaction 1, CSCW(2017), 1–19.

Digital Library

[66]

James Wexler, Mahima Pushkarna, Tolga Bolukbasi, Martin Wattenberg, Fernanda Viégas, and Jimbo Wilson. 2019. The What-If tool: Interactive probing of machine learning models. IEEE Transactions on Visualization and Computer Graphics 26, 1(2019), 56–65.

[67]

Pak-Hang Wong. 2020. Democratizing algorithmic fairness. Philosophy & Technology 33, 2 (2020), 225–244.

[68]

Qian Yang, Aaron Steinfeld, Carolyn Rosé, and John Zimmerman. 2020. Re-examining whether, why, and how human-AI interaction is uniquely difficult to design. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[69]

Yiwei Yang, Eser Kandogan, Yunyao Li, Prithviraj Sen, and Walter S Lasecki. 2019. A study on interaction in human-in-the-loop machine learning for text analytics. In Joint Proceedings of the ACM IUI 2019 Workshops.

[70]

Zining Ye, Xinran Yuan, Shaurya Gaur, Aaron Halfaker, Jodi Forlizzi, and Haiyi Zhu. 2021. Wikipedia ORES Explorer: Visualizing Trade-offs For Designing Applications With Machine Learning API. In Proceedings of the 2021 ACM on Designing Interactive Systems Conference. 1554–1565.

Digital Library

[71]

Bowen Yu, Ye Yuan, Loren Terveen, Zhiwei Steven Wu, Jodi Forlizzi, and Haiyi Zhu. 2020. Keeping designers in the loop: Communicating inherent algorithmic trade-offs across multiple objectives. In Proceedings of the 2020 ACM on Designing Interactive Systems Conference. 1245–1257.

Digital Library

[72]

Haiyi Zhu, Bowen Yu, Aaron Halfaker, and Loren Terveen. 2018. Value-sensitive algorithm design: Method, case study, and lessons. Proceedings of the ACM on Human-Computer Interaction 2, CSCW(2018), 1–23.

Digital Library

Cited By

Kim SHo JLi YFan BYang WRamey JFox SZhu HZimmerman JEslami M(2024)Integrating Equity in Public Sector Data-Driven Decision Making: Exploring the Desired Futures of Underserved StakeholdersProceedings of the ACM on Human-Computer Interaction10.1145/36869058:CSCW2(1-39)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686905
Sarkar PLiem C(2024)"It's the most fair thing to do but it doesn't make any sense": Perceptions of Mathematical Fairness Notions by Hiring ProfessionalsProceedings of the ACM on Human-Computer Interaction10.1145/36373608:CSCW1(1-35)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637360
Eyuboglu SGoel KDesai AChen LMonfort MRé CZou J(2024)Model ChangeLists: Characterizing Updates to ML ModelsProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3659047(2432-2453)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3659047
Show More Cited By

Index Terms

The Model Card Authoring Toolkit: Toward Community-centered, Deliberation-driven AI Design
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing theory, concepts and paradigms
  2. Human computer interaction (HCI)

Index terms have been assigned to the content through auto-classification.

Recommendations

Toward a Community-Oriented Design of Internet Platforms

Building user communities can contribute significantly to the success of an on-line platform. The literature on establishing on-line communities and on extending existing platforms with functionalities that enable community building emphasizes the ...
Community Design: growing one's own information infrastructure
PDC '08: Proceedings of the Tenth Anniversary Conference on Participatory Design 2008

This paper examines the phenomenon of Community Design. It is a radical phenomenon in that community members collectively grow their own community information infrastructures without the intervention of professionals typically associated with such ...
Participatory design in community computing contexts: tales from the field
PDC 04: Proceedings of the eighth conference on Participatory design: Artful integration: interweaving media, materials and practices - Volume 1

As technology becomes more embedded in our daily lives, there is a great deal of hope about the use of information technology to achieve positive community outcomes like increasing access to local information, promoting civic engagement, and creating ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

FAccT '22: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency

June 2022

2351 pages

ISBN:9781450393522

DOI:10.1145/3531146

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

ACM: Association for Computing Machinery

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 June 2022

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

NSF (National Science Foundation)

Conference

FAccT '22

Sponsor:

ACM

FAccT '22: 2022 ACM Conference on Fairness, Accountability, and Transparency

June 21 - 24, 2022

Seoul, Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
1,973
Total Downloads

Downloads (Last 12 months)797
Downloads (Last 6 weeks)120

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kim SHo JLi YFan BYang WRamey JFox SZhu HZimmerman JEslami M(2024)Integrating Equity in Public Sector Data-Driven Decision Making: Exploring the Desired Futures of Underserved StakeholdersProceedings of the ACM on Human-Computer Interaction10.1145/36869058:CSCW2(1-39)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686905
Sarkar PLiem C(2024)"It's the most fair thing to do but it doesn't make any sense": Perceptions of Mathematical Fairness Notions by Hiring ProfessionalsProceedings of the ACM on Human-Computer Interaction10.1145/36373608:CSCW1(1-35)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637360
Eyuboglu SGoel KDesai AChen LMonfort MRé CZou J(2024)Model ChangeLists: Characterizing Updates to ML ModelsProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3659047(2432-2453)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3659047
Tang NZhi JKuo TKainaroi CNorthup JHolstein KZhu HHeidari HShen H(2024)AI Failure Cards: Understanding and Supporting Grassroots Efforts to Mitigate AI Failures in Homeless ServicesProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658935(713-732)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658935
Xiao ZDeng WLam MEslami MKim JLee MLiao Q(2024)Human-Centered Evaluation and Auditing of Language ModelsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3636302(1-6)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3636302
Harrison GBryson KBamba ADovichi LBinion ABorem AUr B(2024)JupyterLab in Retrograde: Contextual Notifications That Highlight Fairness and Bias Issues for Data ScientistsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642755(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642755
Alvarado Garcia AWong-Villacres MHernández BLe Dantec C(2024)Bitacora: A Toolkit for Supporting NonProfits to Critically Reflect on Social Media Data UseProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642673(1-29)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642673
Berman GGoyal NMadaio M(2024)A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness EvaluationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642398(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642398
Shi ZZhi JZeng SZhang ZKapoor AHudson SShen HFang FChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Predicting and Presenting Task Difficulty for Crowdsourcing Food Rescue PlatformsProceedings of the ACM Web Conference 202410.1145/3589334.3648155(4686-4696)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3648155
Goldsteen AFarkash AHind M(2024)Assessing and implementing trustworthy AI across multiple dimensionsEthics in Online AI-based Systems10.1016/B978-0-443-18851-0.00001-9(229-257)Online publication date: 2024
https://doi.org/10.1016/B978-0-443-18851-0.00001-9
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents