More Web Proxy on the site http://driver.im/

research-article

Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

Authors:

Jeffrey Sorensen,

Lucy VassermanAuthors Info & Claims

WWW '19: Companion Proceedings of The 2019 World Wide Web Conference

Pages 491 - 500

https://doi.org/10.1145/3308560.3317593

Published: 13 May 2019 Publication History

Abstract

Unintended bias in Machine Learning can manifest as systemic differences in performance for different demographic groups, potentially compounding existing challenges to fairness in society at large. In this paper, we introduce a suite of threshold-agnostic metrics that provide a nuanced view of this unintended bias, by considering the various ways that a classifier’s score distribution can vary across designated groups. We also introduce a large new test set of online comments with crowd-sourced annotations for identity references. We use this to show how our metrics can be used to find new and potentially subtle unintended bias in existing public models.

References

[1]

Alex Beutel, Jilin Chen, Zhe Zhao, and Ed H. Chi. 2017. Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations. CoRR abs/1707.00075(2017). http://arxiv.org/abs/1707.00075

[2]

Daniel Borkan, Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019. Limitations of Pinned AUC for Measuring Unintended Bias. CoRR (2019).

[3]

Tim Brennan, William Dieterich, and Beate Ehret. 2009. Evaluating the Predictive Validity of the Compas Risk and Needs Assessment System. Criminal Justice and Behavior 36, 1 (2009), 21–40. arXiv:https://doi.org/10.1177/0093854808326545

[4]

Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research), Sorelle A. Friedler and Christo Wilson (Eds.), Vol. 81. PMLR, New York, NY, USA, 77–91. http://proceedings.mlr.press/v81/buolamwini18a.html

[5]

Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018. Measuring and Mitigating Unintended Bias in Text Classification. In Proceedings of AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society.

Digital Library

[6]

Yanai Elazar and Yoav Goldberg. 2018. Adversarial Removal of Demographic Attributes from Text Data. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11–21. http://aclweb.org/anthology/D18-1002

[7]

Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015. Certifying and Removing Disparate Impact. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining(KDD ’15). ACM, New York, NY, USA, 259–268.

Digital Library

[8]

Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. 2016. On the (im)possibility of fairness. CoRR abs/1609.07236(2016). http://arxiv.org/abs/1609.07236

[9]

Sahaj Garg, Vincent Perot, Nicole Limtiaco, Ankur Taly, Ed H. Chi, and Alex Beutel. 2018. Counterfactual Fairness in Text Classification through Robustness. CoRR abs/1809.10610(2018). arxiv:1809.10610http://arxiv.org/abs/1809.10610

[10]

Moritz Hardt, Eric Price, and Nathan Srebro. 2016. Equality of Opportunity in Supervised Learning. CoRR abs/1610.02413(2016). http://arxiv.org/abs/1610.02413

[11]

Jigsaw. 2017. Perspective API. https://www.perspectiveapi.com/

[12]

Jon M. Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2016. Inherent Trade-Offs in the Fair Determination of Risk Scores. CoRR abs/1609.05807(2016). http://arxiv.org/abs/1609.05807

[13]

Lucy Vasserman, John Li, CJ Adams, Lucas Dixon. 2018. Unintended bias and names of frequently targeted groups. https://medium.com/the-false-positive/unintended-bias-and-names-of-frequently-targeted-groups-8e0b81f80a23

[14]

S. J. Mason and N. E. Graham. {n. d.}. Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation. Quarterly Journal of the Royal Meteorological Society 128, 584 ({n. d.}), 2145–2166. arXiv:https://rmets.onlinelibrary.wiley.com/doi/pdf/10.1256/003590002320603584

[15]

Aditya Krishna Menon and Robert C Williamson. 2018. The cost of fairness in binary classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research), Sorelle A. Friedler and Christo Wilson (Eds.), Vol. 81. PMLR, New York, NY, USA, 107–118. http://proceedings.mlr.press/v81/menon18a.html

[16]

Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model Cards for Model Reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency(FAT* ’19). ACM, New York, NY, USA, 220–229.

Digital Library

[17]

Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing Gender Bias in Abusive Language Detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2799–2804. http://aclweb.org/anthology/D18-1302

[18]

Nithum Thain, Lucas Dixon, and Ellery Wulczyn. 2017. Wikipedia Talk Labels: Toxicity. (2 2017).

[19]

Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex machina: Personal attacks seen at scale. In Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 1391–1399.

Digital Library

Cited By

Yang CHong YLewis GWu TKästner CFilkov VRay BZhou M(2024)What Is Wrong with My Model? Identifying Systematic Problems with Semantic Data SlicingProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695033(306-318)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695033
Yang JJin HTang RHan XFeng QJiang HZhong SYin BHu X(2024)Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and BeyondACM Transactions on Knowledge Discovery from Data10.1145/364950618:6(1-32)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3649506
He YTian PXu RShen XZhang XCui PBaeza-Yates RBonchi F(2024)Model-Agnostic Random Weighting for Out-of-Distribution GeneralizationProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671762(1050-1061)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671762
Show More Cited By

Index Terms

Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

Index terms have been assigned to the content through auto-classification.

Recommendations

Exploring Nuanced Gender Perspectives within the HCI Community
IndiaHCI '14: Proceedings of the 6th Indian Conference on Human-Computer Interaction

With the increasing spread and pervasiveness of technologies, the role of gender in the design of these technologies is a topic of growing importance. Several conference panels and journal issues have focused on the contributions feminism, gender theory,...
Measuring Metrics
CIKM '16: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management

You get what you measure, and you can't manage what you don't measure. Metrics are a powerful tool used in organizations to set goals, decide which new products and features should be released to customers, which new tests and experiments should be ...
Measuring the Discriminative Power of Object-Oriented Class Cohesion Metrics

Several object-oriented cohesion metrics have been proposed in the literature. These metrics aim to measure the relationship between class members, namely, methods and attributes. Different metrics use different models to represent the connectivity ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

WWW '19: Companion Proceedings of The 2019 World Wide Web Conference

May 2019

1331 pages

ISBN:9781450366755

DOI:10.1145/3308560

Editors:
Ling Liu
Georgia Tech, USA
,
Ryen White
Microsoft Research, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

IW3C2: International World Wide Web Conference Committee

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '19

WWW '19: The Web Conference

May 13 - 17, 2019

San Francisco, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

105
Total Citations
View Citations
1,166
Total Downloads

Downloads (Last 12 months)224
Downloads (Last 6 weeks)33

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yang CHong YLewis GWu TKästner CFilkov VRay BZhou M(2024)What Is Wrong with My Model? Identifying Systematic Problems with Semantic Data SlicingProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695033(306-318)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695033
Yang JJin HTang RHan XFeng QJiang HZhong SYin BHu X(2024)Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and BeyondACM Transactions on Knowledge Discovery from Data10.1145/364950618:6(1-32)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3649506
He YTian PXu RShen XZhang XCui PBaeza-Yates RBonchi F(2024)Model-Agnostic Random Weighting for Out-of-Distribution GeneralizationProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671762(1050-1061)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671762
Mahomed YCrawford CGautam SFriedler SMetaxa D(2024)Auditing GPT's Content Moderation Guardrails: Can ChatGPT Write Your Favorite TV Show?Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658932(660-686)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658932
Almuzaini APennock DSingh V(2024)Accuracy and Fairness for Web-Based Content Analysis under Temporal Shifts and Delayed LabelingProceedings of the 16th ACM Web Science Conference10.1145/3614419.3644028(268-278)Online publication date: 21-May-2024
https://dl.acm.org/doi/10.1145/3614419.3644028
Kuo THalfaker ACheng ZKim JWu MWu THolstein KZhu H(2024)Wikibench: Community-Driven Data Curation for AI Evaluation on WikipediaProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642278(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642278
Chen HGuo HChen BZhou XChua TNgo CKa-Wei Lee RKumar RLauw H(2024)OODREB: Benchmarking State-of-the-Art Methods for Out-Of-Distribution Generalization on Relation ExtractionProceedings of the ACM Web Conference 202410.1145/3589334.3645695(2294-2303)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645695
Tiwari RSivasubramanian DMekala ARamakrishnan GShenoy P(2024)Using Early Readouts to Mediate Featural Bias in Distillation2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00262(2626-2635)Online publication date: 3-Jan-2024
https://doi.org/10.1109/WACV57701.2024.00262
He XZannettou SShen YZhang Y(2024)You Only Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic Content2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00061(770-787)Online publication date: 19-May-2024
https://doi.org/10.1109/SP54263.2024.00061
Bahrami MSonoda RSrinivasan R(2024)LLM Diagnostic Toolkit: Evaluating LLMs for Ethical Issues2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650995(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650995
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents