[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3551624.3555296acmconferencesArticle/Chapter ViewAbstractPublication PageseaamoConference Proceedingsconference-collections
research-article

It’s Not Fairness, and It’s Not Fair: The Failure of Distributional Equality and the Promise of Relational Equality in Complete-Information Hiring Games

Published: 17 October 2022 Publication History

Abstract

Existing efforts to formulate computational definitions of fairness have largely focused on distributional notions of equality, defined through how resources or decisions are divided. Yet existing discrimination is often the result of unequal social relations, rather than simply an unequal distribution of resources. We show how optimizing for existing computational definitions of fairness fails to prevent unequal social relations by providing an example of a self-confirming equilibrium in a simple hiring market that is relationally unequal but satisfies existing distributional notions of fairness. We introduce a notion of blatant relational unfairness for complete-information games, and discuss how this definition helps initiate a new approach to incorporating relational equality into computational systems.

References

[1]
Mohsen Abbasi, Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. 2019. Fairness in representation: quantifying stereotyping as a representational harm. In Proceedings of the 2019 SIAM International Conference on Data Mining, SDM 2019, Calgary, Alberta, Canada, May 2-4, 2019, Tanya Y. Berger-Wolf and Nitesh V. Chawla (Eds.). SIAM, 801–809. https://doi.org/10.1137/1.9781611975673.90
[2]
Philip E. Agre. 1997. Toward a Critical Technical Practice: Lessons Learned in Trying to Reform AI. In Social Science, Technical Systems and Cooperative Work: The Great Divide, Geoffrey C. Bowker, Les Gasser, Susan Leigh Star, and Bill Turner (Eds.). Hillsdale, NJ, 131–158.
[3]
Elizabeth Anderson. 2007. Fair Opportunity in Education: A Democratic Equality Perspective. Ethics 117 (July 2007), 595–622.
[4]
Elizabeth Anderson. 2008. I—Elizabeth Anderson: Expanding the Egalitarian Toolbox: Equality and Bureaucracy. Aristotelian Society Supplementary Volume 82, 1 (June 2008), 139–160.
[5]
Elizabeth Anderson. 2013. The Fundamental Disagreement between Luck Egalitarians and Relational Egalitarians. Canadian Journal of Philosophy 40, sup1 (July 2013), 1–23.
[6]
Elizabeth Anderson. 2015. Liberty, Equality, and Private Government. The Tanner Lectures in Human Values, Princeton University.
[7]
Elizabeth S Anderson. 1999. What Is the Point of Equality?Ethics 109, 2 (Jan. 1999), 287–337.
[8]
Kenneth Arrow. 1971. The Theory of Discrimination. In Discrimination in Labor Markets. Princeton University, 1–37.
[9]
Kenneth J Arrow. 1998. What has economics to say about racial discrimination?Journal of economic perspectives 12, 2 (1998), 91–100.
[10]
Chelsea Barabas, Madars Virza, Karthik Dinakar, Joichi Ito, and Jonathan Zittrain. 2018. Interventions over Predictions - Reframing the Ethical Debate for Actuarial Risk Assessment. In FAT* 2018.
[11]
Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2019. Fairness and Machine Learning. fairmlbook.org. http://www.fairmlbook.org.
[12]
Sebastian Benthall and Bruce D Haynes. 2019. Racial categories in machine learning. In Conference on Fairness, Accountability, and Transparency 2019. ACM Press, New York, New York, USA, 289–298.
[13]
Marianne Bertrand and Sendhil Mullainathan. 2004. Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. American economic review 94, 4 (2004), 991–1013.
[14]
Rueben Binns. 2018. Fairness in Machine Learning: Lessons from Political Philosophy. Proceedings of Machine Learning Research 81 (2018), 1–11.
[15]
Abeba Birhane. 2021. Algorithmic injustice: a relational ethics approach. Patterns 2, 2 (2021), 100205. https://doi.org/10.1016/j.patter.2021.100205
[16]
Miranda Bogen and Aaron Rieke. 2018. Help wanted: An examination of hiring algorithms, equity, and bias. (2018).
[17]
Alexandra Chouldechova. 2017. Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments. Big Data 5, 2 (2017), 153–163. https://doi.org/10.1089/big.2016.0047
[18]
G. A. Cohen. 1989. On the Currency of Egalitarian Justice. Ethics 99, 4 (July 1989), 906–944.
[19]
Sam Corbett-Davies and Sharad Goel. 2018. The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning. (Sept. 2018), 1–25.
[20]
Sasha Costanza-Chock. 2018. Design Justice: towards an intersectional feminist framework for design theory and practice. In Design Research Society. 1–14.
[21]
Bo Cowgill and Catherine E Tucker. 2019. Economics, fairness and algorithmic bias. preparation for: Journal of Economic Perspectives (2019).
[22]
Alexander D’Amour, Hansa Srinivasan, James Atwood, Pallavi Baljekar, D. Sculley, and Yoni Halpern. 2020. Fairness is not static: deeper understanding of long term fairness via simulation studies. In FAT* ’20: Conference on Fairness, Accountability, and Transparency, Barcelona, Spain, January 27-30, 2020, Mireille Hildebrandt, Carlos Castillo, L. Elisa Celis, Salvatore Ruggieri, Linnet Taylor, and Gabriela Zanfir-Fortuna (Eds.). ACM, 525–534. https://doi.org/10.1145/3351095.3372878
[23]
Lina Dencik, Arne Hintz, Joanna Redden, and Emiliano Treré. 2019. Exploring Data Justice: Conceptions, Applications and Directions. Information, Communication & Society 22, 7 (May 2019), 873–881.
[24]
Emily Denton, Alex Hanna, Razvan Amironesei, Andrew Smart, and Hilary Nicole. 2021. On the Genealogy of Machine Learning Datasets: A Critical History of ImageNet. Big Data and Society(2021).
[25]
Irit Dinur and Kobbi Nissim. 2003. Revealing information while preserving privacy. In Proceedings of the Twenty-Second ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, June 9-12, 2003, San Diego, CA, USA, Frank Neven, Catriel Beeri, and Tova Milo (Eds.). ACM, 202–210. https://doi.org/10.1145/773153.773173
[26]
Andrea Doucet. 2015. Parental Responsibilities: Dilemmas of Measurement and Gender Equality. Journal of Marriage and Family 77, 1 (2015), 224?242.
[27]
Mengnan Du, Fan Yang, Na Zou, and Xia Hu. 2020. Fairness in deep learning: A computational perspective. IEEE Intelligent Systems(2020).
[28]
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2011. Fairness Through Awareness. (Nov. 2011), 1–24.
[29]
Cynthia Dwork and Aaron Roth. 2014. The Algorithmic Foundations of Differential Privacy. Found. Trends Theor. Comput. Sci. 9, 3-4 (2014), 211–407. https://doi.org/10.1561/0400000042
[30]
Ronald Dworkin. 1981. What is Equality? Part 1: Equality of Welfare. Philosophy & Public Affairs 10, 3 (1981), 185–246.
[31]
Hadi Elzayn, Shahin Jabbari, Christopher Jung, Michael J. Kearns, Seth Neel, Aaron Roth, and Zachary Schutzman. 2019. Fair Algorithms for Learning in Allocation Problems. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* 2019, Atlanta, GA, USA, January 29-31, 2019, danah boyd and Jamie H. Morgenstern (Eds.). ACM, 170–179. https://doi.org/10.1145/3287560.3287571
[32]
Virgina Eubanks. 2018. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. St. Martin’s Press, New York.
[33]
Hanming Fang and Andrea Moro. 2011. Theories of statistical discrimination and affirmative action: A survey. Handbook of social economics 1 (2011), 133–200.
[34]
Benjamin Fish, Ashkan Bashardoust, danah boyd, Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. 2019. Gaps in Information Access in Social Networks. In The World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019. 480–490.
[35]
Benjamin Fish and Luke Stark. 2021. Reflexive Design for Fairness and Other Human Values in Formal Models. In AIES ’21: AAAI/ACM Conference on AI, Ethics, and Society, Virtual Event, USA, May 19-21, 2021, Marion Fourcade, Benjamin Kuipers, Seth Lazar, and Deirdre K. Mulligan (Eds.). ACM, 89–99. https://doi.org/10.1145/3461702.3462518
[36]
Batya Friedman and Helen Nissenbaum. 1996. Bias in Computer Systems. ACM Transactions on Information Systems 14, 3 (Sept. 1996), 330–347.
[37]
Drew Fudenberg and David K Levine. 1993. Self-confirming equilibrium. Econometrica: Journal of the Econometric Society (1993), 523–545.
[38]
Oscar H Gandy Jr and Selena Nemorin. 2019. Toward a political economy of nudge: smart city variations. Information, Communication & Society 22, 14 (2019), 2112–2126.
[39]
Stephen Gillen, Christopher Jung, Michael J. Kearns, and Aaron Roth. 2018. Online Learning with an Unknown Fairness Metric. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (Eds.). 2605–2614. https://proceedings.neurips.cc/paper/2018/hash/50905d7b2216bfeccb5b41016357176b-Abstract.html
[40]
Moritz Hardt, Eric Price, and Nathan Srebro. 2016. Equality of Opportunity in Supervised Learning. In 30th Conference on Neural Information Processing Systems. Barcelona, 1–9.
[41]
Hoda Heidari, Claudio Ferrari, Krishna P. Gummadi, and Andreas Krause. 2018. Fairness Behind a Veil of Ignorance: A Welfare Analysis for Automated Decision Making. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (Eds.). 1273–1283. https://proceedings.neurips.cc/paper/2018/hash/be3159ad04564bfb90db9e32851ebf9c-Abstract.html
[42]
Hoda Heidari, Michele Loi, Krishna P. Gummadi, and Andreas Krause. 2018. A Moral Framework for Understanding of Fair ML through Economic Models of Equality of Opportunity. (Sept. 2018), 1–13. arxiv:1809.03400
[43]
Anna Lauren Hoffmann. 2018. Data Violence and How Bad Engineering Choices Can Damage Society. https://medium.com/s/story/data-violence-and-how-bad-engineering-choices-can-damage-society-39e44150e1d4
[44]
Anna Lauren Hoffmann. 2019. Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse. Information, Communication & Society 22, 7 (May 2019), 900–915.
[45]
Lily Hu. 2020. Direct effects. Phenomenal World (2020). https://phenomenalworld.org/analysis/direct-effects
[46]
Lily Hu and Issa Kohler-Hausmann. 2020. What’s sex got to do with fair machine learning?arXiv preprint arXiv:2006.01770(2020).
[47]
Ben Hutchinson and Margaret Mitchell. 2019. 50 Years of Test (Un)fairness. In Conference on Fairness, Accountability, and Transparency 2019. ACM Press, New York, New York, USA, 49–58.
[48]
Christina Ilvento. 2020. Metric Learning for Individual Fairness. In 1st Symposium on Foundations of Responsible Computing, FORC 2020, June 1-3, 2020, Harvard University, Cambridge, MA, USA (virtual conference)(LIPIcs, Vol. 156), Aaron Roth (Ed.). Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2:1–2:11. https://doi.org/10.4230/LIPIcs.FORC.2020.2
[49]
Nassim JafariNaimi, Lisa Nathan, and Ian Hargraves. 2015. Values as Hypotheses: Design, Inquiry, and the Service of Values. Design Issues 31, 4 (Oct. 2015), 91–104.
[50]
Christopher Jung, Michael J. Kearns, Seth Neel, Aaron Roth, Logan Stapleton, and Zhiwei Steven Wu. 2019. Eliciting and Enforcing Subjective Individual Fairness. CoRR abs/1905.10660(2019). arxiv:1905.10660http://arxiv.org/abs/1905.10660
[51]
Faisal Kamiran and Toon Calders. 2009. Classifying without discriminating. In 2009 2nd International Conference on Computer, Control and Communication. IEEE, 1–6.
[52]
Maximilian Kasy and Rediet Abebe. 2021. Fairness, Equality, and Power in Algorithmic Decision-Making. In FAccT ’21: 2021 ACM Conference on Fairness, Accountability, and Transparency, Virtual Event / Toronto, Canada, March 3-10, 2021, Madeleine Clare Elish, William Isaac, and Richard S. Zemel (Eds.). ACM, 576–586. https://doi.org/10.1145/3442188.3445919
[53]
Jon M. Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2017. Inherent Trade-Offs in the Fair Determination of Risk Scores. In 8th Innovations in Theoretical Computer Science Conference, ITCS 2017, January 9-11, 2017, Berkeley, CA, USA(LIPIcs, Vol. 67), Christos H. Papadimitriou (Ed.). Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 43:1–43:23. https://doi.org/10.4230/LIPIcs.ITCS.2017.43
[54]
Issa Kohler-Hausmann. 2018. Eddie Murphy and the dangers of counterfactual causal thinking about detecting racial discrimination. Nw. UL Rev. 113(2018), 1163.
[55]
Matt J. Kusner, Joshua R. Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual Fairness. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 4066–4076. https://proceedings.neurips.cc/paper/2017/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html
[56]
Min Kyung Lee. 2018. Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management. Big Data & Society 5, 1 (March 2018), 205395171875668–16.
[57]
Lydia T. Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. 2019. Delayed Impact of Fair Machine Learning. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, Sarit Kraus (Ed.). ijcai.org, 6196–6200. https://doi.org/10.24963/ijcai.2019/862
[58]
George Lowther. 2016. The Section Theorems. https://almostsuremath.com/2016/11/29/the-section-theorems/.
[59]
Hussein Mouzannar, Mesrob I. Ohannessian, and Nathan Srebro. 2019. From Fair Decision Making To Social Equality. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* 2019, Atlanta, GA, USA, January 29-31, 2019, danah boyd and Jamie H. Morgenstern (Eds.). ACM, 359–368. https://doi.org/10.1145/3287560.3287599
[60]
Sora Park and Justine Humphry. 2019. Exclusion by design: intersections of social, digital and data exclusion. Information, Communication & Society 22, 7 (May 2019), 934–953.
[61]
Samir Passi and Solon Barocas. 2019. Problem Formulation and Fairness. In Conference on Fairness, Accountability, and Transparency 2019. ACM Press, New York, New York, USA, 39–48.
[62]
Judea Pearl. 2009. Causal inference in statistics: An overview. Statistics surveys 3(2009), 96–146.
[63]
Edmund S Phelps. 1972. The Statistical Theory of Racism and Sexism. The American Economic Review 62, 4 (Sept. 1972), 659–661.
[64]
Manish Raghavan, Solon Barocas, Jon M. Kleinberg, and Karen Levy. 2020. Mitigating bias in algorithmic hiring: evaluating claims and practices. In FAT* ’20: Conference on Fairness, Accountability, and Transparency, Barcelona, Spain, January 27-30, 2020, Mireille Hildebrandt, Carlos Castillo, L. Elisa Celis, Salvatore Ruggieri, Linnet Taylor, and Gabriela Zanfir-Fortuna (Eds.). ACM, 469–481. https://doi.org/10.1145/3351095.3372828
[65]
Gokhan Savas. 2010. Social Inequality at Low-wage Work in Neo-liberal Economy: The Case of Women of Color Domestic Workers in The United States. Race, Gender & Class 17, 3/4 (2010), 314?326.
[66]
Samuel Scheffler. 2013. Equality and Tradition: Questions of Value in Moral and Political Theory. Oxford University Press, Oxford and New York.
[67]
Christian Schemmel. 2011. Distributive and relational equality. Politics, Philosophy & Economics 11, 2 (Sept. 2011), 123–148.
[68]
Morgan Klaus Scheuerman, Emily Denton, and Alex Hanna. 2021. Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development. In Proceedings of the 2021 Conference on Computer Supported Cooperative Work (CSCW ?21). https://arxiv.org/abs/2108.04308
[69]
Andrew D. Selbst, danah boyd, Sorelle A. Friedler, Suresh Venkatasubramanian, and Janet Vertesi. 2019. Fairness and Abstraction in Sociotechnical Systems. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* 2019, Atlanta, GA, USA, January 29-31, 2019, danah boyd and Jamie H. Morgenstern (Eds.). ACM, 59–68. https://doi.org/10.1145/3287560.3287598
[70]
Sahil Verma and Julia Rubin. 2018. Fairness definitions explained. In 2018 ACM/IEEE International Workshop on Software Fairness. ACM Press, New York, New York, USA, 1–7.
[71]
Salomé Viljoen. 2020. Democratic data: A relational theory for data governance. Available at SSRN 3727562(2020).
[72]
Michael Walzer. 1983. Spheres of Justice: A Defense of Pluralism and Equality. Basic Books, New York, NY.
[73]
Allison Woodruff, Sarah E Fox, Steven Rousso-Schindler, and Jeffrey Warshaw. 2018. A Qualitative Exploration of Perceptions of Algorithmic Fairness. In Extended Abstracts of the 2018 CHI Conference. ACM Press, New York, New York, USA, 1–14.
[74]
Shoshana Zuboff. 2019. The Age of Surveillance Capitalism. PublicAffairs/Hachette, New York.

Cited By

View all
  • (2024)"It's the most fair thing to do but it doesn't make any sense": Perceptions of Mathematical Fairness Notions by Hiring ProfessionalsProceedings of the ACM on Human-Computer Interaction10.1145/36373608:CSCW1(1-35)Online publication date: 26-Apr-2024
  • (2024)Algorithmic Harms and Algorithmic WrongsThe 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3659001(1725-1732)Online publication date: 5-Jun-2024
  • (2024)More than the Sum of its Parts: Susceptibility to Algorithmic Disadvantage as a Conceptual FrameworkProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658944(909-919)Online publication date: 3-Jun-2024

Index Terms

  1. It’s Not Fairness, and It’s Not Fair: The Failure of Distributional Equality and the Promise of Relational Equality in Complete-Information Hiring Games
    Index terms have been assigned to the content through auto-classification.

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    EAAMO '22: Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization
    October 2022
    239 pages
    ISBN:9781450394772
    DOI:10.1145/3551624
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 17 October 2022

    Permissions

    Request permissions for this article.

    Check for updates

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    EAAMO '22
    Sponsor:

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)72
    • Downloads (Last 6 weeks)6
    Reflects downloads up to 11 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)"It's the most fair thing to do but it doesn't make any sense": Perceptions of Mathematical Fairness Notions by Hiring ProfessionalsProceedings of the ACM on Human-Computer Interaction10.1145/36373608:CSCW1(1-35)Online publication date: 26-Apr-2024
    • (2024)Algorithmic Harms and Algorithmic WrongsThe 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3659001(1725-1732)Online publication date: 5-Jun-2024
    • (2024)More than the Sum of its Parts: Susceptibility to Algorithmic Disadvantage as a Conceptual FrameworkProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658944(909-919)Online publication date: 3-Jun-2024

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media