[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
research-article

Conceptualising Contestability: Perspectives on Contesting Algorithmic Decisions

Published: 22 April 2021 Publication History

Abstract

As the use of algorithmic systems in high-stakes decision-making increases, the ability to contest algorithmic decisions is being recognised as an important safeguard for individuals. Yet, there is little guidance on what 'contestability'--the ability to contest decisions--in relation to algorithmic decision-making requires. Recent research presents different conceptualisations of contestability in algorithmic decision-making. We contribute to this growing body of work by describing and analysing the perspectives of people and organisations who made submissions in response to Australia's proposed 'AI Ethics Framework', the first framework of its kind to include 'contestability' as a core ethical principle. Our findings reveal that while the nature of contestability is disputed, it is seen as a way to protect individuals, and it resembles contestability in relation to human decision-making. We reflect on and discuss the implications of these findings.

References

[1]
Alkhatib, A. and M. Bernstein. 2019. Street--Level Algorithms: A Theory at the Gaps Between Policy and Decision. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1--13.
[2]
Andersen, K. B. 2018. Regeringen vil erklære 20.700 danskere på kontanthjælp klar til arbejde. from https://nyheder.tv2.dk/politik/2018-05--29-regeringen-vil-erklaere-20700-danskere-paa-kontanthjaelp-klar-til-arbejde
[3]
Bannister, F. and R. Connolly. 2014. ICT, public values and transformative government: A framework and programme for research. Government Information Quarterly, 31(1), 119--128.
[4]
Bannon, L. 2011. Reimagining HCI: Toward a More Human-Centered Perspective. Interactions, 50--57.
[5]
Barocas, S. and A. D. Selbst. 2016. Big Data's Disparate Impact. California Law Review, Inc., 671--732.
[6]
Beich, A. 2019. Jobcentre gør mennesker syge -- politikerne må handle nu. from https://jyllands-posten.dk/debat/kronik/ECE11390084/jobcentre-goer-mennesker-syge-politikerne-maa-handle-nu/
[7]
Benjamin, R. 2019. Race After Technology: Abolitionist Tools for the New Jim Code. Polity Press.
[8]
Berg, M. 1998. The Politics of Technology: On Bringing Social Theory into Technological Design. Science, Technology & Human Values, 23, 456--490.
[9]
Bergstrom, C. T. and J. D. West. 2020. Calling Bullshit: The Art of Scepticism in a Data-Driven World: Random House
[10]
Beskæftigelsesministeriet. 2016. Bekendtgørelse af lov om en aktiv beskæftigelsesindsats. from https://www.retsinformation.dk/forms/r0710.aspx?id=184891
[11]
Black, J. 2001. Managing Discretion. ARLC Conference Papers on Penalties: Policy, Principles and Practice in Government Regulation
[12]
Blomberg, J. and H. Karasti. 2013. Reflections on 25 years of ethnography in CSCW. Computer Supported Cooperative Work (CSCW), 22(4--6), 373--423. https://doi.org/10.1007/s10606-012-9183-1
[13]
Boulus-Rødje, N. 2018. In Search for the Perfect Pathway: Supporting Knowledge Work of Welfare Workers. Computer Supported Cooperative Work (CSCW), 27, 841--874. https://doi.org/10.1007/s10606-018-9318-0
[14]
Bourdieu, P. 1999. Site Effects. In P. Bourdieu, A. Accardo, G. Balazs, S. Beaud and F. Bonvin (Eds.), The Weight of the World (Vol. 2, pp. 123--180). Polity Press.
[15]
Bovens, M. and S. Zouridis. 2002. From Street-Level to System?Level Bureaucracies: How Information and Communication Technology is Transforming Administrative Discretion and Constitutional Control. Public Administration Review, 62(2), 174--184.
[16]
Bowers, J., G. Button, and W. Sharrock. 1995. Workflow From Within and Without: Technology and Cooperative Work on the Print Industry Shopfloor. In Proceedings of the European Conference on Computer-Supported Cooperative Work, 51--66.
[17]
Bowker, G. C. 2005. Memory Practices in the Sciences. MIT Press.
[18]
Bowker, G. C. and S. L. Star. 1999. Sorting Things Out: Classification and Its Consequences. MIT Press.
[19]
Bowker, G. C., S. L. Star, W. Turner, and L. Gasser. 2014. Social Science, Technical Systems, And Cooperative Work: Beyond the Great Divide (2 ed.). Psychology Press.
[20]
boyd, d.and K. Crawford. 2012. CRITICAL QUESTIONS FOR BIG DATA. Information, Communication & Society, 15(5), 662--679.
[21]
Brown, A., A. Chouldechova, E. Putnam-Hornstein, A. Tobin, and R. Vaithianathan. 2019. Toward Algorithmic Accountability in Public Services: a Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1--12.
[22]
Bucher, T. 2018. If?then: Algorithmic Power and Politics (Vol. 1). Oxford University Press.
[23]
Campolo, A.and K. Crawford. 2020. Enchanted Determinism: Power without Responsibility in Artificial Intelligence. Engaging Science, Technology, and Society, 6, 1--19.
[24]
Chiusi, F., S. Fischer, N. Kayser-Bril, and M. Spielkamp. 2020. Automating Society Report 2020. AlgorithmWatch.
[25]
Crabtree, A., D. M. Nichols, J. O'Brien, M. Rouncefield, and M. B. Twidale. 2000. Ethnomethodologically Informed Ethnography and Information System Design. Journal of the American Society for Information Science, 51(7), 666- 682.
[26]
Danish Agency for Labour Market and Recruitment. 2018. Active labour market policy measures. from https://www.star.dk/en/active-labour-market-policy-measures/
[27]
Dolata, M., B. Schenk, J. Fuhrer, A. Marti, and G. Schwabe. 2020. When the System does not Fit: Coping Strategies of Employment Consultants. Computer Supported Cooperative Work (CSCW), 1--40.
[28]
Douglas, J. D. 1967. Social Meanings of Suicide (Vol. 1). Princeton University Press.
[29]
Douglas, M. 1966. Purity and Danger: An analysis of the concepts of pollution and taboo (Vol. 1). Routledge
[30]
Douglas, M. 1986. How Institutions Think (1 ed.). Syracuse University Press.
[31]
Durkheim, É. and M. Mauss. 1903. De Quelques Formes Primitives de Classification. Contribution à l'étude des Représentations Collectives. L'Année Sociologique, 6, 1--72.
[32]
Elish, M. C. 2018. The Stakes of Uncertainty: Developing and Integrating Machine Learning in Clinical Care. In Ethnographic Praxis in Industry Conference Proceedings, 364--380.
[33]
Eubanks, V. 2017. Automating Inequality: How High-tech Tools Profile, Police, and Punish the Poor. St. Martin's Press.
[34]
Fitzgerald, R. and W. Housley. 2015. Advances in Membership Categorisation Analysis. Sage Publications Ltd.
[35]
Flügge, A. A., T. Hildebrandt, and N. H. Møller. 2021. Street-Level Algorithms and AI in Bureaucratic Decision-Making: A Caseworker Perspective. in Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), Article 40, 22. https://doi.org/10.1145/3449114
[36]
Flyverbom, M. 2019. The Digital Prism: Transparancy and Manageed Visibilities in a Datafied World. Cambridge University Press.
[37]
Friedman, B., P. H. Kahn, and A. Borning. 2008. Value sensitive design and information systems. The handbook of information and computer ethics, 69--101.
[38]
Garfinkel, H. 1967. Good organizational reasons for ?bad' clinic records. In H. Garfinkel (Ed.), Studies on Ethnomethodology (pp. 186--207). Prentice-Hall, Inc.
[39]
Garfinkel, H. 1967. Studies in Etnomethodology (Vol. 1). Blackwell Publishers Inc.
[40]
Garsten, C. and K. Jacobsson. 2013. Sorting people in and out: The plasticity of the categories of employability, work capacity and disability as technologies of government. ephemera: theory & politics in organization, 13(4), 825--850.
[41]
Gillespie, T. 2014. The Relevance of Algorithms. Media technologies: Essays on communication, materiality, and society, 1--32.
[42]
Hacking, I. 1985. Making Up People. In T. L. Heller, M. Sosna andD. E. Wellbery (Eds.), Reconstructing Individualism. Stanford University Press.
[43]
Hammersley, M. and P. Atkinson. 2007. Ethnography: Principles in practice (Vol. 3). Taylor & Francis: Routledge.
[44]
Harper, R. 2019. The Role of HCI in the Age of AI. International Journal of Human--Computer Interaction, 35(15), 1331--1344.
[45]
Harper, R., D. Randall, and W. Sharrock. 2016. Choice: The Sciences of Reason in the 21st Century: A Critical Assessment. Polity Press.
[46]
Henriksen, L. R. 2019. Jobcentret gør sygemeldte endnu mere syge. from https://www.fyens.dk/danmark/Jobcentret-goer-sygemeldte-endnu-mere-syge/artikel/3359664
[47]
Høybye-Mortensen, M. and P. Ejbye-Ernst. 2018. The long way to data-driven decision-making: How do casework registrations become management information? STS Encounters, 10(2.2), 5--36.
[48]
Hughes, J. A., D. Randall, and D. Shapiro. 1993. From Ethnographic Record to System Design: Some experiences from the field. Computer Supported Cooperative Work (CSCW), 123--141.
[49]
Keddell, E. 2011. Reasoning Processes in Child Protection Decision Making: Negotiating Moral Minefields and Risky Relationships. The British Journal of Social Work, 41(7), 1251--1270.
[50]
Keyes, O. 2018. The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition. Proceedings of the ACM on Human-Computer Interaction, 88(CSCW, Article 88), 1--22.
[51]
Keymolen, E. and D. Broeders. 2011. Innocence Lost: Care and Control in Dutch Digital Youth Care. British Journal of Social Work, 43(1), 41--63.
[52]
Lampland, M. and S. L. Star. 2009. Standards and their stories: how quantifying, classifying, and formalizing practices shape everyday life. Cornell University Press.
[53]
Law, J. and J. Urry. 2013. Enacting the social. Economy and Society, 33(3), 390--410.
[54]
Lipsky, M. 1980. Street-Level Bureaucracy: The Dilemmas of the Individual in Public Service. Russell Sage Foundation.
[55]
Loxha, A. and M. Morgandi. 2014. Profiling the Unemployed: A Review of OECD Experiences and Implications for Emerging Economies. Social Protection & Labor, 1424, 1--45.
[56]
Martin, D., J. O'Neill, D. Randall, and M. Rouncefield. 2007. How Can I Help You? Call Centres, Classification Work and Coordination. Computer Supported Cooperative Work (CSCW), 16, 231--264.
[57]
Matthiesen, S. and P. Bjørn. 2017. When Distribution of Tasks and Skills Are Fundamentally Problematic: A Failure Story from Global Software Outsourcing. PACM on Human-Computer Interaction, 1(CSCW), 74--90.
[58]
Maynard, D. W. 1984. Inside Plea Bargaining: The Language of Negotiation. Plenum Press.
[59]
Maynard-Moody, S. and M. Musheno. 2003. Cops, Teachers, Counselors: Stories from the Front Lines of Public Service (Vol. 1). The University of Michigan Press.
[60]
Ministry of Finance. 2019. National Strategy for Artificial Intelligence: The Danish Government.
[61]
Møller, N. H. and P. Bjørn. 2011. Layers in Sorting Practices: Sorting out Patiens with Potential Cancer. Computer Supported Cooperative Work (CSCW), 20, 123--153. https://doi.org/10.1007/s10606-011-9133-3
[62]
Møller, N. H., I. Shklovski, and T. T. Hildebrandt. 2020. Shifting Concepts of Value: Designing Algorithmic Decision-Support Systems for Public Services. In Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society, 1--12. https://doi.org/10.1145/3419249.3420149
[63]
Paanakker, H. L. 2020. Perceptions of the Frontline Craft: Assessing Value Convergence Between Policy Makers, Managers, and Street-Level Professionals in the Prison Sector. Administration & Society, https://doi.org/10.1177/0095399720933815
[64]
Pasquale, F. 2020. New Laws of Robotics: Defending Human Expertise in the Age of AI. Harvard University Press.
[65]
Persson, J., A. Reinwald, E. Skorve, and P. Nielsen. 2017. Value positions in e-government strategies: Something is (not) changing in the state of Denmark.
[66]
Petersen, A. C. M., L. R. Christensen, and T. T. Hildebrandt. 2020. The Role of Discretion in the Age of Automation. Computer Supported Cooperative Work (CSCW), 29(3), 303--333. https://doi.org/10.1007/s10606-020-09371-3
[67]
QSR International. 2019. NVivo 12. from https://www.qsrinternational.com/nvivo/nvivo-products/nvivo-12-mac
[68]
Randall, D. 1995. A Comment on Lucy Suchman's ?do Categories Have Politics?". Computer Supported Cooperative Work (CSCW), 3, 47--50.
[69]
Randall, D., R. Harper, and M. Rouncefield. 2007. Fieldwork for Design: Theory and Practice. Springer.
[70]
Ranerup, A. and H. Z. Henriksen. 2019. Value positions viewed through the lens of automated decision-making: The case of social services. Government Information Quarterly, 36(4), 101377.
[71]
Rawls, A. W. 2010. Social Order as Moral Order. In S. Hitlin andS. Vaisey (Eds.), Handbook of the Sociology of Morality (pp. 95--122). New York: Springer Science+Business Media.
[72]
Rieder, G. 2018. Big Data: Or, the Vision that Would Not Fade. (PhD Dissertation), IT University of Copenhagen, Copenhagen, Denmark.
[73]
Rouncefield, M. 2011. Fieldwork, Ethnography and Ethnomethodology LSCITS Socio-Technical Systems Engineering Handbook (pp. 44--48). http://archive.cs.st-andrews.ac.uk/STSE-Handbook/: University of St Andrew.
[74]
Sacks, H. 1992. Lectures on Conversation (Vol. I & II). Blackwell Publishing.
[75]
Schmidt, K.and I. Wagner. 2003. Ordering Systems: Coordinative Practices in Architectural Design and Planning. In Proceedings of the 2003 international ACM SIGGROUP conference on Supporting group work, 274--283.
[76]
Schultz. 2018. MØD ASTA -- DIN NYE ASSISTENT: Interview med Simon Kohrtz, produktchef, Schultz. from https://schultz.dk/om-schultz/nyt-fra-schultz/moed-asta-din-nye-assistent/
[77]
Sendak, M., M. C. Elish, M. Gao, J. Futoma, W. Ratliff, M. Nichols, et al. 2020. "The Human Body is a Black Box": Supporting Clinical Decision-Making with Deep Learning. In Proceedings of the 2020 conference on fairness, accountability and transparency, 99--109.
[78]
Skolnick, J. H. 1966. Justice Without Trial: Law Enforcement in Democratic Society. New York: John Wiley & Sons.
[79]
Smith, R. J. and P. Atkinson. 2015. Method and Measurement in Sociology, fifty years on. International Journal of Social Research Methodology, 19(1), 99--110.
[80]
Star, S. L. and G. C. Bowker. 2007. Enacting silence: Residual categories as a challenge for ethics, information systems, and communication. Ethics and Information Technology, 9, 273--280.
[81]
Star, S. L. and A. Strauss. 1999. Layers of silence, arenas of voice: The ecology of visible and invisible work. Computer Supported Cooperative Work (CSCW), 8(1--2), 9--30.
[82]
Steensland, B. 2010. Moral Classification and Social Policy. In S. Hitlin and S. Vaisey (Eds.), Handbook of the Sociology of Morality (pp. 455--467). Springer.
[83]
Suchman, L. 1993. Do Categories Have Politics? The language/action perspective reconsidered. In Proceedings of the Third European Conference on Computer-Supported Cooperative Work, Milan, Italy. ECSCW'93.
[84]
Suchman, L. 1995. Making Work Visible. Communication of the ACM, 38(9), 56--64.
[85]
Suchman, L. A. 1987. Plans and situated actions: The problem of human-machine communication. Cambridge University Press.
[86]
Suchman, L. A. 2007. Human-machine reconfigurations: Plans and situated actions. Cambridge University Press.
[87]
Timmermans, S. and M. Berg. 1997. Standardization in action: Achieving local universality through medical protocols. Social Studies of Science, 27, 273--305.
[88]
Verne, G. 2015. "The winners are those who have used the old paper form" On citizens and automated public services. (PhD Dissertation), University of Oslo, Oslo, Norway. (No. 1650)
[89]
Villesen, K. 2010. Socialrådgivere modarbejder beskæftigelsespolitikken. from https://www.information.dk/indland/2010/01/socialraadgivere-modarbejder-beskaeftigelsespolitikken
[90]
Voida, A., L. Dombrowski, G. R. Hayes, and M. Mazmanian. 2014. Shared values/conflicting logics: working around e-government systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 3583--3592.

Cited By

View all
  • (2024)U.S. Job-Seekers' Organizational Justice Perceptions of Emotion AI-Enabled InterviewsProceedings of the ACM on Human-Computer Interaction10.1145/36869938:CSCW2(1-42)Online publication date: 8-Nov-2024
  • (2024)From Stem to Stern: Contestability Along AI Value ChainsCompanion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing10.1145/3678884.3681831(720-723)Online publication date: 11-Nov-2024
  • (2024)Misfitting With AI: How Blind People Verify and Contest AI ErrorsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675659(1-17)Online publication date: 27-Oct-2024
  • Show More Cited By

Index Terms

  1. Conceptualising Contestability: Perspectives on Contesting Algorithmic Decisions

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image Proceedings of the ACM on Human-Computer Interaction
    Proceedings of the ACM on Human-Computer Interaction  Volume 5, Issue CSCW1
    CSCW
    April 2021
    5016 pages
    EISSN:2573-0142
    DOI:10.1145/3460939
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 22 April 2021
    Published in PACMHCI Volume 5, Issue CSCW1

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. algorithmic decision-making
    2. algorithmic fairness, accountability, and transparency
    3. artificial intelligence
    4. contestability

    Qualifiers

    • Research-article

    Funding Sources

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)240
    • Downloads (Last 6 weeks)29
    Reflects downloads up to 12 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)U.S. Job-Seekers' Organizational Justice Perceptions of Emotion AI-Enabled InterviewsProceedings of the ACM on Human-Computer Interaction10.1145/36869938:CSCW2(1-42)Online publication date: 8-Nov-2024
    • (2024)From Stem to Stern: Contestability Along AI Value ChainsCompanion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing10.1145/3678884.3681831(720-723)Online publication date: 11-Nov-2024
    • (2024)Misfitting With AI: How Blind People Verify and Contest AI ErrorsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675659(1-17)Online publication date: 27-Oct-2024
    • (2024)Understanding Practices around Computational News Discovery Tools in the Domain of Science JournalismProceedings of the ACM on Human-Computer Interaction10.1145/36374198:CSCW1(1-36)Online publication date: 26-Apr-2024
    • (2024)Seamful XAI: Operationalizing Seamful Design in Explainable AIProceedings of the ACM on Human-Computer Interaction10.1145/36373968:CSCW1(1-29)Online publication date: 26-Apr-2024
    • (2024)AppealMod: Inducing Friction to Reduce Moderator Workload of Handling User AppealsProceedings of the ACM on Human-Computer Interaction10.1145/36372968:CSCW1(1-35)Online publication date: 26-Apr-2024
    • (2024)Against Predictive Optimization: On the Legitimacy of Decision-making Algorithms That Optimize Predictive AccuracyACM Journal on Responsible Computing10.1145/36365091:1(1-45)Online publication date: 20-Mar-2024
    • (2024)A Critical Survey on Fairness Benefits of Explainable AIProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658990(1579-1595)Online publication date: 3-Jun-2024
    • (2024)Community Begins Where Moderation Ends: Peer Support and Its Implications for Community-Based RehabilitationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642675(1-18)Online publication date: 11-May-2024
    • (2024)Patient Perspectives on AI-Driven Predictions of Schizophrenia Relapses: Understanding Concerns and Opportunities for Self-Care and TreatmentProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642369(1-20)Online publication date: 11-May-2024
    • Show More Cited By

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media