More Web Proxy on the site http://driver.im/

research-article

Open access

Shifting Concepts of Value: Designing Algorithmic Decision-Support Systems for Public Services

Authors:

Naja Holten Møller,

Irina Shklovski,

Thomas T. HildebrandtAuthors Info & Claims

NordiCHI '20: Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society

Article No.: 70, Pages 1 - 12

https://doi.org/10.1145/3419249.3420149

Published: 26 October 2020 Publication History

All formats PDF

Abstract

Calls for responsible design in algorithmic decision-support systems, especially those used in public services, are increasingly common. While an algorithmic system might promise greater precision and efficiency in domains such as critical care, the same efficiency is difficult to replicate in the public service domain, where caseworkers must exercise discretion in applying complex legal frameworks that directly affect individual lives. In this paper we examine the challenges in responsibly designing such an algorithmic decision-support system. We report findings from a large cross-disciplinary research project, aiming to develop an algorithmic component for municipal decision-support systems for job placement in Denmark. Our data showcases insights from how a team of data scientists, caseworkers, and system developers negotiated notions of value metrics and usefulness in a participatory design set-up. Whereas data scientists expected to focus on profiling individuals, the caseworkers instead pushed for systems that could help mitigate organizational contradictions and obscured processes in casework. We close with a discussion of challenges and future directions for participatory algorithmic systems design in municipal contexts.

References

[1]

Ali Alkhatib and Michael Bernstein. 2019. Street-level algorithms: A theory at the gaps between policy and decisions. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI), Article 530.

Digital Library

[2]

Karla Bardillo-Urquiola, Xinru Page and Pamela J. Wisniewski. 2019. Risk vs. restriction: The tension between providing a sense of normalcy and keeping foster teens safe online. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI), Article 267.

[3]

Reuben Binns, Max Van Kleek, Michael Veale, Ulrik Lyngs, Jun Zhao, and Nigel Shadbolt. 2018. ‘It's Reducing a Human Being to a Percentage’: Perceptions of Justice in Algorithmic Decisions. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI), Article 377.

Digital Library

[4]

Pernille Bjørn & Ellen Balka. 2007. Healthcare Categories Have Politics Too: Unpacking the Managerial Agendas of Electronic Triage Systems. In Proceedings of The European Conference on Computer-Supported Cooperative Work (ECSCW), 371-390.

[5]

Nikolaj Gandrup Borchorst & Susanne Bødker. 2011. ”You Probably Shouldn't Give Them Too Much Information“. Supporting Citizen-Government Collaboration. In Proceedings of The European Conference on Computer-Supported Cooperative Work (ECSCW), 173-192.

[6]

Claus Bossen, Yunan Chen & Kathleen H. Pine. 2019. The Emergence of New Data Work Occupations in Healthcare: The Case of Medical Scribes. International Journal of Medical Informatics 123, March 2019, 76-83.

[7]

Nina Boulus-Rødje. 2018. In Search for the Perfect Pathway: Supporting Knowledge Work of Welfare Workers. Computer Supported Cooperative Work: An International Journal (JCSCW), 27(3-6), 841-874.

Digital Library

[8]

Geoffrey C. Bowker & Susan Leigh-Star. 1999. Sorting Things Out. Classification and Its Consequences, MIT Press.

[9]

Tone Bratteteig and Ina Wagner. 2014. Disentangling Participation: Power and Decision-Making in Participatory Design. Springer.

[10]

Anna Brown, Alexandra Chouldechova, Emily Putnam-Hornstein, Andrew Tobin, and Rhema Vaithianathan. 2019. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI), Article 41.

[11]

Jenna Burrell. 2016. How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big Data & Society, January-June 2016, 1-12.

[12]

Peter André Busch and Helle Zinner Henriksen. 2018. Digital discretion: A systematic literature review of ICT and street-level discretion. Information Polity, 23, 3-28.

Digital Library

[13]

Susanne Bødker, Christian Dindler & Ole Sejer Iversen. Tying Knots. Participatory Infrastructuring at Work. Computer Supported Cooperative Work: An International Journal (JCSCW), 26(1-2), 245-273.

[14]

Susanne Bødker, Pelle Ehn, Dan Sjögren & Yngve Sundblad. 2000. Cooperative Design – Perspectives on 20 Years With ’the Scandinavian IT Design Model’. Invited paper in Proceedings of the 11 th Nordic Conference on Human-Computer Interaction (NordiCHI), 1-11.

[15]

Carrie J. Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2019. "Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making. Proceedings of ACM Human Computer Interaction 3, CSCW, Article 104.

Digital Library

[16]

Alexander Campolo and Kate Crawford. 2020. Enchanted determinism. Power without responsibility in Artificial Intelligence. Engaging Science, Technology & Society, 6(1), 1-19.

[17]

Christopher A. Le Dantec. 2016. Designing Publics. The MIT Press.

[18]

Paul Dourish. 2019. User Experience as Legitimacy Trap. ACM Interactions, Nov-Dec. 2019, 46-49.

[19]

Andy Dow, Rob Comber, and John Vines. 2018. Between grassroots and the hierarchy: Lessons learned from the design of a public services directory. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI), Article 442.

Digital Library

[20]

Virginia Eubanks. 2018. Automating Inequality: How High-Tech Tools Profile, Police and Punish the Poor. St. Martins Press.

[21]

Gladsaxe Kommune. 2020. Beskæftigelsesplan 2020 – Vækst, Velfærd og Unge i Uddannelse og Job, 1-10 https://rar-bm.dk/media/9719/beskaeftigelsesplan-gladsaxe.pdf (accessed May 2020).

[22]

Gladsaxe Kommune. 2020. Gladsaxestrategien 2018-2022. Bæredygtig Vækst og Velfærd, 1-24 https://gladsaxe.dk/Files//Files/Faelles-dokumenter/Planer-politikker-visioner/glx/Gladsaxestrategien-2018-2022-web-5.pdf (accessed May 2020).

[23]

Lise Justesen, & Ursula Plesner. 2018. Fra Skøn til Algoritme: Digitaliseringsklar Lovgivning og Automatisering af Administrativ Sagsbehandling. Tidsskrift for Arbejdsliv, 20(3), 9–23.

[24]

Elizabeth Kaziunas, Mark S. Ackerman, Silvia Lindtner & Joyce M. Lee. 2017. Caring through Data: Attending to the Social and Emotional Experiences of Health Datafication. In Proceedings of the ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW), 2260-2272.

Digital Library

[25]

Finn Kensing & Joan Greenbaum. 2013. Heritage. Having a Say. Simonsen, J. & Robertson (Eds), T. Routledge International Handbook of Participatory Design, Routledge, 21-36.

[26]

Kristian Helbo Kristiansen, Mathias A. Valeur-Meller, Lynn Dombrowski & Naja Holten Møller. 2018. Accountability in the blue-collar data-driven workplace. In proceedings of the ACM Conference on Human Factors in Computing (CHI), Paper 332.

Digital Library

[27]

David Lazer, Ryan Kennedy, Gary King & Alessandro Vespignani. 2014. The parable of Google flu: Traps in Big Data analysis. Science 343: 1203– 1205. Available at: www.sciencemag.org/content/343/ 6176/1203.full/ (accessed May 2020).

[28]

Ann Light & Anna Seravalli. 2019. The breakdown of the municipality as caring platform: Lessons for co-design and co-learning in the age of platform capitalism. In CoDesign. International Journal of Co-creation in Design and the Arts, 192-211.

[29]

Ann Light, Alison Powell & Irina Shklovski. 2017. Design for existential crisis in the anthropocene age. In proceedings of the ACM Conference on Human Factors in Computing (CHI), 722-734.

Digital Library

[30]

Michael Lipsky. 1980. Toward a Theory of Street-Level Bureaucracy. Russell Sage.

[31]

Thomas Lodato and Carl DiSalvo. 2018. Institutional constraints: The forms and limits of Participatory Design in the public realm. In Proceedings of the ACM Participatory Design Conference (PDC), Article 5.

Digital Library

[32]

Caitlin Lustig, Katie Pine, Bonnie Nardi, Lilly Irani, Min Kyung Lee, Dawn Nafus, and Christian Sandvig. 2016. Algorithmic authority: The ethics, politics, and economics of algorithms that interpret, decide, and manage. In Proceedings of the ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA), 1057–1062.

Digital Library

[33]

Hanne Marie Motzfeldt. 2019. Socialrådgivere og Fagforeninger: Grib dog redningskransen! Uden for Nummer, 39(19), 32-41.

[34]

Naja Holten Møller; Geraldine Fitzpatrick & Christopher A. Le Dantec. 2019. Assembling the case: Citizens' strategies for exercising authority and personal autonomy in social welfare. In Proceedings of the ACM on Human-Computer Interaction 3, GROUP, Article 244.

[35]

Naja Holten Møller. 2018. The future of clerical work is precarious. ACM Interactions 25(4), 75-77.

Digital Library

[36]

Naja Holten Møller & Pernille Bjørn. 2011. Layers in sorting practices. Sorting out patients with potential cancer. Computer Supported Cooperative Work: An International Journal (JCSCW), 20(3), 123-153.

Digital Library

[37]

Klaus Lindgaard Høyer. 2019. Data as promise: Reconfiguring Danish public health through personalized medicine. Social Studies of Science, 49(4), 531-555.

[38]

Gina Neff, Anissa Tanweer, Brittany Fiore-Gartland & Laura Osborn. 2017. Critique and contribute. A practice-based framework for improving critical data studies and data science. Big Data, 5(2), 85-97.

[39]

Wendy Norris, Amy Voida, Leysia Palen & Stephen Voida. 2019. ‘Is the time right now?': Reconciling sociotemporal disorder in distributed team work. In Proceedings of the ACM Conference on Human Factors in Computing (CHI), Article 98.

Digital Library

[40]

Samir Passi & Steven J. Jackson. 2017. Data vision. Learning to see through algorithmic abstraction. In Proceedings of the ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW), 2436-2447.

[41]

Anette C. M. Petersen, Lars Rune Christensen & Thomas T. Hildebrandt. 2020. The role of discretion in the age of automation. Computer Supported Cooperative Work: An International Journal (JCSCW).

[42]

Kathleen H. Pine & Max Liboiron. 2015. The politics of measurement and action. In proceedings of the ACM Conference on Human Factors in Computing (CHI), 3147-3156.

Digital Library

[43]

David Pinelle & Carl Gutwin. 2006. Loose coupling and healthcare organizations: Deployment strategies for groupware. Computer Supported Cooperative Work (CSCW): An International Journal (JCSCW), 15(6), 537–572.

[44]

Anthony T. Pinter, Pamela Wisnewski, Heng Xu, Mary Beth Rosson, and John M. Carroll. 2017. Adolescent online safety: Moving beyond formative evaluations to designing solutions to the future. Note in Proceedings of the ACM Interaction Design and Children Conference (IDC), 352-357.

[45]

Ursula Plesner, Lise Justesen & Cecilie Glerup. 2018. The transformation of work in digitized public sector organizations. Journal of Organizational Change Management, 31(5), 1176–1190.

[46]

Juho Pääkkönen, Matti Nelimarkka, Jesse Haapoja & Airi Lampinen. 2020. Bureacracy as a lens for analyzing and designing algorithmic systems. In Proceedings of the ACM Conference on Human Factors in Computing (CHI), Article 651.

[47]

Dave Randall, Richard Harper & Mark Rouncefield. 2007. Fieldwork for Design. Theory and Practice. Springer.

Digital Library

[48]

Toni Robertson & Ina Wagner. 2013. Simonsen, J. & Robertson (Eds), T. Routledge International Handbook of Participatory Design, Routledge, 64 85.

[49]

Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2020b. A human-centered review of the algorithms used within the U.S. child welfare system.". In Proceedings of the ACM Conference on Human Factors in Computing Systems, Article 102.

Digital Library

[50]

Irina Shklovski, Emily Troshynski & Paul Dourish. 2015. Mobile technologies and spatiotemporal configurations of institutional practice. Journal of the Association for Information Science and Technology, 66(10), 2098–2115.

Digital Library

[51]

Lucy Suchman. 1993. Do categories have politics? The language/ action perspective reconsidered. The European Conference on Computer Supported Cooperative Work (ECSCW), 1-14.

[52]

Michael Veale, Max Van Kleek, and Reuben Binns. 2018. Fairness and accountability design needs for algorithmic support in high-stakes public sector decision-making. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI), Article 440.

Digital Library

[53]

Amy Voida, Lynn Dombrowski, Gillian R Hayes & Melissa Mazmanian. 2014. Shared values / conflicting logics: Working around e-government systems. In Proceedings of the ACM Conference on Human Factors in Computing (CHI), 3583-3592.

Digital Library

Cited By

Jacob SSouissi S(2024)Social Workers in the Context of Digital TransformationsRevue des politiques sociales et familiales10.3917/rpsf.149.0207n°149:4(207-216)Online publication date: 10-Jan-2024
https://doi.org/10.3917/rpsf.149.0207
Nielsen TGammeltoft-Hansen THolten Møller N(2024)Mobile Phone Data Transforming Casework in Asylum Decision-making: Insights from the Danish CaseACM Journal on Responsible Computing10.1145/36964691:4(1-29)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3696469
Kawakami ACoston AHeidari HHolstein KZhu H(2024)Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and UseProceedings of the ACM on Human-Computer Interaction10.1145/36869898:CSCW2(1-24)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686989
Show More Cited By

Index Terms

Shifting Concepts of Value: Designing Algorithmic Decision-Support Systems for Public Services
1. Information systems

Index terms have been assigned to the content through auto-classification.

Recommendations

Street-Level Algorithms and AI in Bureaucratic Decision-Making: A Caseworker Perspective
CSCW

Studies of algorithmic decision-making in Computer-Supported Cooperative Work (CSCW) and related fields of research increasingly recognize an analogy between AI and bureaucracies. We elaborate this link with an empirical study of AI in the context of ...
The Role of Physical Cues in Co-located and Remote Casework
Abstract
Across the world, large swaths of society closed in response to the COVID-19 (C-19) pandemic, transforming the provision of government services, including welfare. The shift to remote work afforded a glimpse of what a future digitized public ...
Auditing Risk Prediction of Long-Term Unemployment
GROUP

As more and more governments adopt algorithms to support bureaucratic decision-making processes, it becomes urgent to address issues of responsible use and accountability. We examine a contested public service algorithm used in Danish job placement for ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

NordiCHI '20: Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society

October 2020

1177 pages

ISBN:9781450375795

DOI:10.1145/3419249

General Chairs:
David Lamas
Tallinn University, Estonia
,
Hegle Sarapuu
Trinidad Wiseman, Estonia
,
Publications Chairs:
Ilja Šmorgun
Tallinn University, Estonia
,
Gerd Berget
Oslo Metropolitan University, Norway

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

NordiCHI '20

NordiCHI '20: Shaping Experiences, Shaping Society

October 25 - 29, 2020

Tallinn, Estonia

Acceptance Rates

NordiCHI '20 Paper Acceptance Rate 89 of 399 submissions, 22%;

Overall Acceptance Rate 379 of 1,572 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

67
Total Citations
View Citations
2,006
Total Downloads

Downloads (Last 12 months)514
Downloads (Last 6 weeks)88

Reflects downloads up to 10 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jacob SSouissi S(2024)Social Workers in the Context of Digital TransformationsRevue des politiques sociales et familiales10.3917/rpsf.149.0207n°149:4(207-216)Online publication date: 10-Jan-2024
https://doi.org/10.3917/rpsf.149.0207
Nielsen TGammeltoft-Hansen THolten Møller N(2024)Mobile Phone Data Transforming Casework in Asylum Decision-making: Insights from the Danish CaseACM Journal on Responsible Computing10.1145/36964691:4(1-29)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3696469
Kawakami ACoston AHeidari HHolstein KZhu H(2024)Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and UseProceedings of the ACM on Human-Computer Interaction10.1145/36869898:CSCW2(1-24)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686989
Lee SDourish P(2024)Reconfiguring Data Relations: Institutional Dynamics around Data in Local GovernanceProceedings of the ACM on Human-Computer Interaction10.1145/36869598:CSCW2(1-28)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686959
Kim SHo JLi YFan BYang WRamey JFox SZhu HZimmerman JEslami M(2024)Integrating Equity in Public Sector Data-Driven Decision Making: Exploring the Desired Futures of Underserved StakeholdersProceedings of the ACM on Human-Computer Interaction10.1145/36869058:CSCW2(1-39)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686905
Moon ESaxena DMaharaj TGuha S(2024)Beyond Predictive Algorithms in Child WelfareProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670976(1-13)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670976
Kim SFan BYang WRamey JFox SZhu HZimmerman JEslami M(2024)Public Technologies Transforming Work of the Public and the Public SectorProceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work10.1145/3663384.3663407(1-12)Online publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1145/3663384.3663407
Sistrunk ASelf NBiswas SLuther KVerdezoto NRamakrishnan N(2024)Redistrict: Online Public Deliberation Support that Connects and Rebuilds Inclusive CommunitiesProceedings of the ACM on Human-Computer Interaction10.1145/36373938:CSCW1(1-23)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637393
Yoo DWoo HPendse SLu NBirnbaum MAbowd GDe Choudhury M(2024)Missed Opportunities for Human-Centered AI Research: Understanding Stakeholder Collaboration in Mental Health AI ResearchProceedings of the ACM on Human-Computer Interaction10.1145/36373728:CSCW1(1-24)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637372
Olsen HHildebrandt TWiesener CLarsen MFlügge A(2024)The Right to Transparency in Public Governance: Freedom of Information and the Use of Artificial Intelligence by Public AgenciesDigital Government: Research and Practice10.1145/36327535:1(1-15)Online publication date: 12-Mar-2024
https://dl.acm.org/doi/10.1145/3632753
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents