[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
research-article

'Transparency is Meant for Control' and Vice Versa: Learning from Co-designing and Evaluating Algorithmic News Recommenders

Published: 11 November 2022 Publication History

Abstract

Algorithmic systems that recommend content often lack transparency about how they come to their suggestions. One area in which recommender systems are increasingly prevalent is online news distribution. In this paper, we explore how a lack of transparency of (news) recommenders can be tackled by involving users in the design of interface elements. In the context of automated decision-making, legislative frameworks such as the GDPR in Europe introduce a specific conception of transparency, granting 'data subjects' specific rights and imposing obligations on service providers. An important related question is how people using personalized recommender systems relate to the issue of transparency, not as legal data subjects but as users. This paper builds upon a two-phase study on how users conceive of transparency and related issues in the context of algorithmic news recommenders. We organized co-design workshops to elicit participants' 'algorithmic imaginaries' and invited them to ideate interface elements for increased transparency. This revealed the importance of combining legible transparency features with features that increase user control. We then conducted a qualitative evaluation of mock-up prototypes to investigate users' preferences and concerns when dealing with design features to increase transparency and control. Our investigation illustrates how users' expectations and impressions of news recommenders are closely related to their news reading practices. On a broader level, we show how transparency and control are conceptually intertwined. Transparency without control leaves users frustrated. Conversely, without a basic level of transparency into how a system works, users remain unsure of the impact of controls.

References

[1]
Alessandro Acquisti, Idris Adjerid, Rebecca Balebako, Laura Brandimarte, Lorrie Faith Cranor, Saranga Komanduri, Pedro Giovanni Leon, Norman Sadeh, Florian Schaub, Manya Sleeper, Yang Wang, and Shomir Wilson. 2017. Nudges for Privacy and Security: Understanding and Assisting Users' Choices Online. ACM Comput. Surv. 50, 3, Article 44 (aug 2017), 41 pages. https://doi.org/10.1145/3054926
[2]
Alessandro Acquisti, Laura Brandimarte, and George Loewenstein. 2015. Privacy and human behavior in the age of information. Science 347, 6221 (2015), 509--514. https://doi.org/10.1126/science.aaa1465
[3]
Oscar Alvarado, Vero Vanden Abeele, David Geerts, and Katrien Verbert. 2019. "I Really Don't Know What ?Thumbs Up' Means": Algorithmic Experience in Movie Recommender Algorithms. In Human-Computer Interaction -- INTERACT 2019. INTERACT 2019. Lecture Notes in Computer Science, vol 11748. Springer, Cham, Lamas D., Loizides F., Nacke L., Petrie H., Winckler M., and Zaphiris P. (Eds.). Springer, Cham, 521--541. https://doi.org/10.1007/978--3-030--29387--1_30
[4]
Oscar Alvarado, Elias Storms, David Geerts, and Katrien Verbert. 2020. Foregrounding Algorithms: Preparing Users for Co-Design with Sensitizing Activities. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3419249.3421237
[5]
Mike Ananny and Kate Crawford. 2016. Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society (2016), 146144481667664. https://doi.org/10.1177/1461444816676645
[6]
Pauline Anthonysamy, Phil Greenwood, and Awais Rashid. 2013. Social Networking Privacy: Understanding the Disconnect from Policy to Controls. Computer 46, 6 (June 2013), 60--67. https://doi.org/10.1109/MC.2012.326
[7]
Kaustav Bhattacharjee, Min Chen, and Aritra Dasgupta. 2020. Privacy-Preserving Data Visualization: Reflections on the State of the Art and Research Opportunities. Computer Graphics Forum 39, 3 (2020), 675--692. https://doi.org/10.1111/cgf.14032
[8]
Engin Bozdag. 2013. Bias in Algorithmic Filtering and Personalization. Ethics and Inf. Technol. 15, 3 (Sept. 2013), 209--227. https://doi.org/10.1007/s10676-013--9321--6
[9]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (jan 2006), 77--101. https://doi.org/10.1191/1478088706qp063oa
[10]
Taina Bucher. 2017. The algorithmic imaginary: exploring the ordinary affects of Facebook algorithms. Information, Communication & Society 20, April (2017), 30--44. https://doi.org/10.1080/1369118X.2016.1154086
[11]
Andrea Bunt, Matthew Lount, and Catherine Lauzon. 2012. Are Explanations Always Important? A Study of Deployed, Low-Cost Intelligent Interactive Systems. In Proceedings of the 2012 ACM International Conference on Intelligent User Interfaces (Lisbon, Portugal) (IUI '12). Association for Computing Machinery, New York, NY, USA, 169--178. https://doi.org/10.1145/2166966.2166996
[12]
Ana S. Cardenal, Carlos Aguilar-Paredes, Carol Galais, and Mario Pérez-Montoro. 2019. Digital Technologies and Selective Exposure: How Choice and Filter Bubbles Shape News Media Exposure. The International Journal of Press/Politics 24, 3 (July 2019), 1--22. https://doi.org/10.1177/1940161219862988
[13]
Michael A. DeVito, Darren Gergle, and Jeremy Birnholtz. 2017. Algorithms ruin everything: RIPTwitter, Folk Theories, and Resistance to Algorithmic Change in Social Media. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI '17 (2017), 3163--3174. https://doi.org/10.1145/3025453.3025659
[14]
Nicholas Diakopoulos and Michael Koliska. 2016. Algorithmic Transparency in the News Media. Digital Journalism 5, 7 (2016), 809--828. https://doi.org/10.1080/21670811.2016.1208053
[15]
Olha Drozd and Sabrina Kirrane. 2019. I Agree: Customize Your Personal Data Processing with the CoRe User Interface. In Trust, Privacy and Security in Digital Business, Stefanos Gritzalis, Edgar R. Weippl, Sokratis K. Katsikas, Gabriele Anderst-Kotsis, A Min Tjoa, and Ismail Khalil (Eds.). Springer International Publishing, Cham, 17--32.
[16]
Olha Drozd and Sabrina Kirrane. 2020. Privacy CURE: Consent Comprehension Made Easy. In ICT Systems Security and Privacy Protection, Marko Hölbl, Kai Rannenberg, and Tatjana Welzer (Eds.). Springer International Publishing, Cham, 124--139.
[17]
Motahhare Eslami, Karrie Karahalios, Christian Sandvig, Kristen Vaccaro, Aimee Rickman, Kevin Hamilton, and Alex Kirlik. 2016. First I "like" it, then I hide it: Folk Theories of Social Feeds. Conference on Human Factors in Computing Systems (2016), 2371--2382. https://doi.org/10.1145/2858036.2858494
[18]
Motahhare Eslami, Aimee Rickman, Kristen Vaccaro, Amirhossein Aleyasen, Andy Vuong, Karrie Karahalios, Kevin Hamilton, and Christian Sandvig. 2015. "I always assumed that I wasn't really that close to [her]". In Proceedings of the 2015 CHI Conference on Human Factors in Computing Systems - CHI '15. Seoul, 153--162. https://doi.org/10.1145/2702123.2702556
[19]
European Union. 2016. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016. OJ L 119, 04.05.2016, p. 1--88. Official Journal of the European Union 59, L 119 (may 2016), 1--88.
[20]
Benjamin Fabian, Tatiana Ermakova, and Tino Lentz. 2017. Large-Scale Readability Analysis of Privacy Policies. In Proceedings of the International Conference on Web Intelligence (Leipzig, Germany) (WI '17). Association for Computing Machinery, New York, NY, USA, 18--25. https://doi.org/10.1145/3106426.3106427
[21]
Daniel Fallman. 2007. Why Research-Oriented Design Isn't Design-Oriented Research: On the Tensions Between Design and Research in an Implicit Design Discipline. Knowledge, Technology & Policy 20, 3 (2007), 193--200. https://doi.org/10.1007/s12130-007--9022--8
[22]
Richard Fletcher and Rasmus Kleis Nielsen. 2018. Generalised Scepticism: How people navigate news on social media. Information, Communication & Society 22, 12 (2018), 1751--1769. https://doi.org/10.1080/1369118x.2018.1450887
[23]
Megan French and Jeff Hancock. 2017. What's the Folk Theory? Reasoning About Cyber-Social Systems. SSRN Electronic Journal (feb 2017). https://doi.org/10.2139/ssrn.2910571
[24]
Colin M. Gray, Cristiana Santos, Nataliia Bielova, Michael Toth, and Damian Clifford. 2021. Dark Patterns and the Legal Requirements of Consent Banners: An Interaction Criticism Perspective. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3411764.3445779
[25]
Kevin Hamilton, Karrie Karahalios, Christian Sandvig, and Motahhare Eslami. 2014. A path to understanding the effects of algorithm awareness. In Proceedings of the extended abstracts of ACM conference on Human factors in computing systems - CHI EA '14. 631--642. https://doi.org/10.1145/2559206.2578883
[26]
Jaron Harambam, Dimitrios Bountouridis, Mykola Makhortykh, and Joris van Hoboken. 2019. Designing for the better by taking users into account: a qualitative evaluation of user control mechanisms in (news) recommender systems. In Proceedings of the 13th ACM Conference on Recommender Systems - RecSys '19. ACM Press, Copenhagen, Denmark, 69--77. https://doi.org/10.1145/3298689.3347014
[27]
Natascha Just and Michael Latzer. 2017. Governance by algorithms: reality construction by algorithmic selection on the Internet. Media, Culture and Society 39, 2 (2017), 238--258. https://doi.org/10.1177/0163443716643157
[28]
Andrés Lucero. 2015. Using Affinity Diagrams to Evaluate Interactive Prototypes. In Human-Computer Interaction - INTERACT 2015. 231--248. https://doi.org/10.1007/978--3--319--22668--2_19
[29]
Célestin Matte, Nataliia Bielova, and Cristiana Santos. 2020. Do Cookie Banners Respect my Choice? : Measuring Legal Compliance of Banners from IAB Europe's Transparency and Consent Framework. In 2020 IEEE Symposium on Security and Privacy (SP). 791--809. https://doi.org/10.1109/SP40000.2020.00076
[30]
Aleecia M. McDonald, Robert W. Reeder, Patrick Gage Kelley, and Lorrie Faith Cranor. 2009. A Comparative Study of Online Privacy Policies and Formats. In Privacy Enhancing Technologies, Ian Goldberg and Mikhail J. Atallah (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 37--55.
[31]
Brent Daniel Mittelstadt, Patrick Allo, Mariarosaria Taddeo, Sandra Wachter, and Luciano Floridi. 2016. The ethics of algorithms: Mapping the debate. Big Data & Society 3, 2 (dec 2016), 1--21. https://doi.org/10.1177/2053951716679679
[32]
Philip M. Napoli. 2014. Automated media: An institutional theory perspective on algorithmic media production and consumption. Communication Theory 24, 3 (2014), 340--360. https://doi.org/10.1111/comt.12039
[33]
Laurens Naudts, Pierre Dewitte, and Jef Ausloos. Forthcoming 2022. Meaningful Transparency through Data Rights: A Multidimensional Analysis. In Research Handbook on EU Data Protection, Eleni Kosta and Leenes Ronald (Eds.). Edward Elgar Publishing.
[34]
Nic Newman, Richard Fletcher, David Levy, and Rasmus Kleis Nielsen. 2015. The Reuters Institute digital news report 2016. Reuters Institute for the Study of Journalism.
[35]
Daniel Neyland and Norma Möllers. 2017. Algorithmic IF... THEN rules and the conditions and consequences of power. Information Communication and Society 20, 1 (2017), 45--62. https://doi.org/10.1080/1369118X.2016.1156141
[36]
Midas Nouwens, Ilaria Liccardi, Michael Veale, David Karger, and Lalana Kagal. 2020. Dark Patterns after the GDPR: Scraping Consent Pop-Ups and Demonstrating Their Influence. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3313831.3376321
[37]
Eli Pariser. 2012. The Filter Bubble. Penguin Books Limited.
[38]
Article 29 Data Protection Working Party. 2007. Opinion 4/2007 on the concept of personal data. (June 2007). https://ec.europa.eu/justice/article-29/documentation/opinion-recommendation/files/2007/wp136_en.pdf
[39]
Article 29 Data Protection Working Party. 2018. Guidelines on Transparency under Regulation 2016/679. (Apr 2018). https://ec.europa.eu/newsroom/article29/document.cfm?action=display&doc_id=51025
[40]
Presidência da República do Brasil. 2018. Regulation (BRAZIL) 2018/13.709 of the Presidency of the Republic of Brazil in 14 August 2018. OJ L 13, 14.08.2018, p. 59--64. Diário Oficial da União 157 (august 2018), 59--64. http://www.planalto.gov.br/ccivil_03/_ato2015--2018/2018/lei/l13709.htm
[41]
Karen Renaud and Lynsay A. Shepherd. 2018. How to Make Privacy Policies both GDPR-Compliant and Usable. In 2018 International Conference On Cyber Situational Awareness, Data Analytics And Assessment (Cyber SA). 1--8. https://doi.org/10.1109/CyberSA.2018.8551442
[42]
Elizabeth Sanders and Pieter Jan Stappers. 2012. Convivial Design Toolbox: Generative Research for the Front End of Design. BIS, Amsterdam.
[43]
Elizabeth B.-N. Sanders and Pieter Jan Stappers. 2008. Co-Creation and the New Landscapes of Design. CoDesign 4, 1 (2008), 5--18.
[44]
Florian Schaub, Rebecca Balebako, and Lorrie Faith Cranor. 2017. Designing Effective Privacy Notices and Controls. IEEE Internet Computing (2017), 1--1. https://doi.org/10.1109/MIC.2017.265102930
[45]
Florian Schaub, Rebecca Balebako, Adam L. Durity, and Lorrie Faith Cranor. 2015. A Design Space for Effective Privacy Notices. In Eleventh Symposium On Usable Privacy and Security (SOUPS 2015). USENIX Association, Ottawa, 1--17. https://www.usenix.org/conference/soups2015/proceedings/presentation/schaub
[46]
Nick Seaver. 2017. Algorithms as culture: Some tactics for the ethnography of algorithmic systems. Big Data & Society 4, 2 (2017), 1--12. https://doi.org/10.1177/2053951717738104
[47]
H. Sharp, Y. Rogers, and J. Preece. 2019. Interaction Design: Beyond human-computer interaction (2 ed.). Wiley, Indianapolis.
[48]
Supreeth Shastri, Melissa Wasserman, and Vijay Chidambaram. 2021. GDPR Anti-Patterns. Commun. ACM 64, 2 (Jan. 2021), 59--65. https://doi.org/10.1145/3378061
[49]
Patrícia R. Sousa, Rolando Martins, and Luís Antunes. 2020. Empowering Users Through a Privacy Middleware Watchdog. In Trust, Privacy and Security in Digital Business, Stefanos Gritzalis, Edgar R. Weippl, Gabriele Kotsis, A. Min Tjoa, and Ismail Khalil (Eds.). Springer International Publishing, Cham, 156--170.
[50]
Kristen Vaccaro, Dylan Huang, Motahhare Eslami, Christian Sandvig, Kevin Hamilton, and Karrie Karahalios. 2018. The Illusion of Control. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18. https://doi.org/10.1145/3173574.3173590
[51]
Sami Vihavainen, Airi Lampinen, Antti Oulasvirta, Suvi Silfverberg, and Asko Lehmuskallio. 2014. The Clash between Privacy and Automation in Social Media. IEEE Pervasive Computing 13, 1 (Jan 2014), 56--63. https://doi.org/10.1109/MPRV.2013.25
[52]
Michele Willson. 2017. Algorithms (and the) everyday. Information Communication and Society 20, 1 (2017), 137--150. https://doi.org/10.1080/1369118X.2016.1200645

Cited By

View all
  • (2024)Human-centered NLP Fact-checking: Co-Designing with Fact-checkers using Matchmaking for AIProceedings of the ACM on Human-Computer Interaction10.1145/36869628:CSCW2(1-44)Online publication date: 8-Nov-2024
  • (2024)User Characterization Strategies in GDPR-Compliant Environments: Beekeeper AG Case Study2024 Tenth International Conference on eDemocracy & eGovernment (ICEDEG)10.1109/ICEDEG61611.2024.10702054(1-6)Online publication date: 24-Jun-2024
  • (2024)The impacts of relevance of recommendations and goal commitment on user experience in news recommender designUser Modeling and User-Adapted Interaction10.1007/s11257-024-09405-134:4(925-953)Online publication date: 1-Sep-2024
  • Show More Cited By

Index Terms

  1. 'Transparency is Meant for Control' and Vice Versa: Learning from Co-designing and Evaluating Algorithmic News Recommenders

        Recommendations

        Comments

        Please enable JavaScript to view thecomments powered by Disqus.

        Information & Contributors

        Information

        Published In

        cover image Proceedings of the ACM on Human-Computer Interaction
        Proceedings of the ACM on Human-Computer Interaction  Volume 6, Issue CSCW2
        CSCW
        November 2022
        8205 pages
        EISSN:2573-0142
        DOI:10.1145/3571154
        Issue’s Table of Contents
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 11 November 2022
        Published in PACMHCI Volume 6, Issue CSCW2

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. algorithms
        2. co-design
        3. news recommenders
        4. transparency

        Qualifiers

        • Research-article

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)99
        • Downloads (Last 6 weeks)14
        Reflects downloads up to 11 Dec 2024

        Other Metrics

        Citations

        Cited By

        View all
        • (2024)Human-centered NLP Fact-checking: Co-Designing with Fact-checkers using Matchmaking for AIProceedings of the ACM on Human-Computer Interaction10.1145/36869628:CSCW2(1-44)Online publication date: 8-Nov-2024
        • (2024)User Characterization Strategies in GDPR-Compliant Environments: Beekeeper AG Case Study2024 Tenth International Conference on eDemocracy & eGovernment (ICEDEG)10.1109/ICEDEG61611.2024.10702054(1-6)Online publication date: 24-Jun-2024
        • (2024)The impacts of relevance of recommendations and goal commitment on user experience in news recommender designUser Modeling and User-Adapted Interaction10.1007/s11257-024-09405-134:4(925-953)Online publication date: 1-Sep-2024
        • (2024)Conclusions and Open ChallengesTechnical and Regulatory Perspectives on Information Retrieval and Recommender Systems10.1007/978-3-031-69978-8_6(143-146)Online publication date: 24-Oct-2024
        • (2024)Privacy and SecurityTechnical and Regulatory Perspectives on Information Retrieval and Recommender Systems10.1007/978-3-031-69978-8_5(103-141)Online publication date: 24-Oct-2024
        • (2024)TransparencyTechnical and Regulatory Perspectives on Information Retrieval and Recommender Systems10.1007/978-3-031-69978-8_4(69-102)Online publication date: 24-Oct-2024
        • (2024)Biases, Fairness, and Non-discriminationTechnical and Regulatory Perspectives on Information Retrieval and Recommender Systems10.1007/978-3-031-69978-8_3(29-67)Online publication date: 24-Oct-2024
        • (2024)Regulatory InitiativesTechnical and Regulatory Perspectives on Information Retrieval and Recommender Systems10.1007/978-3-031-69978-8_2(11-27)Online publication date: 24-Oct-2024
        • (2024)IntroductionTechnical and Regulatory Perspectives on Information Retrieval and Recommender Systems10.1007/978-3-031-69978-8_1(1-10)Online publication date: 24-Oct-2024
        • (2023)Exploring users’ desire for transparency and control in news recommender systems: A five-nation studyJournalism10.1177/1464884923122209925:10(2001-2021)Online publication date: 18-Dec-2023
        • Show More Cited By

        View Options

        Login options

        Full Access

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Media

        Figures

        Other

        Tables

        Share

        Share

        Share this Publication link

        Share on social media