NORMalize 2024: The Second Workshop on Normative Design and Evaluation of Recommender Systems
Authors: 
Alain Starke, 
Sanne Vrijenhoek, 
Lien Michiels, Johannes Kruse, Nava TintarevAuthors Info & Claims
Pages 1242 - 1244
Abstract
Recommender systems are among the most widely used applications of artificial intelligence. Their use can have far-reaching consequences for users, stakeholders, and society at large. In this second edition of the NORMalize workshop, we once again seek to advance the research agenda of normative thinking, considering the norms and values that underpin recommender systems, as well as to introduce the concept to a broader audience. We aim to bring together a growing community of researchers and practitioners across disciplines who want to think about the norms and values that should be considered in the design and evaluation of recommender systems, and to further educate them on how to reflect on, prioritise, and operationalise such norms and values. NORMalize 2024 is a half-day workshop consisting of a combination of paper presentations and an interactive session, building upon its successful full-day run last year at RecSys’23.
References
[1]
Steve Buckler. 2010. Normative theory. Theory and methods in political science 3 (2010), 156–180.
[2]
Paul Byron and Kath Albury. 2018. ‘There are literally no rules when it comes to these things’: Ethical practice and the use of dating/hook-up apps. Digital intimate publics and social media (2018), 213–229.
[3]
Theresia Anita Christiani. 2016. Normative and empirical research methods: Their usefulness and relevance in the study of law as an object. Procedia-Social and Behavioral Sciences 219 (2016), 201–207.
[4]
Manoj Reddy Dareddy. 2016. Challenges in Recommender Systems for Tourism. In RecTour@ RecSys. 59–61.
[5]
Tim Draws, Oana Inel, Nava Tintarev, Christian Baden, and Benjamin Timmermans. 2022. Comprehensive viewpoint representations for a deeper understanding of user interactions with debated topics. In Proceedings of the 2022 Conference on Human Information Interaction and Retrieval. 135–145.
[6]
Tim Draws, Alisa Rieger, Oana Inel, Ujwal Gadiraju, and Nava Tintarev. 2021. A checklist to combat cognitive biases in crowdsourcing. In Proceedings of the AAAI conference on human computation and crowdsourcing, Vol. 9. 48–59.
[7]
Michael D Ekstrand, Mucun Tian, Mohammed R Imran Kazi, Hoda Mehrpouyan, and Daniel Kluver. 2018. Exploring author gender in book rating and recommendation. In Proceedings of the 12th ACM conference on recommender systems. 242–250.
[8]
Hanna Hauptmann, Alan Said, and Christoph Trattner. 2022. Research directions in recommender systems for health and well-being: A Preface to the Special Issue. User Modeling and User-Adapted Interaction (2022), 1–6.
[9]
Natali Helberger. 2019. On the democratic role of news recommenders. Digital Journalism 7, 8 (2019), 993–1012.
[10]
Andre Holzapfel, Bob Sturm, and Mark Coeckelbergh. 2018. Ethical dimensions of music information retrieval technology. Transactions of the International Society for Music Information Retrieval 1, 1 (2018), 44–55.
[11]
Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2016. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807 (2016).
[12]
Bart P Knijnenburg, Saadhika Sivakumar, and Daricia Wilkinson. 2016. Recommender systems for self-actualization. In Proceedings of the 10th acm conference on recommender systems. 11–14.
[13]
Rishabh Mehrotra, James McInerney, Hugues Bouchard, Mounia Lalmas, and Fernando Diaz. 2018. Towards a fair marketplace: Counterfactual evaluation of the trade-off between relevance, fairness & satisfaction in recommendation systems. In Proceedings of the 27th acm international conference on information and knowledge management. 2243–2251.
[14]
Lien Michiels, Jorre Vannieuwenhuyze, Jens Leysen, Robin Verachtert, Annelien Smets, and Bart Goethals. 2023. How Should We Measure Filter Bubbles? A Regression Model and Evidence for Online News. In Proceedings of the 17th ACM Conference on Recommender Systems (Singapore, Singapore) (RecSys ’23). Association for Computing Machinery, New York, NY, USA, 640–651. https://doi.org/10.1145/3604915.3608805
[15]
Erasmo Purificato, Ludovico Boratto, and Ernesto William De Luca. 2022. Do Graph Neural Networks Build Fair User Models? Assessing Disparate Impact and Mistreatment in Behavioural User Profiling. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 4399–4403.
[16]
Alisa Rieger, Tim Draws, Mariët Theune, and Nava Tintarev. 2021. This item might reinforce your opinion: Obfuscation and labeling of search results to mitigate confirmation bias. In Proceedings of the 32nd ACM conference on hypertext and social media. 189–199.
[17]
Bernd Carsten Stahl. 2012. Morality, ethics, and reflection: a categorization of normative IS research. Journal of the association for information systems 13, 8 (2012), 1.
[18]
Alain Starke, Martijn Willemsen, and Chris Snijders. 2021. Promoting energy-efficient behavior by depicting social norms in a recommender interface. ACM Transactions on Interactive Intelligent Systems (TiiS) 11, 3-4 (2021), 1–32.
[19]
Alain D Starke, Cataldo Musto, Amon Rapp, Giovanni Semeraro, and Christoph Trattner. 2024. “Tell Me Why”: using natural language justifications in a recipe recommender system to support healthier food choices. User Modeling and User-Adapted Interaction 34, 2 (2024), 407–440.
[20]
Alain D Starke and Martijn C Willemsen. 2024. Psychologically Informed Design of Energy Recommender Systems: Are Nudges Still Effective in Tailored Choice Environments? In A Human-Centered Perspective of Intelligent Personalized Environments and Systems. Springer, 221–259.
[21]
Judith Jarvis Thomson. 2010. Normativity.
[22]
Christoph Trattner and David Elsweiler. 2019. What online data say about eating habits. Nature Sustainability 2, 7 (2019), 545–546.
[23]
Elmira van den Broek, Anastasia Sergeeva, and Marleen Huysman. 2019. Hiring algorithms: An ethnography of fairness in practice. (2019).
[24]
Sanne Vrijenhoek, Gabriel Bénédict, Mateo Gutierrez Granada, Daan Odijk, and Maarten De Rijke. 2022. RADio – Rank-Aware Divergence Metrics to Measure Normative Diversity in News Recommendations. In Proceedings of the 16th ACM Conference on Recommender Systems (Seattle, WA, USA) (RecSys ’22). Association for Computing Machinery, New York, NY, USA, 208–219. https://doi.org/10.1145/3523227.3546780
[25]
Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21.
Index Terms
- NORMalize 2024: The Second Workshop on Normative Design and Evaluation of Recommender Systems
Recommendations
NORMalize: The First Workshop on Normative Design and Evaluation of Recommender Systems
RecSys '23: Proceedings of the 17th ACM Conference on Recommender SystemsRecommender systems are among the most widely used applications of artificial intelligence. Since they are so widely used, it is important that we, as practitioners and researchers, think about the impact these systems may have on users, society, and ...
NORMalize: A Tutorial on the Normative Design and Evaluation of Information Access Systems
CHIIR '24: Proceedings of the 2024 Conference on Human Information Interaction and RetrievalInformation access systems, such as Google News or YouTube, increasingly employ algorithms to rank diverse content such as music, recipes, and news articles. Acknowledging the influential role of these algorithms as gatekeepers to online content, the ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2024
1438 pages
ISBN:9798400705052
DOI:10.1145/3640457
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
- SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web
- SIGAI: ACM Special Interest Group on Artificial Intelligence
- SIGKDD: ACM Special Interest Group on Knowledge Discovery in Data
- SIGIR: ACM Special Interest Group on Information Retrieval
- SIGCHI: ACM Special Interest Group on Computer-Human Interaction
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 October 2024
Check for updates
Author Tags
Qualifiers
- Extended-abstract
- Research
- Refereed limited
Funding Sources
- LTP KIC
- Research Foundation Flanders
- Platform Intelligence in News
- ROBUST: Trustworthy AI-based Systems for Sustainable Growth
- Innovation Foundation Denmark
- MediaFutures: Research Centre for Responsible Media Technology and Innovation
Conference
Acceptance Rates
Overall Acceptance Rate 254 of 1,295 submissions, 20%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 51Total Downloads
- Downloads (Last 12 months)51
- Downloads (Last 6 weeks)15
Reflects downloads up to 10 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format