More Web Proxy on the site http://driver.im/

research-article

Measuring the diversity of recommendations: a preference-aware approach for evaluating and adjusting diversity

Authors:

Rouzbeh Meymandpour,

Joseph G. DavisAuthors Info & Claims

Knowledge and Information Systems, Volume 62, Issue 2

Pages 787 - 811

https://doi.org/10.1007/s10115-019-01371-0

Published: 01 February 2020 Publication History

Abstract

While recent research has highlighted the importance of identifying diverse aspects of user preferences in terms of the quality of recommendations, most of the widely used performance measures tend to consider the accuracy of the recommendations and ignore other important aspects such as preference for diversity in recommendations. This is despite the emerging consensus that improving the diversity in recommendations allows users to discover a wider variety of items and encourage them to extend their range of interests in domains such as books, movies, and music. By proposing a novel diversity evaluation metric, this paper aims to address the problem of measuring the diversity with respect to the distribution of preferences for diversity among recommendation system users. We perform several experiments in order to provide a better understanding of the diversity preferences of users and present the results of diversity evaluations of several recommendation methods. These experiments highlight the accuracy–diversity trade-off and show that higher accuracy does not lead to higher performance in terms of the diversity of the recommendations and that the users’ preferred level of diversity should be considered when designing and evaluating recommender systems. This paper also proposes our Diversity Adjustment algorithm that modifies the diversity of recommendations to suit each user’s preferences while preserving the accuracy. Our experiments suggest that diversifying the recommendations without considering the user’s preferences can lead to a dramatic decline in accuracy, while adjusting the diversity based on users’ diversity needs can support recommender systems in maintaining overall accuracy.

References

[1]

Adomavicius G and Tuzhilin A Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions IEEE Trans Knowl Data Eng 2005 17 6 734-749

[2]

Bell RM and Koren Y Lessons from the Netflix prize challenge ACM SIGKDD Explor Newsl 2007 9 2 75-79

[3]

Paterek A (2007) Improving regularized singular value decomposition for collaborative filtering. In: KDD cup and workshop, pp 5–8

[4]

Koren Y (2008) Factorization meets the neighborhood: a multifaceted collaborative filtering model. In: 14th ACM SIGKDD international conference on knowledge discovery and data mining, Las Vegas, NV, USA, pp 426–434

[5]

Koren Y, Bell R, and Volinsky C Matrix factorization techniques for recommender systems Computer 2009 42 8 30-37

[6]

Takács G, Pilászy I, Németh B, and Tikk D Scalable collaborative filtering approaches for large recommender systems J Mach Learn Res 2009 10 623-656

[7]

Herlocker JL, Konstan JA, Terveen LG, and Riedl JT Evaluating collaborative filtering recommender systems ACM Trans Inf Syst 2004 22 1 5-53

[8]

Shani G and Gunawardana A Ricci F, Rokach L, Shapira B, and Kantor PB Evaluating recommendation systems Recommender systems handbook 2011 New York Springer 257-297

[9]

Ge M, Delgado-Battenfeld C, Jannach D (2010) Beyond accuracy: evaluating recommender systems by coverage and serendipity. In: 4th ACM conference on recommender systems, Barcelona, Spain, pp 257–260

[10]

Ziegler C-N, McNee SM, Konstan JA, Lausen G (2005) Improving recommendation lists through topic diversification. In: 14th International conference on World Wide Web, Chiba, Japan, pp 22–32

[11]

Murakami T, Mori K, and Orihara R Satoh K, Inokuchi A, Nagao K, and Kawamura T Metrics for evaluating the serendipity of recommendation lists New frontiers in artificial intelligence, lecture notes in computer science 2008 Berlin Springer 40-46

[12]

Hurley N and Zhang M Novelty and diversity in top-N recommendation—analysis and evaluation ACM Trans Internet Technol 2011 10 4 1-30

[13]

Shi L (2013) Trading-off among accuracy, similarity, diversity, and long-tail: a graph-based recommendation approach. In: 7th ACM conference on recommender systems, Hong Kong, China, pp 57–64

[14]

Konstan JA, Miller BN, Maltz D, Herlocker JL, Gordon LR, and Riedl J GroupLens: applying collaborative filtering to usenet news Commun ACM 1997 40 3 77-87

[15]

McNee SM, Albert I, Cosley D, Gopalkrishnan P, Lam SK, Rashid AM, Konstan JA, Riedl J (2002) On the recommending of citations for research papers. In: Proceedings of the 2002 ACM conference on computer supported cooperative work, pp 116–125

[16]

Iaquinta L, de Gemmis M, Lops P, Semeraro G, Filannino M, Molino P (2008) Introducing serendipity in a content-based recommender system. In: 8th International conference on hybrid intelligent systems (HIS ’08), pp 168–173

[17]

Meymandpour R and Davis JG A semantic similarity measure for linked data: an information content-based approach Knowl Based Syst 2016 109 276-293

[18]

Meymandpour R, Davis JG (2015) Enhancing recommender systems using linked open data-based semantic analysis of items. In: Proceedings of the 3rd Australasian web conference (AWC 2015), Sydney, Australia, pp 11–17

[19]

Jones N, Pu P (2007) User technology adoption issues in recommender systems. In: Networking and electronic conference

[20]

McNee SM, Riedl J, Konstan JA (2006) Being accurate is not enough: how accuracy metrics have hurt recommender systems. In: CHI ‘06 extended abstracts on human factors in computing systems, Montreal, QC, Canada, 2006, pp 1097–1101

[21]

Billsus D and Pazzani M User modeling for adaptive news access User Model User Adapt Interact 2000 10 2–3 147-180

[22]

McGinty L, Smyth B (2003) On the role of diversity in conversational recommender systems. In: 5th International conference on case-based reasoning: research and development, Trondheim, Norway, pp 276–290

[23]

Agrawal R, Gollapudi S, Halverson A, Ieong S (2009) Diversifying search results. In: Proceedings of the 2nd ACM international conference on web search and data mining, pp 5–14

[24]

Santos RLT, Peng J, Macdonald C, Ounis I (2010) Explicit search result diversification through sub-queries. Berlin, Heidelberg, pp 87–99

[25]

Radlinski F, Dumais S (2006) Improving personalized web search using result diversification. In: Proceedings of the 29th annual international ACM SIGIR conference on research and development in information retrieval, pp 691–692

[26]

Carterette B, Chandar P (2009) Probabilistic models of novel document rankings for faceted topic retrieval. In: Proceedings of CIKM

[27]

Vargas S, Castells P, Vallet D (2011) Intent-oriented diversity in recommender systems. In: Proceedings of the 34th international ACM SIGIR conference on research and development in information retrieval, Beijing, China, pp 1211–1212

[28]

Carbonell J, Goldstein J (1998) The use of MMR, diversity-based reranking for reordering documents and producing summaries. In: Proceedings of the 21st annual international ACM SIGIR conference on research and development in information retrieval, pp 335–336

[29]

Adomavicius G and Kwon Y Improving aggregate recommendation diversity using ranking-based techniques IEEE Trans Knowl Data Eng 2012 24 5 896-911

[30]

Zhang M, Hurley N (2008) Avoiding monotony: improving the diversity of recommendation lists. In: 2008 ACM conference on recommender systems, Lausanne, Switzerland, pp 123–130

[31]

Oh J, Park S, Yu H, Song M, Park S-T (2011) Novel recommendation based on personal popularity tendency. In: Proceedings of the 2011 IEEE 11th international conference on data mining, 2011, pp 507–516

[32]

Jugovac M, Jannach D, and Lerche L Efficient optimization of multiple recommendation quality factors according to individual user tendencies Expert Syst Appl 2017 81 C 321-331

[33]

Nagatani K, Sato M (2017) Accurate and diverse recommendation based on users’ tendencies toward temporal item popularity. In: RecTemp@ RecSys, pp 35–39

[34]

Ekstrand MD, Harper FM, Willemsen MC, Konstan JA (2014) User perception of differences in recommender algorithms. In: Proceedings of the 8th ACM conference on recommender systems, Foster City, Silicon Valley, CA, USA, pp 161–168

[35]

Ge M, Jannach D, Gedikli F (2013) Bringing diversity to recommendation lists—an analysis of the placement of diverse items. Berlin, pp 293–305

[36]

Steck H (2018) Calibrated recommendations. In: Proceedings of the 12th ACM conference on recommender systems, pp 154–162

[37]

Bradley K, Smyth B (2001) Improving recommendation diversity. In: 12th National conference in artificial intelligence and cognitive science (AICS-01), pp 75–84

[38]

Willmott CJ and Matsuura K On the use of dimensioned measures of error to evaluate the performance of spatial interpolators Int J Geogr Inf Sci 2006 20 1 89-102

[39]

Pontius RG, Thontteh O, and Chen HComponents of information for multiple resolution comparison between maps that share a real variableEnviron Ecol Stat2008152111-1422399076

[40]

Vallet D, Castells P (2012) Personalized diversification of search results. In: Proceedings of the 35th international ACM SIGIR conference on research and development in information retrieval, Portland, OR, USA, pp 841–850

[41]

Deshpande M and Karypis G Item-based top-N recommendation algorithms ACM Trans Inf Syst 2004 22 1 143-177

[42]

Karypis G (2001) Evaluation of item-based top-N recommendation algorithms. In: 10th International conference on information and knowledge management, Atlanta, GA, USA, pp 247–254

[43]

Gantner Z, Rendle S, Freudenthaler C, Schmidt-Thieme L (2011) MyMediaLite: a free recommender system library. In: 5th ACM conference on recommender systems, Chicago, IL, USA, pp 305–308

[44]

Sarwar B, Karypis G, Konstan J, Riedl J (2001) Item-based collaborative filtering recommendation algorithms. In: 10th International conference on World Wide Web, Hong Kong, Hong Kong, pp 285–295

[45]

Candillier L, Meyer F, Boulle M (2007) Comparing state-of-the-art collaborative filtering systems. In: 5th International conference on machine learning and data mining in pattern recognition, Leipzig, Germany, pp 548–562

[46]

Schafer JB, Frankowski D, Herlocker J, and Sen S Brusilovsky P, Kobsa A, and Nejdl W Collaborative filtering recommender systems The adaptive web, lecture notes in computer science 2007 Berlin Springer 291-324

[47]

Su X and Khoshgoftaar TM A survey of collaborative filtering techniques Adv Artif Intell 2009 2009 1-19

[48]

Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L (2009) BPR: Bayesian personalized ranking from implicit feedback. In: 25th Conference on uncertainty in artificial intelligence, Montreal, QC, Canada, pp 452–461

[49]

Pan R, Zhou Y, Cao B, Liu NN, Lukose R, Scholz M, Yang Q (2008) One-class collaborative filtering. In: 8th IEEE international conference on data mining (ICDM ‘08), pp 502–511

[50]

Hu Y, Koren Y, Volinsky C (2008) Collaborative filtering for implicit feedback datasets. In: 8th IEEE International conference on data mining, pp 263–272

[51]

Rendle S Context-aware ranking with factorization models 2010 Berlin Springer

[52]

Xia N, Karypis G (2011) SLIM: sparse linear methods for top-N recommender systems. In: 11th IEEE international conference on data mining (ICDM’11), pp 497–506

[53]

Friedman J, Hastie T, and Tibshirani R Regularization paths for generalized linear models via coordinate descent J Stat Softw 2010 33 1 1

[54]

Kalia A. “Music was better back then”: When do we stop keeping up with popular music? https://skynetandebert.com/author/amkalia/

Cited By

Lechiakh MEl-Moutaouakkil ZMaurer A(2024)Towards long-term depolarized interactive recommendationsInformation Processing and Management: an International Journal10.1016/j.ipm.2024.10383361:6Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1016/j.ipm.2024.103833
Du YRanwez SSutton-Charani NRanwez V(2022)Is diversity optimization always suitable? Toward a better understanding of diversity within recommendation approachesInformation Processing and Management: an International Journal10.1016/j.ipm.2021.10272158:6Online publication date: 22-Apr-2022
https://dl.acm.org/doi/10.1016/j.ipm.2021.102721
Parapar JRadlinski FLewin-Eytan LCarmel DYom-Tov EAgichtein EGabrilovich E(2021)Diverse User Preference Elicitation with Multi-Armed BanditsProceedings of the 14th ACM International Conference on Web Search and Data Mining10.1145/3437963.3441786(130-138)Online publication date: 8-Mar-2021
https://dl.acm.org/doi/10.1145/3437963.3441786

Recommendations

A Clustering Approach for Personalizing Diversity in Collaborative Recommender Systems
UMAP '17: Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization

Much of the focus of recommender systems research has been on the accurate prediction of users' ratings for unseen items. Recent work has suggested that objectives such as diversity and novelty in recommendations are also important factors in the ...
Enhancing the diversity of conversational collaborative recommendations: a comparison

In conversational collaborative recommender systems, user feedback influences the recommendations. We report mechanisms for enhancing the diversity of the recommendations made by collaborative recommenders. We focus on techniques for increasing ...
A Novel Framework for Improving Recommender Diversity
International Workshop on Behavior and Social Informatics on Behavior and Social Computing - Volume 8178

Recommender systems are being used to assist users in finding relevant items from a large set of alternatives in many online applications. However, while most research up to this point has focused on improving the accuracy of recommender systems, other ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Knowledge and Information Systems

Knowledge and Information Systems Volume 62, Issue 2

Feb 2020

412 pages

ISSN:0219-1377

Issue’s Table of Contents

© Springer-Verlag London Ltd., part of Springer Nature 2019.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 February 2020

Accepted: 10 June 2019

Revision received: 07 June 2019

Received: 24 April 2017

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lechiakh MEl-Moutaouakkil ZMaurer A(2024)Towards long-term depolarized interactive recommendationsInformation Processing and Management: an International Journal10.1016/j.ipm.2024.10383361:6Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1016/j.ipm.2024.103833
Du YRanwez SSutton-Charani NRanwez V(2022)Is diversity optimization always suitable? Toward a better understanding of diversity within recommendation approachesInformation Processing and Management: an International Journal10.1016/j.ipm.2021.10272158:6Online publication date: 22-Apr-2022
https://dl.acm.org/doi/10.1016/j.ipm.2021.102721
Parapar JRadlinski FLewin-Eytan LCarmel DYom-Tov EAgichtein EGabrilovich E(2021)Diverse User Preference Elicitation with Multi-Armed BanditsProceedings of the 14th ACM International Conference on Web Search and Data Mining10.1145/3437963.3441786(130-138)Online publication date: 8-Mar-2021
https://dl.acm.org/doi/10.1145/3437963.3441786

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents