More Web Proxy on the site http://driver.im/

article

Estimating confidence of individual rating predictions in collaborative filtering recommender systems

Author:

Maciej A. MazurowskiAuthors Info & Claims

Expert Systems with Applications: An International Journal, Volume 40, Issue 10

Pages 3847 - 3857

https://doi.org/10.1016/j.eswa.2012.12.102

Published: 01 August 2013 Publication History

Abstract

Collaborative filtering algorithms predict a rating for an item based on the user's previous ratings for other items as well as ratings of other users. This approach has gained notable popularity both in academic research and in commercial applications. One aspect of collaborative filtering systems that received interest, but little systematic analysis, is confidence of the rating predictions by collaborative filtering algorithms. In this paper, I address this issue. Specifically: (1) I offer a discussion on the definition of confidence, (2) I propose a method for evaluating performance of confidence estimation algorithms, and (3) I evaluate six different confidence estimation algorithms. Three of those algorithms are introduced in this paper and three have been previously suggested for this purpose. The comparative experimental evaluation demonstrates that two of the algorithms proposed in this study: one using resampling of available ratings and one using noise injection to the available ratings provide the best performance in terms of separation between predictions of high and low confidence. The algorithms that use only the number of ratings available for the user of interest or for the item of interest turned out to be of limited use for confidence estimation.

References

[1]

Adomavicius, G., Kamireddy, S., & Kwon, Y. O. (2007). Towards more confident recommendations: Improving recommender systems using filtering approach based on rating variance. In Proceedings of 17th workshop on information technology and systems (WITS'07).

[2]

Bennett, J., Lanning, S., & Netflix, N. (2007). The netflix prize. In KDD cup and workshop in conjunction with KDD.

[3]

Collaborative filtering recommender systems. Springer-Verlag, Berlin, Heidelberg.

[4]

Breese, J. S., Heckerman, D., & Kadie, C. (1998). Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence (pp. 43-52). Morgan Kaufmann Publishers Inc.

[5]

An introduction to roc analysis. Pattern Recognition Letters. v27. 861-874.

Digital Library

[6]

A survey of accuracy evaluation metrics of recommendation tasks. Journal of Machine Learning Research. v10. 2935-2962.

Digital Library

[7]

Habas, P. A., Zurada, J. M., Elmaghraby, A. S., & Tourassi, G. D. (2007). Reliability analysis framework for computer-assisted medical decision systems. Medical physics, 34, 763.

[8]

Explaining collaborative filtering recommendations. In: Proceedings of the 2000 ACM conference on computer supported cooperative work, ACM. pp. 241-250.

Digital Library

[9]

Estimating the confidence interval for prediction errors of support vector machine classifiers. Journal of Machine Learning Research. v9. 521-540.

Digital Library

[10]

Attribute selection-based recommendation framework for short-head user group: An empirical study by movielens and imdb. Expert Systems with Applications. v39. 4049-4054.

Digital Library

[11]

Koren, Y., & Sill, J. (2011). Ordrec: An ordinal model for predicting personalized item rating distributions. In ACM Recommender Systems 2011.

Digital Library

[12]

Advances in collaborative filtering. Springer, US, Boston, MA, USA.

[13]

Matrix factorization techniques for recommender systems. Computer. v42 i8. 30-37.

Digital Library

[14]

Combining pattern classifiers. Willey-Interscience.

[15]

Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Computing. v7 i1. 76-80.

Digital Library

[16]

Mazurowski, M. A., Zurada, J. M., & Tourassi, G. D. (2008) Reliability assessment of ensemble classifiers: Application in mammography. In International workshop on Digital Mammography, Lecture notes in computer science (Vol. 5116, pp. 366-370). Tucson, AZ, USA.

Digital Library

[17]

Comparative analysis of instance selection algorithms for instance-based classifiers in the context of medical decision support. Physics in Medicine and Biology. v56. 473-489.

[18]

A collaborative filtering algorithm and evaluation metric that accurately model the user experience. In: Proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval, ACM. pp. 329-336.

Digital Library

[19]

McNee, S. M., Lam, S. K., Guetzlaff, C., Konstan, J. A., & Riedl, J. (2003). Confidence displays and training in recommender systems. In Proceedings of IFIP TC13 international conference on human-computer interaction.

[20]

Sarwar, B., Karypis, G., Konstan, J., & Reidl, J. (2001). Item-based collaborative filtering recommendation algorithms. In World Wide Web (pp. 285-295).

Digital Library

[21]

A tutorial on conformal prediction. Journal of Machine Learning Research. v9. 371-421.

Digital Library

[22]

Evaluating recommendation systems. Springer, US, Boston, MA, USA.

[23]

Su, X., & Khoshgoftaar, T. M. (2009). A survey of collaborative filtering techniques. Advances in Artificial Intelligence, 2009:N/A.

Digital Library

[24]

Tintarev, N., & Masthoff, J. (2011). Recommender Systems Handbook, (pp. 479-510) {chap. Designing and Evaluating Explanations for Recommender Systems}.

[25]

Wang, J. (2009). Mean-variance analysis: A new document ranking theory in information retrieval. In Advances in information retrieval (pp. 4-16).

Digital Library

[26]

Wang, X., & Du, J. (2009). A collaborative filtering algorithm based on variance analysis of attributes-value preference. In International Conference on Management of e-Commerce and e-Government (pp. 24-27). IEEE.

Digital Library

[27]

Machine learning for user modeling. User Modeling and User-Adapted Interaction. v11. 19-29.

Digital Library

[28]

Jackknife, bootstrap and other resampling methods in regression analysis. The Annals of Statistics. v14. 1261-1295.

[29]

Scalable collaborative filtering using cluster-based smoothing. In: Proceedings of the 28th annual international ACM SIGIR conference on research and development in information retrieval, SIGIR '05, ACM, New York, NY, USA. pp. 114-121.

Digital Library

[30]

Zuccon, G., Azzopardi, L., & van Rijsbergen, K. (2011). Back to the roots: Mean-variance analysis of relevance estimations. In Advances in information retrieval.

Digital Library

Cited By

Knyazev NOosterhuis H(2023)A Lightweight Method for Modeling Confidence in Recommendations with Learned Beta DistributionsProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608788(306-317)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608788
Coscrato VBridge D(2023)Estimating and Evaluating the Uncertainty of Rating Predictions and Top-n Recommendations in Recommender SystemsACM Transactions on Recommender Systems10.1145/35840211:2(1-34)Online publication date: 24-Apr-2023
https://dl.acm.org/doi/10.1145/3584021
Li YWu GLiu C(2022)Research on Intelligent Recommendation Model of E-Commerce Commodity Based on Feature Selection and Deep Belief NetworkSecurity and Communication Networks10.1155/2022/64692172022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/6469217
Show More Cited By

Recommendations

Estimating Confidence of Individual User Predictions in Item-based Recommender Systems
UMAP '19: Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization

This paper focuses on recommender systems based on item-item collaborative filtering (CF). Although research on item-based methods is not new, current literature does not provide any reliable insight on how to estimate confidence of recommendations. The ...
Trust-based collaborative filtering: tackling the cold start problem using regular equivalence
RecSys '18: Proceedings of the 12th ACM Conference on Recommender Systems

User-based Collaborative Filtering (CF) is one of the most popular approaches to create recommender systems. This approach is based on finding the most relevant k users from whose rating history we can extract items to recommend. CF, however, suffers ...
A recursive prediction algorithm for collaborative filtering recommender systems
RecSys '07: Proceedings of the 2007 ACM conference on Recommender systems

Collaborative filtering (CF) is a successful approach for building online recommender systems. The fundamental process of the CF approach is to predict how a user would like to rate a given item based on the ratings of some nearest-neighbor users (user-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Expert Systems with Applications: An International Journal

Expert Systems with Applications: An International Journal Volume 40, Issue 10

August, 2013

427 pages

ISSN:0957-4174

Issue’s Table of Contents

Copyright © Elsevier Ltd © 2013.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 August 2013

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Knyazev NOosterhuis H(2023)A Lightweight Method for Modeling Confidence in Recommendations with Learned Beta DistributionsProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608788(306-317)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608788
Coscrato VBridge D(2023)Estimating and Evaluating the Uncertainty of Rating Predictions and Top-n Recommendations in Recommender SystemsACM Transactions on Recommender Systems10.1145/35840211:2(1-34)Online publication date: 24-Apr-2023
https://dl.acm.org/doi/10.1145/3584021
Li YWu GLiu C(2022)Research on Intelligent Recommendation Model of E-Commerce Commodity Based on Feature Selection and Deep Belief NetworkSecurity and Communication Networks10.1155/2022/64692172022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/6469217
Treuillier CCastagnos SDufraisse EBrun ABellogin ABoratto LSantos OArdissono LKnijnenburg B(2022)Being Diverse is Not Enough: Rethinking Diversity Evaluation to Meet Challenges of News Recommender SystemsAdjunct Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3511047.3538030(222-233)Online publication date: 4-Jul-2022
https://dl.acm.org/doi/10.1145/3511047.3538030
Wang JRen YLi JDeng K(2021)The Footprint of Factorization Models and Their Applications in Collaborative FilteringACM Transactions on Information Systems10.1145/349047540:4(1-32)Online publication date: 29-Nov-2021
https://dl.acm.org/doi/10.1145/3490475
Bernardis CCremonesi P(2021)Eigenvalue Perturbation for Item-based Recommender SystemsProceedings of the 15th ACM Conference on Recommender Systems10.1145/3460231.3478862(656-660)Online publication date: 13-Sep-2021
https://dl.acm.org/doi/10.1145/3460231.3478862
Mesas RBellogín A(2020)Exploiting recommendation confidence in decision-aware recommender systemsJournal of Intelligent Information Systems10.1007/s10844-018-0526-354:1(45-78)Online publication date: 1-Feb-2020
https://dl.acm.org/doi/10.1007/s10844-018-0526-3
Tan WHe YZhu B(2020)Improvement of Co-training Based Recommender System with Machine LearningArtificial Intelligence and Security10.1007/978-3-030-57881-7_44(499-509)Online publication date: 17-Jul-2020
https://dl.acm.org/doi/10.1007/978-3-030-57881-7_44
Bernardis CFerrari Dacrema MCremonesi PPapadopoulos GSamaras GWeibelzahl SJannach DSantos O(2019)Estimating Confidence of Individual User Predictions in Item-based Recommender SystemsProceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3320435.3320453(149-156)Online publication date: 7-Jun-2019
https://dl.acm.org/doi/10.1145/3320435.3320453
Ahmadian SAfsharchi MMeghdadi M(2019)A novel approach based on multi-view reliability measures to alleviate data sparsity in recommender systemsMultimedia Tools and Applications10.1007/s11042-018-7079-x78:13(17763-17798)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1007/s11042-018-7079-x
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents