More Web Proxy on the site http://driver.im/

short-paper

A simple multi-armed nearest-neighbor bandit for interactive recommendation

Authors:

Javier Sanz-Cruzado,

Pablo Castells,

Esther LópezAuthors Info & Claims

RecSys '19: Proceedings of the 13th ACM Conference on Recommender Systems

Pages 358 - 362

https://doi.org/10.1145/3298689.3347040

Published: 10 September 2019 Publication History

Abstract

The cyclic nature of the recommendation task is being increasingly taken into account in recommender systems research. In this line, framing interactive recommendation as a genuine reinforcement learning problem, multi-armed bandit approaches have been increasingly considered as a means to cope with the dual exploitation/exploration goal of recommendation. In this paper we develop a simple multi-armed bandit elaboration of neighbor-based collaborative filtering. The approach can be seen as a variant of the nearest-neighbors scheme, but endowed with a controlled stochastic exploration capability of the users' neighborhood, by a parameter-free application of Thompson sampling. Our approach is based on a formal development and a reasonably simple design, whereby it aims to be easy to reproduce and further elaborate upon. We report experiments using datasets from different domains showing that neighbor-based bandits indeed achieve recommendation accuracy enhancements in the mid to long run.

References

[1]

P. Auer, N. Cesa-Bianchi and P. Fischer (2002). Finite-time Analysis of the Multiarmed Bandit Problem. Machine Learning, 47 (May 2002), 235--256.

Digital Library

[2]

P. Auer, N. Cesa-Bianchi, Y. Freund and R. E. Schapire (2003). The nonstochastic multiarmed bandit problem. SIAM journal on computing, 32, 1 (January 2003), 48--77.

Digital Library

[3]

R. Cañamares and P. Castells (2017). A Probabilistic Reformulation of Memory-Based Collaborative Filtering - Implications and Popularity Biases. In Proceedings of the 40<sup>th</sup> Annual International Conference on Research and Development in Information Retrieval (SIGIR 2017). ACM, New York, NY, USA, 215--224.

Digital Library

[4]

O. Chapelle and L. Li (2011). An empirical evaluation of Thompson Sampling. In Proceedings of Neural Information Processing Systems (NIPS 2011). Curran Associates, Inc., Red Hook, NY, USA, 2249--2257.

Digital Library

[5]

C. Gentile, S. Li and G. Zappella (2014). Online Clustering of Bandits. In Proceedings of the 31<sup>st</sup> International Conference on Machine Learning (ICML 2014). Proceedings of Machine Learning Research, Sheffield, UK, 757--765.

Digital Library

[6]

Y. Hu, Y. Koren and C. Volinsky (2008). Collaborative Filtering for Implicit Feedback Datasets. In Proceedings of the 8<sup>th</sup> IEEE International Conference on Data Mining (ICDM 2008). IEEE Computer Society, Washington, DC, USA, 15--19.

Digital Library

[7]

J. Kawale, H. H. Bui, B. Kveton, L. Tran-Thanh and S. Chawla (2015). Efficient Thompson Sampling for Online Matrix-Factorization Recommendation. In Proceedings of Neural Information Processing Systems (NIPS 2015). Curran Associates, Inc., Red Hook, NY, USA, 1297--1305.

Digital Library

[8]

L. Li, W. Chu, J. Langford and R. Schapire (2010). A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19<sup>th</sup> International Conference on World Wide Web (WWW 2010). ACM, New York, NY, USA, 661--670.

Digital Library

[9]

S. Li, A. Karatzoglou and C. Gentile (2016). Collaborative Filtering Bandits. In Proceedings of the 39<sup>th</sup> International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2016). ACM New York, NY, USA, 539--548.

Digital Library

[10]

J. Louëdec, M. Chevalier, J. Mothe, A. Garivier and S Gerchinovitz (2015). A Multiple-Play Bandit Algorithm Applied to Recommender Systems. In Proceedings of the 28<sup>th</sup> International Florida Artificial Intelligence Research Society Conference (FLAIRS 2015). AAAI Press, Menlo Park, CA, USA, 67--72.

[11]

F. M. Maxwell and J. A. Konstan (2015). The MovieLens Datasets: History and Context. ACM Transactions on Interactive Intelligent Systems, 5, 4 (December 2015).

Digital Library

[12]

J. McInerney, B. Lacker, S. Hansen, K. Higley, H. Bouchard, A. Gruson and R. Mehrotra (2018). Explore, exploit, and explain: personalizing explainable recommendations with bandits. In Proceedings of the 12<sup>th</sup> ACM Conference on Recommender Systems (RecSys 2018). ACM, New York, NY, USA, 31--39.

Digital Library

[13]

X. Ning, C. Desrosiers and G. Karypis (2015). A Comprehensive Survey of Neighborhood-Based Recommender Systems. In: F. Ricci, L. Rokach and B. Shapira (Eds.), Recommender Systems Handbook (2<sup>nd</sup> ed.). Springer, New York, NY, USA, 37--76.

[14]

I. Pilászy, D. Zibriczky and D. Tikk (2010). Fast ALS-based Matrix Factorization for Explicit and Implicit Feedback Datasets. In Proceedings of the 4<sup>th</sup> ACM Conference on Recommender Systems (RecSys 2010). ACM, New York, NY, USA, 71--78.

Digital Library

[15]

R. Salakhutdinov and A. Mnih (2007). Probabilistic matrix factorization. In Proceedings of Neural Information Processing Systems (NIPS 2011). Curran Associates, Inc., Red Hook, NY, USA, 1257--1264.

Digital Library

[16]

P. Sánchez and A. Bellogín (2018). A novel approach for venue recommendation using cross-domain techniques. In Proceedings of the 2<sup>nd</sup> Workshop on Intelligent Recommender Systems by Knowledge Transfer and Learning (RecSysKTL) at the 12<sup>th</sup> ACM Conference on Recommender Systems (RecSys 2018). ACM, New York, NY, USA, 260--268.

Digital Library

[17]

J. Sanz-Cruzado and P. Castells (2018). Contact Recommendations in Social Networks. In: I. Cantador, S. Berkovsky, D. Tikk (Eds.), Collaborative Recommendations: Algorithms, Practical Challenges and Applications. World Scientific Publishing, Singapore, 2018, 519--569.

[18]

J. Sanz-Cruzado and P. Castells (2018). Enhancing Structural Diversity in Social Networks by Recommending Weak Ties. In Proceedings of the 12<sup>th</sup> ACM Conference on Recommender Systems (RecSys 2018). ACM, New York, NY, USA, 233--241.

Digital Library

[19]

G. Shani, D. Heckerman and R. I. Brafman (2005). An MDP-Based Recommender System. Journal of Machine Learning Research 6 (December 2005), 1265--1295.

Digital Library

[20]

R. Sutton and A. Barto (2018). Reinforcement Learning: An Introduction (2<sup>nd</sup> ed.). MIT Press, Cambridge, MA, USA, 2018.

Digital Library

[21]

Q. Wang, C. Zeng, W. Zhou, T. Li, S. S. Iyengar, L. Shwartz and G. Grabarnik (2019). Online Interactive Collaborative Filtering Using Multi-Armed Bandit with Dependent Arms. IEEE Transactions on Knowledge and Data Engineering, 31, 8 (August 2019), 1569--1580.

Digital Library

[22]

D. Yang, D. Zhang, V. W. Zheng and Z. Yu (2015). Modeling User Activity Preference by Leveraging User Spatial Temporal Characteristics in LBSNs. IEEE Transactions on Systems, Man and Cybernetics: Systems, 45, 1 (January 2015), 129--142.

[23]

X. Zhao, W. Zhang and J. Wang (2013). Interactive Collaborative Filtering. In Proceedings of the 22<sup>nd</sup> ACM International Conference on Information and Knowledge Management (CIKM 2013). ACM, New York, NY, USA, 1411--1420.

Digital Library

Cited By

Mohammadi ASoleimani AParvin S(2024)Dynamic Strategy Optimizer (DSO): Application In Enhancing New User Engagement in Hybrid Recommender System2024 IEEE 15th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON)10.1109/UEMCON62879.2024.10754751(750-756)Online publication date: 17-Oct-2024
https://doi.org/10.1109/UEMCON62879.2024.10754751
Xu XZhou QWang QCao L(2024)SWCB: An Efficient Switch-Clustering of Bandit Model2024 36th Chinese Control and Decision Conference (CCDC)10.1109/CCDC62350.2024.10587841(1066-1071)Online publication date: 25-May-2024
https://doi.org/10.1109/CCDC62350.2024.10587841
Ashraf Cheema AShahzad Sarfraz MUsman MUz Zaman QHabib UBoonchieng E(2024)KT-CDULF: Knowledge Transfer in Context-Aware Cross-Domain Recommender Systems via Latent User ProfilingIEEE Access10.1109/ACCESS.2024.343019312(102111-102125)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3430193
Show More Cited By

Index Terms

A simple multi-armed nearest-neighbor bandit for interactive recommendation
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Reinforcement learning
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems
  2. World Wide Web
    1. Web searching and information discovery
      1. Collaborative filtering

Recommendations

Multi-armed recommender system bandit ensembles
RecSys '19: Proceedings of the 13th ACM Conference on Recommender Systems

It has long been found that well-configured recommender system ensembles can achieve better effectiveness than the combined systems separately. Sophisticated approaches have been developed to automatically optimize the ensembles' configuration to ...
A Multi-Armed Bandit Model Selection for Cold-Start User Recommendation
UMAP '17: Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization

How can we effectively recommend items to a user about whom we have no information? This is the problem we focus on in this paper, known as the cold-start problem. In most existing works, the cold-start problem is handled through the use of many kinds ...
Ballooning Multi-Armed Bandits
AAMAS '20: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems

We introduce ballooning multi-armed bandits (BL-MAB), a novel extension to the classical stochastic MAB model. In the BL-MAB model, the set of available arms grows (or balloons) over time. The regret in a BL-MAB setting is computed with respect to the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

RecSys '19: Proceedings of the 13th ACM Conference on Recommender Systems

September 2019

635 pages

ISBN:9781450362436

DOI:10.1145/3298689

General Chairs:
Toine Bogers
Aalborg University Copenhagen, Denmark
,
Alan Said
University of Gothenburg, Sweden
,
Program Chairs:
Peter Brusilovsky
University of Pittsburgh
,
Domonkos Tikk
Gravity R&D, Hungary

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 September 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Ministerio de Ciencia, Innovación y Universidades

Conference

RecSys '19

RecSys '19: Thirteenth ACM Conference on Recommender Systems

September 16 - 20, 2019

Copenhagen, Denmark

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
670
Total Downloads

Downloads (Last 12 months)48
Downloads (Last 6 weeks)4

Reflects downloads up to 16 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mohammadi ASoleimani AParvin S(2024)Dynamic Strategy Optimizer (DSO): Application In Enhancing New User Engagement in Hybrid Recommender System2024 IEEE 15th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON)10.1109/UEMCON62879.2024.10754751(750-756)Online publication date: 17-Oct-2024
https://doi.org/10.1109/UEMCON62879.2024.10754751
Xu XZhou QWang QCao L(2024)SWCB: An Efficient Switch-Clustering of Bandit Model2024 36th Chinese Control and Decision Conference (CCDC)10.1109/CCDC62350.2024.10587841(1066-1071)Online publication date: 25-May-2024
https://doi.org/10.1109/CCDC62350.2024.10587841
Ashraf Cheema AShahzad Sarfraz MUsman MUz Zaman QHabib UBoonchieng E(2024)KT-CDULF: Knowledge Transfer in Context-Aware Cross-Domain Recommender Systems via Latent User ProfilingIEEE Access10.1109/ACCESS.2024.343019312(102111-102125)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3430193
Zhang EMa WZhang JXia X(2023)A Service Recommendation System Based on Dynamic User Groups and Reinforcement LearningElectronics10.3390/electronics1224503412:24(5034)Online publication date: 17-Dec-2023
https://doi.org/10.3390/electronics12245034
Andrade YSilva NSilva TPereira ADias DAlbergaria ERocha L(2023)A Complete Framework for Offline and Counterfactual Evaluations of Interactive Recommendation SystemsProceedings of the 29th Brazilian Symposium on Multimedia and the Web10.1145/3617023.3617049(193-197)Online publication date: 23-Oct-2023
https://dl.acm.org/doi/10.1145/3617023.3617049
Bölz FNurbakova DCalabretto SGerl ABrunie LKosch H(2023)HUMMUS: A Linked, Healthiness-Aware, User-centered and Argument-Enabling Recipe Data Set for RecommendationProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3609491(1-11)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3609491
Zhu ZVan Roy BFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Scalable Neural Contextual Bandit for Recommender SystemsProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615048(3636-3646)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615048
Silva NSilva TWerneck HRocha LPereira A(2023)User Cold-start Problem in Multi-armed Bandits: When the First Recommendations Guide the User’s ExperienceACM Transactions on Recommender Systems10.1145/35548191:1(1-24)Online publication date: 27-Jan-2023
https://dl.acm.org/doi/10.1145/3554819
Silva NSilva THott HRibeiro YPereira ARocha LChen HDuh WHuang HKato MMothe JPoblete B(2023)Exploring Scenarios of Uncertainty about the Users' Preferences in Interactive Recommendation SystemsProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591684(1178-1187)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591684
Feng WShi HZhao PGao X(2023)Mixtron: Bandit Online Multiclass Prediction with Implicit Feedback2023 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM58522.2023.00115(1004-1012)Online publication date: 1-Dec-2023
https://doi.org/10.1109/ICDM58522.2023.00115
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents