More Web Proxy on the site http://driver.im/

Article

Usage-based web recommendations: a reinforcement learning approach

Authors:

Nima Taghipour,

Saeed Shiry GhidaryAuthors Info & Claims

RecSys '07: Proceedings of the 2007 ACM conference on Recommender systems

Pages 113 - 120

https://doi.org/10.1145/1297231.1297250

Published: 19 October 2007 Publication History

Abstract

Information overload is no longer news; the explosive growth of the Internet has made this issue increasingly serious for Web users. Users are very often overwhelmed by the huge amount of information and are faced with a big challenge to find the most relevant information in the right time. Recommender systems aim at pruning this information space and directing users toward the items that best meet their needs and interests. Web Recommendation has been an active application area in Web Mining and Machine Learning research. In this paper we propose a novel machine learning perspective toward the problem, based on reinforcement learning. Unlike other recommender systems, our system does not use the static patterns discovered from web usage data, instead it learns to make recommendations as the actions it performs in each situation. We model the problem as Q-Learning while employing concepts and techniques commonly applied in the web usage mining domain. We propose that the reinforcement learning paradigm provides an appropriate model for the recommendation problem, as well as a framework in which the system constantly interacts with the user and learns from her behavior. Our experimental evaluations support our claims and demonstrate how this approach can improve the quality of web recommendations.

References

[1]

Burke, R. Hybrid recommender systems: Survey and experiments. User Modeling and User-Adapted Interaction, 2002.

Digital Library

[2]

Cooley, R., Tan, P. N., Srivastava, J. WebSIFT: The Web Site Information Filter System. In Proceedings of the Web Usage Analysis and User Profiling Workshop (WEBKDD'99), 1999.

[3]

Deshpande, M., Karypis, G. Item-based top-N recommendation algorithms. ACM Transactions on Information Systems (TOIS), 2004.

Digital Library

[4]

Fu, X., Budzik, J., Hammond, K. J. Mining navigation history for recommendation. Intelligent User Interfaces, 2000.

Digital Library

[5]

Golovin, N., Rahm, E. Reinforcement Learning Architecture for Web Recommendations. Proceedings of the ITCC2004. IEEE, 2004.

Digital Library

[6]

Herlocker, J., Konstan, J., Brochers, A., Riedel, J. An Algorithmic Framework for Performing Collaborative Filtering. Proceedings of 200 Conference on Research and development in Information Retrieval, 2000.

Digital Library

[7]

Joachims, T., Freitag, D., Mitchell, T. M. WebWatcher: A tour guide for the world wide web. Proceedings of International Joint Conference on Artificial Intelligence, 1997.

[8]

Konstan, J., Riedl, J., Borchers, A., Herlocker, J. Recommender Systems: A GroupLens Perspective. In: Recommender Systems: Papers from the 1998 Workshop (AAAI Technical Report WS-98-08), 1998.

[9]

Li, J., Zaiane, O. R. Combining Usage, Content and Structure Data to Improve Web Site Recommendation, 5th International Conference on Electronic Commerce and Web, 2004.

[10]

Liu, J., Zhang, S., Yang, J. Characterizing Web usage regularities with information foraging agents. IEEE Transactions on Knowledge and Data Engineering, 16(5), 566--584. 2004.

Digital Library

[11]

Mobasher, B., Cooley, R., Srivastava, J. Automatic Personalization based on Web Usage Mining. Communications of the ACM. 43 (8), pp. 142--151, 2000.

Digital Library

[12]

Mobasher, B., Dai, H., Luo, T., Sun, Y., Zhu, J. Integrating web usage and content mining for more effective personalization. In EC-Web, pages 165--176, 2000.

Digital Library

[13]

Nakagawa M., Mobasher, B. A Hybrid Web Personalization Model Based on Site Connectivity. Proc. 5th WEBKDD workshop, 2003.

[14]

Resnick, P., Varian, H. R. Recommender Systems. Communications of the ACM, 40 (3), 56--58, 1997.

Digital Library

[15]

Shahabi, C., M. Zarkesh, A., Abidi, J., Shah, V. Knowledge discovery from user's Web-page navigation. In Proceedings of the 7th IEEE Intl. Workshop on Research Issues in Data Engineering, 1997.

Digital Library

[16]

Shany, G., Heckerman, D., Barfman, R. An MDP-Based Recommender System. Journal of Machine Learning Research, 2005.

Digital Library

[17]

Srivastava, J., Cooley, R., Deshpande, M., Tan, P. N. Web Usage Mining: Discovery and Applications of Usage Patterns from Web Data. SIGKDD Explorations, 1(2):12--23, 2000.

Digital Library

[18]

Srivihok, A., Sukonmanee, V. E-commerce intelligent agent: personalization travel support agent using Q Learning. ACM International Conference Proceeding Series; Proceedings of the 7th international conference on Electronic commerce, 2005.

Digital Library

[19]

Su, Z., Yang, Q., Lu, Y., Zhang, H. What next: A prediction System for Web Requests Using N-gram Sequence Models. In Proceedings of the First International Conference on Web Information Systems and Engineering Conference. 2000.

Digital Library

[20]

Sutton, R. S., Barto, A. G. Reinforcement Learning: An Introduction, MIT Press, Cambridge, 1998.

Digital Library

[21]

Wasfi, A. M. Collecting User Access Patterns for Building User Profiles and Collaborative Filtering. In: IUI '99: Proceedings of the 1999 International Conference on Intelligent User Interfaces. 1999.

Digital Library

[22]

Zhang, B., Seo, Y. Personalized web-document filtering using reinforcement learning. Applied Artificial Intelligence, 15(7):665--685, 2001.

Cited By

Liu ZLiu SYang BXue ZCai QZhao XZhang ZHu LLi HJiang PBaeza-Yates RBonchi F(2024)Modeling User Retention through Generative Flow NetworksProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671531(5497-5508)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671531
Wang XLiu SWang XCai QHu LLi HJiang PGai KXie GBaeza-Yates RBonchi F(2024)Future Impact Decomposition in Request-level RecommendationsProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671506(5905-5916)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671506
Liu ZLiu SZhang ZCai QZhao XZhao KHu LJiang PGai KHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Sequential Recommendation for Optimizing Both Immediate Feedback and Long-term RetentionProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657829(1872-1882)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657829
Show More Cited By

Index Terms

Usage-based web recommendations: a reinforcement learning approach
1. Computing methodologies
  1. Machine learning
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Document filtering
      2. Information extraction
  2. Information systems applications
    1. Data mining

Recommendations

Web personalization based on usage mining
FDIA'09: Proceedings of the Third BCS-IRSG conference on Future Directions in Information Access

Personalized or recommender systems are a particular type of information filtering applications. User profiles, representing the information needs and preferences of users, can be inferred from log or clickthrough data, or the ratings that users provide ...
A hybrid web recommender system based on Q-learning
SAC '08: Proceedings of the 2008 ACM symposium on Applied computing

Different efforts have been made to address the problem of information overload on the Internet. Recommender systems aim at directing users through this information space, toward the resources that best meet their needs and interests. Web Content ...
Integrating collaborative filtering and matching-based search for product recommendations

Currently, recommender systems (RS) have been widely applied in many commercial e-commerce sites to help users deal with the information overload problem. Recommender systems provide personalized recommendations to users and, thus, help in making good ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

RecSys '07: Proceedings of the 2007 ACM conference on Recommender systems

October 2007

222 pages

ISBN:9781595937308

DOI:10.1145/1297231

General Chair:
Joseph A. Konstan
University of Minnesota, USA
,
Program Chairs:
John Riedl
University of Minnesota, USA
,
Barry Smyth
University College, Dublin, Ireland

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

RecSys07

Sponsor:

RecSys07: ACM Conference on Recommender Systems

October 19 - 20, 2007

MN, Minneapolis, USA

Acceptance Rates

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

65
Total Citations
View Citations
1,756
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)3

Reflects downloads up to 15 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu ZLiu SYang BXue ZCai QZhao XZhang ZHu LLi HJiang PBaeza-Yates RBonchi F(2024)Modeling User Retention through Generative Flow NetworksProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671531(5497-5508)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671531
Wang XLiu SWang XCai QHu LLi HJiang PGai KXie GBaeza-Yates RBonchi F(2024)Future Impact Decomposition in Request-level RecommendationsProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671506(5905-5916)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671506
Liu ZLiu SZhang ZCai QZhao XZhao KHu LJiang PGai KHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Sequential Recommendation for Optimizing Both Immediate Feedback and Long-term RetentionProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657829(1872-1882)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657829
Zhou FLuo BWu ZHuang T(2024)SMONAC: Supervised Multiobjective Negative Actor–Critic for Sequential RecommendationIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.331735335:12(18525-18537)Online publication date: Dec-2024
https://doi.org/10.1109/TNNLS.2023.3317353
Lin YLiu YLin FZou LWu PZeng WChen HMiao C(2024)A Survey on Reinforcement Learning for Recommender SystemsIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.328016135:10(13164-13184)Online publication date: Oct-2024
https://doi.org/10.1109/TNNLS.2023.3280161
Wu XWang HZhang YZou BHong H(2024)A Tutorial-Generating Method for Autonomous Online LearningIEEE Transactions on Learning Technologies10.1109/TLT.2024.339059317(1558-1567)Online publication date: 2024
https://doi.org/10.1109/TLT.2024.3390593
Iftikhar AGhazanfar MAyub MAli Alahmari SQazi NWall J(2024)A reinforcement learning recommender system using bi-clustering and Markov Decision ProcessExpert Systems with Applications10.1016/j.eswa.2023.121541237(121541)Online publication date: Mar-2024
https://doi.org/10.1016/j.eswa.2023.121541
Stray JHalevy AAssar PHadfield-Menell DBoutilier CAshar ABakalar CBeattie LEkstrand MLeibowicz CMoon Sehat CJohansen SKerlin LVickrey DSingh SVrijenhoek SZhang AAndrus MHelberger NProutskova PMitra TVasan N(2023)Building Human Values into Recommender Systems: An Interdisciplinary SynthesisACM Transactions on Recommender Systems10.1145/36322972:3(1-57)Online publication date: 13-Nov-2023
https://dl.acm.org/doi/10.1145/3632297
Kasirzadeh AEvans C(2023)User Tampering in Reinforcement Learning Recommender SystemsProceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society10.1145/3600211.3604669(58-69)Online publication date: 8-Aug-2023
https://dl.acm.org/doi/10.1145/3600211.3604669
Liu ZTian JCai QZhao XGao JLiu SChen DHe TZheng DJiang PGai K(2023)Multi-Task Recommendations with Reinforcement LearningProceedings of the ACM Web Conference 202310.1145/3543507.3583467(1273-1282)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583467
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents