More Web Proxy on the site http://driver.im/

research-article

Hierarchical Temporal Convolutional Networks for Dynamic Recommender Systems

Authors:

Pong Eksombatchai,

Chuck Rosenburg,

Jure LeskovecAuthors Info & Claims

WWW '19: The World Wide Web Conference

Pages 2236 - 2246

https://doi.org/10.1145/3308558.3313747

Published: 13 May 2019 Publication History

Abstract

Recommender systems that can learn from cross-session data to dynamically predict the next item a user will choose are crucial for online platforms. However, existing approaches often use out-of-the-box sequence models which are limited by speed and memory consumption, are often infeasible for production environments, and usually do not incorporate cross-session information, which is crucial for effective recommendations. Here we propose Hierarchical Temporal Convolutional Networks (HierTCN), a hierarchical deep learning architecture that makes dynamic recommendations based on users' sequential multi-session interactions with items. HierTCN is designed for web-scale systems with billions of items and hundreds of millions of users. It consists of two levels of models: The high-level model uses Recurrent Neural Networks (RNN) to aggregate users' evolving long-term interests across different sessions, while the low-level model is implemented with Temporal Convolutional Networks (TCN), utilizing both the long-term interests and the short-term interactions within sessions to predict the next interaction. We conduct extensive experiments on a public XING dataset and a large-scale Pinterest dataset that contains 6 million users with 1.6 billion interactions. We show that HierTCN is 2.5x faster than RNN-based models and uses 90% less data memory compared to TCN-based models. We further develop an effective data caching scheme and a queue-based mini-batch generator, enabling our model to be trained within 24 hours on a single GPU. Our model consistently outperforms state-of-the-art dynamic recommendation methods, with up to 18% improvement in recall and 10% in mean reciprocal rank.

References

[1]

Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, 2016. Tensorflow: a system for large-scale machine learning. In OSDI, Vol. 16.

Digital Library

[2]

Shaojie Bai, J Zico Kolter, and Vladlen Koltun. 2018. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271(2018).

[3]

RecSys Challenge. 2016. RecSys Challenge 2016. In Proceedings of the Recommender Systems Challenge. ACM.

[4]

Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555(2014).

[5]

Tim Cooijmans, Nicolas Ballas, Ce´sar Laurent, Çaglar Gülçehre, and Aaron Courville. 2016. Recurrent batch normalization. arXiv preprint arXiv:1603.09025(2016).

[6]

Hanjun Dai, Yichen Wang, Rakshit Trivedi, and Le Song. 2016. Deep coevolutionary network: Embedding user and item features for recommendation. arXiv preprint arXiv:1609.03675(2016).

[7]

Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for ctr prediction. IJCAI (2017).

Digital Library

[8]

Michael U Gutmann and Aapo Hyvärinen. 2012. Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. Journal of Machine Learning Research 13, Feb (2012), 307-361.

Digital Library

[9]

Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems.

Digital Library

[10]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition.

[11]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee.

Digital Library

[12]

Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939(2015).

[13]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735-1780.

Digital Library

[14]

Cheng-Kang Hsieh, Longqi Yang, Yin Cui, Tsung-Yi Lin, Serge Belongie, and Deborah Estrin. 2017. Collaborative metric learning. In Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee.

Digital Library

[15]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167(2015).

[16]

Dietmar Jannach and Malte Ludewig. 2017. When recurrent neural networks meet the neighborhood for session-based recommendation. In Proceedings of the Eleventh ACM Conference on Recommender Systems. ACM.

Digital Library

[17]

Christoph Lameter. 2006. Local and remote memory: Memory in a Linux/NUMA system. In Linux Symposium.

[18]

Yann LeCun, Yoshua Bengio, 1995. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks 3361, 10(1995), 1995.

Digital Library

[19]

Tomáš Mikolov, Martin Karafiát, Lukáš Burget, Jan Cernock?, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In Eleventh Annual Conference of the International Speech Communication Association.

[20]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems.

Digital Library

[21]

Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. 2016. Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499(2016).

[22]

Aaron van den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. 2016. Pixel recurrent neural networks. International Conference on Machine Learning (2016).

Digital Library

[23]

Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, and Paolo Cremonesi. 2017. Personalizing session-based recommendations with hierarchical recurrent neural networks. In Proceedings of the Eleventh ACM Conference on Recommender Systems. ACM.

Digital Library

[24]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence. AUAI Press, 452-461.

Digital Library

[25]

Tim Salimans and Diederik P Kingma. 2016. Weight normalization: A simple reparameterization to accelerate training of deep neural networks. In Advances in Neural Information Processing Systems.

Digital Library

[26]

Suvash Sedhain, Aditya Krishna Menon, Scott Sanner, and Lexing Xie. 2015. AutoRec: Autoencoders meet collaborative filtering. In Proceedings of the 24th International Conference on World Wide Web. ACM.

Digital Library

[27]

Stanislau Semeniuta, Aliaksei Severyn, and Erhardt Barth. 2016. Recurrent dropout without memory loss. arXiv preprint arXiv:1603.05118(2016).

[28]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556(2014).

[29]

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research 15, 1 (2014), 1929-1958.

Digital Library

[30]

Johan AK Suykens and Joos Vandewalle. 1999. Least squares support vector machine classifiers. Neural processing letters 9, 3 (1999), 293-300.

Digital Library

[31]

Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In 11th ACM International Conference on Web Search and Data Mining. ACM.

Digital Library

[32]

Yichen Wang, Nan Du, Rakshit Trivedi, and Le Song. 2016. Coevolutionary latent feature processes for continuous-time user-item interactions. In Advances in Neural Information Processing Systems. 4547-4555.

Digital Library

[33]

Ronald J Williams and Jing Peng. 1990. An efficient gradient-based algorithm for on-line training of recurrent network trajectories. Neural computation 2, 4 (1990), 490-501.

Digital Library

[34]

Chao-Yuan Wu, Amr Ahmed, Alex Beutel, Alexander J Smola, and How Jing. 2017. Recurrent recommender networks. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. ACM.

Digital Library

[35]

Sai Wu, Weichao Ren, Chengchao Yu, Gang Chen, Dongxiang Zhang, and Jingbo Zhu. 2016. Personal recommendation using deep recurrent neural networks in NetEase. In 32nd International Conference on Data Engineering. IEEE.

[36]

Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph Convolutional Neural Networks for Web-Scale Recommender Systems. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2018).

Digital Library

[37]

Jiaxuan You, Rex Ying, Xiang Ren, William Hamilton, and Jure Leskovec. 2018. Graphrnn: Generating realistic graphs with deep auto-regressive models. In International Conference on Machine Learning. 5694-5703.

[38]

Fajie Yuan, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M Jose, and Xiangnan He. 2018. A Simple but Hard-to-Beat Baseline for Session-based Recommendations. arXiv preprint arXiv:1808.05163(2018).

Cited By

Bhirangi RWang CPattabiraman VMajidi CGupta AHellebrekers TPinto LSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Hierarchical state space models for continuous sequence-to-sequence modelingProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692223(3795-3816)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3692223
Yang XEsquivel J(2024)Time-Aware LSTM Neural Networks for Dynamic Personalized Recommendation on Business IntelligenceTsinghua Science and Technology10.26599/TST.2023.901002529:1(185-196)Online publication date: Feb-2024
https://doi.org/10.26599/TST.2023.9010025
Tahir SHafeez YHumayun MAhmad FKhan MShaheen M(2024)Harnessing hybrid deep learning approach for personalized retrieval in e-learningPLOS ONE10.1371/journal.pone.030860719:11(e0308607)Online publication date: 13-Nov-2024
https://doi.org/10.1371/journal.pone.0308607
Show More Cited By

Recommendations

Temporal diversity in recommender systems
SIGIR '10: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval

Collaborative Filtering (CF) algorithms, used to build web-based recommender systems, are often evaluated in terms of how accurately they predict user ratings. However, current evaluation techniques disregard the fact that users continue to rate items ...
Acquiring User Information Needs for Recommender Systems
WI-IAT '13: Proceedings of the 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT) - Volume 03

Most recommender systems attempt to use collaborative filtering, content-based filtering or hybrid approach to recommend items to new users. Collaborative filtering recommends items to new users based on their similar neighbours, and content-based ...
Hybrid Recommender System Based on Multi-Hierarchical Ontologies
WebMedia '18: Proceedings of the 24th Brazilian Symposium on Multimedia and the Web

Recommender Systems (RSs) are usually based in User Profiles (UP) to identify items of interest to a user, among the items of a usually vast collection. Traditional RSs are mostly based on ratings of items made by users and do not attempt to estimate ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

WWW '19: The World Wide Web Conference

May 2019

3620 pages

ISBN:9781450366748

DOI:10.1145/3308558

Editors:
Ling Liu
Georgia Tech, USA
,
Ryen White
Microsoft Research, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

IW3C2: International World Wide Web Conference Committee

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '19

WWW '19: The Web Conference

May 13 - 17, 2019

CA, San Francisco, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

74
Total Citations
View Citations
1,475
Total Downloads

Downloads (Last 12 months)93
Downloads (Last 6 weeks)13

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bhirangi RWang CPattabiraman VMajidi CGupta AHellebrekers TPinto LSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Hierarchical state space models for continuous sequence-to-sequence modelingProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692223(3795-3816)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3692223
Yang XEsquivel J(2024)Time-Aware LSTM Neural Networks for Dynamic Personalized Recommendation on Business IntelligenceTsinghua Science and Technology10.26599/TST.2023.901002529:1(185-196)Online publication date: Feb-2024
https://doi.org/10.26599/TST.2023.9010025
Tahir SHafeez YHumayun MAhmad FKhan MShaheen M(2024)Harnessing hybrid deep learning approach for personalized retrieval in e-learningPLOS ONE10.1371/journal.pone.030860719:11(e0308607)Online publication date: 13-Nov-2024
https://doi.org/10.1371/journal.pone.0308607
Zhang ZWang XChen HLi HZhu W(2024)Disentangled Dynamic Graph Attention Network for Out-of-distribution Sequential RecommendationACM Transactions on Information Systems10.1145/3701988Online publication date: 29-Oct-2024
https://doi.org/10.1145/3701988
Tran VSalha-Galvan GSguerra BHennequin R(2024)Transformers Meet ACT-R: Repeat-Aware and Sequential Listening Session RecommendationProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688139(486-496)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688139
Chen XXiong YZhang SZhang JZhang YZhou SWu XZhang MLiu TWang WSerra ESpezzano F(2024)DTFormer: A Transformer-Based Method for Discrete-Time Dynamic Graph Representation LearningProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679568(301-311)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679568
Wang JKaratzoglou AArapakis IXin XGe XJose JSerra ESpezzano F(2024)Sparks of Surprise: Multi-objective Recommendations with Hierarchical Decision Transformers for Diversity, Novelty, and SerendipityProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679533(2358-2368)Online publication date: 21-Oct-2024
https://doi.org/10.1145/3627673.3679533
Anwar TUma VSrivastava G(2024)CDRec-CAS: Cross-Domain Recommendation Using Context-Aware SequencesIEEE Transactions on Computational Social Systems10.1109/TCSS.2022.323378111:4(4934-4943)Online publication date: Aug-2024
https://doi.org/10.1109/TCSS.2022.3233781
Zhu XWu YWang LSu HLi Z(2024)Continuous-Time Dynamic Interaction Network Learning Based on Evolutionary ExpectationIEEE Transactions on Cognitive and Developmental Systems10.1109/TCDS.2023.330528516:3(840-849)Online publication date: Jun-2024
https://doi.org/10.1109/TCDS.2023.3305285
Muangrux DAthipatcharawat PChuangsuwanich EKantavat P(2024)Article Feed Recommendations Using Position-Aware Deep Cross Network2024 Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering (ECTI DAMT & NCON)10.1109/ECTIDAMTNCON60518.2024.10479991(44-49)Online publication date: 31-Jan-2024
https://doi.org/10.1109/ECTIDAMTNCON60518.2024.10479991
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents