BERD+: A Generic Sequential Recommendation Framework by Eliminating Unreliable Data with Item- and Attribute-level Signals
Authors: 
Yatong Sun, Xiaochun Yang, Zhu Sun, Bin WangAuthors Info & Claims
Article No.: 41, Pages 1 - 33
Abstract
Most sequential recommendation systems (SRSs) predict the next item as the target for users given its preceding items as input, assuming the target is definitely related to its input. However, users may unintentionally click items that are inconsistent with their preference due to external factors, causing unreliable instances whose target mismatches the input. We, for the first time, verify SRSs can be misguided by such unreliable instances and design a generic SRS framework By Eliminating unReliable Data (BERD+), which can be flexibly plugged into existing SRSs. Specifically, BRED+ is guided with observations on the training process of instances: Unreliable instances generally have high training loss; high-loss instances are not necessarily unreliable but uncertain ones caused by blurry sequential patterns; and item attributes help rectify instance loss and uncertainty, but may also introduce disturbance. Accordingly, BERD+ models both the loss and uncertainty of each instance via a Gaussian distribution, whereby a heterogeneous uncertainty-aware graph convolution network is designed to learn accurate embeddings for different entities while reducing the disturbance caused by uncertain attribute values. Thereafter, an explicit preference extractor rectifies instance loss and uncertainty and reduces the disturbance caused by less-focused attribute types. Finally, instances with high loss and low uncertainty are eliminated as unreliable data. Extensive experiments verify the efficacy of BERD+.
References
[1]
Sambaran Bandyopadhyay, Lokesh N., Saley Vishal Vivek, and M. Narasimha Murty. 2020. Outlier resistant unsupervised deep architectures for attributed network embedding. In WSDM. 25–33.
[2]
Yixin Cao, Xiang Wang, Xiangnan He, Zikun Hu, and Tat-Seng Chua. 2019. Unifying knowledge graph learning and recommendation: Towards a better understanding of user preferences. In WWW. 151–161.
[3]
Huiyuan Chen, Yusan Lin, Menghai Pan, Lan Wang, Chin-Chia Michael Yeh, Xiaoting Li, Yan Zheng, Fei Wang, and Hao Yang. 2022. Denoising self-attentive sequential recommendation. In RecSys. 92–101.
[4]
Wanyu Chen, Pengjie Ren, Fei Cai, Fei Sun, and Maarten De Rijke. 2021. Multi-interest diversification for end-to-end sequential recommendation. ACM Trans. Inf. Syst. 40, 1 (2021), 1–30.
[5]
Xu Chen, Hongteng Xu, Yongfeng Zhang, Jiaxi Tang, Yixin Cao, Zheng Qin, and Hongyuan Zha. 2018. Sequential recommendation with user memory networks. In WSDM. 108–116.
[6]
Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In EMNLP. 1724–1734.
[7]
Dan Cosley, Shyong K Lam, Istvan Albert, Joseph A. Konstan, and John Riedl. 2003. Is seeing believing? How recommender system interfaces affect users’ opinions. In SIGCHI. 585–592.
[8]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. CoRR abs/1810.04805 (2018).
[9]
Ziwei Fan, Zhiwei Liu, Shen Wang, Lei Zheng, and Philip S. Yu. 2021. Modeling sequences as distributions with uncertainty for sequential recommendation. In CIKM. 3019–3023.
[10]
Ziwei Fan, Zhiwei Liu, Yu Wang, Alice Wang, Zahra Nazari, Lei Zheng, Hao Peng, and Philip S. Yu. 2022. Sequential recommendation via stochastic self-attention. In WWW. 2036–2047.
[11]
Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In AISTATS. 249–256.
[12]
Jacob Goldberger and Ehud Ben-Reuven. 2017. Training deep neural-networks using a noise adaptation layer. In ICLR.
[13]
Bo Han, Jiangchao Yao, Gang Niu, Mingyuan Zhou, Ivor Tsang, Ya Zhang, and Masashi Sugiyama. 2018. Masking: A new perspective of noisy supervision. In NeurIPS. 5841–5851.
[14]
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. 2018. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In NeurIPS. 8536–8546.
[15]
F. Maxwell Harper and Joseph A. Konstan. 2015. The MovieLens datasets: History and context. Trans. Interact. Intell. Syst. 5, 4 (2015), 1–19.
[16]
Zhankui He, Handong Zhao, Zhe Lin, Zhaowen Wang, Ajinkya Kale, and Julian J. McAuley. 2021. Locker: Locally constrained self-attentive sequential recommendation. In CIKM. 3088–3092.
[17]
Dan Hendrycks, Mantas Mazeika, Duncan Wilson, and Kevin Gimpel. 2018. Using trusted data to train deep networks on labels corrupted by severe noise. In NeurIPS. 10477–10486.
[18]
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based recommendations with recurrent neural networks. In ICLR.
[19]
Jin Huang, Wayne Xin Zhao, Hongjian Dou, Ji-Rong Wen, and Edward Y. Chang. 2018. Improving sequential recommendation with knowledge-enhanced memory networks. In SIGIR. 505–514.
[20]
Simon Jenni and Paolo Favaro. 2018. Deep bilevel learning. In ECCV. 618–633.
[21]
Ishan Jindal, Matthew Nokleby, and Xuewen Chen. 2016. Learning deep networks from noisy labels with dropout regularization. In ICDM. 967–972.
[22]
Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In ICDM. 197–206.
[23]
Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR.
[24]
Diederik P. Kingma and Max Welling. 2014. Auto-encoding variational Bayes. In ICLR.
[25]
Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In ICLR.
[26]
Walid Krichene and Steffen Rendle. 2020. On sampled metrics for item recommendation. In KDD. 1748–1757.
[27]
Chao Li, Zhiyuan Liu, Mengmeng Wu, Yuchi Xu, Huan Zhao, Pipei Huang, Guoliang Kang, Qiwei Chen, Wei Li, and Dik Lun Lee. 2019. Multi-interest network with dynamic routing for recommendation at Tmall. In CIKM. 2615–2623.
[28]
Dongsheng Li, Chao Chen, Zhilin Gong, Tun Lu, and Ning Gu. 2019. Collaborative filtering with noisy ratings. In SDM. 747–755.
[29]
Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017. Neural attentive session-based recommendation. In CIKM. 747–755.
[30]
Defu Lian, Qi Liu, and Enhong Chen. 2020. Personalized ranking with importance sampling. In WWW. 1093–1103.
[31]
Yueming Lyu and Ivor W. Tsang. 2019. Curriculum loss: Robust learning and generalization against label corruption. In ICLR.
[32]
Chen Ma, Liheng Ma, Yingxue Zhang, Jianing Sun, Xue Liu, and Mark Coates. 2020. Memory augmented graph neural networks for sequential recommendation. In AAAI. 5045–5052.
[33]
Jianxin Ma, Chang Zhou, Hongxia Yang, Peng Cui, Xin Wang, and Wenwu Zhu. 2020. Disentangled self-supervision in sequential recommenders. In KDD. 483–491.
[34]
Xiaoxiao Ma, Jia Wu, Shan Xue, Jian Yang, Chuan Zhou, Quan Z. Sheng, Hui Xiong, and Leman Akoglu. 2021. A comprehensive survey on graph anomaly detection with deep learning. IEEE Trans. Knowl. Data Eng. (2021), 1–25.
[35]
Julian McAuley and Jure Leskovec. 2013. Hidden factors and hidden topics: Understanding rating dimensions with review text. In RecSys. 165–172.
[36]
Bhaskar Mehta, Thomas Hofmann, and Peter Fankhauser. 2007. Lies and propaganda: Detecting spam users in collaborative filtering. In IUI. 14–21.
[37]
Vinod Nair and Geoffrey E. Hinton. 2010. Rectified linear units improve restricted Boltzmann machines. In ICML. 807–814.
[38]
Michael P. O’Mahony, Neil J. Hurley, and Guénolé C. M. Silvestre. 2006. Detecting noise in recommender system databases. In IUI. 109–115.
[39]
Wenjie Pei, Jie Yang, Zhu Sun, Jie Zhang, Alessandro Bozzon, and David M. J. Tax. 2017. Interacting attention-gated recurrent networks for recommendation. In CIKM. 1459–1468.
[40]
Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. 2018. Learning to reweight examples for robust deep learning. In ICML. 4334–4343.
[41]
Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized Markov chains for next-basket recommendation. In WWW. 811–820.
[42]
Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. 2022. Learning from noisy labels with deep neural networks: A survey. IEEE Trans. Neural Netw. Learn. Syst. (2022), 1–19.
[43]
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In CIKM. 1441–1450.
[44]
Peijie Sun, Le Wu, and Meng Wang. 2018. Attentive recurrent social recommendation. In SIGIR. 185–194.
[45]
Rui Sun, Xuezhi Cao, Yan Zhao, Junchen Wan, Kun Zhou, Fuzheng Zhang, Zhongyuan Wang, and Kai Zheng. 2020. Multi-modal knowledge graphs for recommender systems. In CIKM. 1405–1414.
[46]
Yizhou Sun and Jiawei Han. 2013. Mining heterogeneous information networks: A structural analysis approach. ACM SIGKDD Explor. Newslett. 14, 2 (2013), 20–28.
[47]
Yatong Sun, Bin Wang, Zhu Sun, and Xiaochun Yang. 2021. Does every data instance matter? Enhancing sequential recommendation by eliminating unreliable data. In IJCAI. 1579–1585.
[48]
Jiaxi Tang and Ke Wang. 2018. Personalized Top-N sequential recommendation via convolutional sequence embedding. In WSDM. 565–573.
[49]
Trinh Xuan Tuan and Minh Phuong Tu. 2017. 3D convolutional networks for session-based recommendation with content features. In RecSys. 138–146.
[50]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS. 5998–6008.
[51]
Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2018. RippleNet: Propagating user preferences on the knowledge graph for recommender systems. In CIKM. 417–426.
[52]
Hongwei Wang, Fuzheng Zhang, Xing Xie, and Minyi Guo. 2018. DKN: Deep knowledge-aware network for news recommendation. In WWW. 1835–1844.
[53]
Hongwei Wang, Fuzheng Zhang, Mengdi Zhang, Jure Leskovec, Miao Zhao, Wenjie Li, and Zhongyuan Wang. 2019. Knowledge graph convolutional networks for recommender systems with label smoothness regularization. CoRR abs/1905.04413 (2019).
[54]
Hongwei Wang, Miao Zhao, Xing Xie, Wenjie Li, and Minyi Guo. 2019. Knowledge graph convolutional networks for recommender systems. In WWW. 3307–3313.
[55]
Shoujin Wang, Liang Hu, Yan Wang, Quan Z. Sheng, and Longbing Cao. 2019. Modeling multi-purpose sessions for next-item recommendations via mixture-channel purpose routing networks. In IJCAI. 3771–3777.
[56]
Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019. KGAT: Knowledge graph attention network for recommendation. In KDD. 950–958.
[57]
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S. Yu. 2019. Heterogeneous graph attention network. In WWW. 2022–2032.
[58]
Yisen Wang, Weiyang Liu, Xingjun Ma, James Bailey, Hongyuan Zha, Le Song, and Shu-Tao Xia. 2018. Iterative learning with open-set noisy labels. In CVPR. 8688–8696.
[59]
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In AAAI. 1112–1119.
[60]
Hongxin Wei, Lei Feng, Xiangyu Chen, and Bo An. 2020. Combating noisy labels by agreement: A joint training method with co-regularization. In CVPR. 13723–13732.
[61]
Tianxin Wei, Fuli Feng, Jiawei Chen, Ziwei Wu, Jinfeng Yi, and Xiangnan He. 2021. Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system. In KDD. 1791–1800.
[62]
Shiwen Wu, Yuanxing Zhang, Chengliang Gao, Kaigui Bian, and Bin Cui. 2020. GARG: Anonymous recommendation of point-of-interest in mobile networks by graph convolution network. Data Sci. Eng. 5, 2 (2020).
[63]
Xin Xia, Hongzhi Yin, Junliang Yu, Qinyong Wang, Lizhen Cui, and Xiangliang Zhang. 2020. Self-supervised hypergraph convolutional networks for session-based recommendation. In AAAI. 4503–4511.
[64]
Tong Xiao, Tian Xia, Yi Yang, Chang Huang, and Xiaogang Wang. 2015. Learning from massive noisy labeled data for image classification. In CVPR. 2691–2699.
[65]
Chengfeng Xu, Pengpeng Zhao, Yanchi Liu, Victor S. Sheng, Jiajie Xu, Fuzhen Zhuang, Junhua Fang, and Xiaofang Zhou. 2019. Graph contextualized self-attention network for session-based recommendation. In IJCAI. 3940–3946.
[66]
Jiangchao Yao, Jiajie Wang, Ivor W. Tsang, Ya Zhang, Jun Sun, Chengqi Zhang, and Rui Zhang. 2018. Deep learning from noisy image labels with quality embedding. IEEE Trans. Image Process. 28, 4 (2018), 1909–1922.
[67]
Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor Tsang, and Masashi Sugiyama. 2019. How does disagreement help generalization against label corruption? In ICML. 7164–7173.
[68]
Fajie Yuan, Xiangnan He, Haochuan Jiang, Guibing Guo, Jian Xiong, Zhezhao Xu, and Yilin Xiong. 2020. Future data helps training: Modeling future contexts for session-based recommendation. In WWW. 303–313.
[69]
Jing Zhang, Bowen Hao, Bo Chen, Cuiping Li, Hong Chen, and Jimeng Sun. 2019. Hierarchical reinforcement learning for course recommendation in MOOCs. In AAAI. 435–442.
[70]
Kun Zhou, Hui Yu, Wayne Xin Zhao, and Ji-Rong Wen. 2022. Filter-enhanced MLP is all you need for sequential recommendation. In WWW. 2388–2399.
Index Terms
- BERD+: A Generic Sequential Recommendation Framework by Eliminating Unreliable Data with Item- and Attribute-level Signals
Recommendations
A Collaborative Recommender Combining Item Rating Similarity and Item Attribute Similarity
ISBIM '08: Proceedings of the 2008 International Seminar on Business and Information Management - Volume 02Collaborative filtering (CF) is the most popular recommendation technique nowadays. Traditional CF approaches compute a similarity value between the target user and each other user by computing the relativity of their rating style, which is the set of ...
ICAI-SR: Item Categorical Attribute Integrated Sequential Recommendation
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information RetrievalSequential recommendation (SR) has attracted much research attention in the past few years. Most existing attribute integrated SR models do not directly model the complex relations between items and categorical attributes, as well do not exploit the ...
Modeling and predicting user preferences with multiple item attributes for sequential recommendations
AbstractSequential recommendations have become a focus of attention across the deep learning community owing to their fitness to the actual application scenario. Although recently we have witnessed a surge of work on sequential recommender ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 November 2023
Online AM: 10 August 2023
Accepted: 11 July 2023
Revised: 15 June 2023
Received: 10 January 2023
Published in TOIS Volume 42, Issue 2
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- Ten Thousand Talent Program
- Science and technology projects in Liaoning Province
- 111 Project
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 683Total Downloads
- Downloads (Last 12 months)342
- Downloads (Last 6 weeks)36
Reflects downloads up to 11 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in