基于DeepFM的深度兴趣因子分解机网络

计算机科学 ›› 2021, Vol. 48 ›› Issue (1): 226-232.doi: 10.11896/jsjkx.191200098

基于DeepFM的深度兴趣因子分解机网络

王瑞平, 贾真, 刘畅, 陈泽威, 李天瑞

西南交通大学信息科学与技术学院成都 611756

收稿日期:2019-12-16 修回日期:2020-05-17 出版日期:2021-01-15 发布日期:2021-01-15
通讯作者: 李天瑞(trli@swjtu.edu.cn)
作者简介:abcwrp@163.com
基金资助:
国家重点研发计划 (2017YFB1401400)

Deep Interest Factorization Machine Network Based on DeepFM

WANG Rui-ping, JIA Zhen, LIU Chang, CHEN Ze-wei, LI Tian-rui

School of Information Science and Technology,Southwest Jiaotong University,Chengdu 611756,China

Received:2019-12-16 Revised:2020-05-17 Online:2021-01-15 Published:2021-01-15
About author:WANG Rui-ping,born in 1995,postgraduate.Her main research interests include recommendation algorithm and natural language processing.
LI Tian-rui,born in 1969,Ph.D,professor,Ph.D supervisor,is a distinguished member of China Computer Federation.His main research interests include big data intelligence,rough sets and granular computing.
Supported by:
National Key R&D Program of China(2017YFB1401400).

摘要/Abstract

摘要： 推荐系统能够根据用户的喜好从海量信息中筛选出其可能感兴趣的信息并进行排序展示。随着深度学习在多个研究领域取得了良好的效果,其也开始应用于推荐系统。目前基于深度学习的推荐排序算法常采用Embedding&MLP模式,只能获得高阶的特征交互。为了解决该问题,DeepFM在上述模式中加入了因子分解机(Factorization Machine,FM),能够实现端到端的低阶与高阶特征交互学习,但其缺乏用户兴趣多样性的表示。鉴于此,通过将多头注意力机制引入DeepFM,提出了深度兴趣因子分解机网络(Deep Interest Factorization Machine Network,DIFMN)。DIFMN能够根据待推荐的不同物品自适应地学习用户表示,展示用户兴趣的多样性。此外,该模型根据用户历史行为的种类添加了喜好表征,从而不仅能够应用于只记录用户爱好的历史行为的任务,还可以处理同时记录用户喜欢与不喜欢的历史行为的任务。采用tensorflow-gpu进行算法的实现,在Amazon(Electronics)和movieLen-20m两个公开数据集上进行对比测试,实验表明所提算法相比DeepFM分别有17.70%和35.24%的RelaImpr提升,验证了其可行性与有效性。

关键词: CTR预测, DeepFM, 多头注意力机制, 深度学习, 推荐算法, 用户兴趣建模

Abstract: The recommendation system can sort out and display the information that may be of interest from the mass of information according to users' preferences.As deep learning has achieved good results in multiple research fields,it has also begun to be applied to recommendation systems.However,the current recommendation ranking algorithms based on deep learning often use Embedding & MLP mode and can only obtain high-level feature interactions.In order to solve the problem that only high-order feature interaction can be obtained,DeepFM adds FM to the above mode,which can learn the low-order and high-order feature interaction end-to-end.But the DeepFM cannot express the diversity of user interests.In view of this,this paper proposes a Deep Interest Factorization Machine Network(DIFMN) by introducing the multi-head attention mechanism into DeepFM.DIFMN can adaptively learn the user representation according to the different items to be recommended,showing the diversity of user intere-sts.In addition,the model adds preference representations according to the type of user's historical behaviors,so that it can be applied not only to tasks that record only historical behaviors that the user likes,but also to tasks that record both historical beha-viors that the user likes and dislikes.This paper uses tensorflow-gpu to implement the algorithm,and performs comparative tests on two public datasets of Amazon(Electronics) and movieLen-20 m.Experiment results show that RelaImprimproves by 17.70% and 35.24% respectively compared to DeepFM,which validates the feasibility and effectiveness of the proposed method.

Key words: CTR prediction, Deep learning, DeepFM, Multi-head attention mechanism, Recommendation algorithm, User interest modeling

中图分类号:

TP391

王瑞平, 贾真, 刘畅, 陈泽威, 李天瑞. 基于DeepFM的深度兴趣因子分解机网络[J]. 计算机科学, 2021, 48(1): 226-232. https://doi.org/10.11896/jsjkx.191200098

WANG Rui-ping, JIA Zhen, LIU Chang, CHEN Ze-wei, LI Tian-rui. Deep Interest Factorization Machine Network Based on DeepFM[J]. Computer Science, 2021, 48(1): 226-232. https://doi.org/10.11896/jsjkx.191200098

参考文献

[1] MARZ N,WARREN J.Big Data:Principles and best practices of scalable realtime data systems[M].Manning Publications,2015.
[2] RICCI F,ROKACH L,SHAPIRA B.Introduction to Recom-mender Systems Handbook[M]//Recommender Systems Handbook.Boston:Springer,2011:1-35.
[3] YU C J,ZHUANG Y,WEI S C,et al.Field-aware factorization machines for CTR prediction[C]//Proceedings of the 10th ACM Conference on Recommender Systems.ACM,2016:43-50.
[4] COVINGTON P,ADAMS J,SARGIN E.Deep neural networks for youtube recommendations[C]//Proceedings of the 10th ACM Conference on Recommender Systems.ACM,2016:191-198.
[5] ZHOU G R,ZHU X Q,SONG C R,et al.Deep interest network for click-through rate prediction[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.ACM,2018:1059-1068.
[6] CHEN Q,ZHAO H,LI W,et al.Behavior Sequence Transformer for E-commerce Recommendation in Alibaba[J].arXiv:1905.06874,2019.
[7] ZHOU G R,MOU N,FAN Y,et al.Deep interest evolution network for click-through rate prediction[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:5941-5948.
[8] CHENG H T,KOC L,HARMSEN J,et al.Wide & deep learning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems.ACM,2016:7-10.
[9] GUO H F,TANG R M,YE Y M,et al.DeepFM:a factorization-machine based neural network for CTR prediction[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence.AAAI Press,2017:1725-1731.
[10] RENDLE S.Factorization machines[C]//Proceedings of 2010 IEEE International Conference on Data Mining.IEEE,2010:995-1000.
[11] VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Advances in Neural Information Processing Systems.2017:5998-6008.
[12] MCAULEY J,TARGETT C,SHI Q,et al.Image-based recommendations on styles and substitutes[C]//Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval.ACM,2015:43-52.
[13] HE R,MCAULEY J.Ups and downs:Modeling the visual evolution of fashion trends with one-class collaborative filtering[C]//Proceedings of the 25th International Conference on World Wide Web.2016:507-517.
[14] HARPER F M,KONSTAN J A.The movielens datasets:His-tory and context[J].ACM Transactions on Interactive Intelligent Systems,2015,5(4):19.
[15] QU Y,CAI H,REN K,et al.Product-based neural networks for user response prediction[C]//Proceedings of 2016 IEEE 16th International Conference on Data Mining.2016:1149-1154.
[16] ZHU H,JIN J,TAN C,et al.Optimized cost per click in taobao display advertising[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.ACM:2191-2200.
[17] YAN L,LI W J,XUE G R,et al.Coupled group lasso for web-scale ctr prediction in display advertising[C]//Proceedings of International Conference on Machine Learning.2014:802-810.
[18] RICHARDSON M,DOMINOWSKA E,RAGNO R.Predicting clicks:estimating the click-through rate for new ads[C]//Proceedings of the 16th International Conference on World Wide Web.ACM,2007:521-530.
[19] FENG YF,LV F Y,SHEN W C,et al.Deep session interest network for click-through rate prediction[C]//Proceedings of 28th International Joint Conference on Artificial Intelligence.2019.
[20] ZHU H,LI X,ZhANG P,et al.Learning tree-based deep model for recommender systems[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2018:1079-1088.

相关文章 15

[1]	饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2]	汤凌韬, 王迪, 张鲁飞, 刘盛云. 基于安全多方计算和差分隐私的联邦学习方案 Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy 计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[3]	徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺. 时序知识图谱表示学习 Temporal Knowledge Graph Representation Learning 计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[4]	王剑, 彭雨琦, 赵宇斐, 杨健. 基于深度学习的社交网络舆情信息抽取方法综述 Survey of Social Network Public Opinion Information Extraction Based on Deep Learning 计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[5]	郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6]	姜梦函, 李邵梅, 郑洪浩, 张建朋. 基于改进位置编码的谣言检测模型 Rumor Detection Model Based on Improved Position Embedding 计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[7]	孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[8]	侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木. 中文预训练模型研究进展 Advances in Chinese Pre-training Models 计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[9]	周慧, 施皓晨, 屠要峰, 黄圣君. 基于主动采样的深度鲁棒神经网络学习 Robust Deep Neural Network Learning Based on Active Sampling 计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[10]	苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫. 小样本雷达辐射源识别的深度学习方法综述 Survey of Deep Learning for Radar Emitter Identification Based on Small Sample 计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[11]	胡艳羽, 赵龙, 董祥军. 一种用于癌症分类的两阶段深度特征选择提取算法 Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification 计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[12]	程成, 降爱莲. 基于多路径特征提取的实时语义分割方法 Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction 计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[13]	刘伟业, 鲁慧民, 李玉鹏, 马宁. 指静脉识别技术研究综述 Survey on Finger Vein Recognition Research 计算机科学, 2022, 49(6A): 1-11. https://doi.org/10.11896/jsjkx.210400056
[14]	孙福权, 崔志清, 邹彭, 张琨. 基于多尺度特征的脑肿瘤分割算法 Brain Tumor Segmentation Algorithm Based on Multi-scale Features 计算机科学, 2022, 49(6A): 12-16. https://doi.org/10.11896/jsjkx.210700217
[15]	康雁, 徐玉龙, 寇勇奇, 谢思宇, 杨学昆, 李浩. 基于Transformer和LSTM的药物相互作用预测 Drug-Drug Interaction Prediction Based on Transformer and LSTM 计算机科学, 2022, 49(6A): 17-21. https://doi.org/10.11896/jsjkx.210400150

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed