More Web Proxy on the site http://driver.im/

Article

Attentional factorization machines: learning the weight of feature interactions via attention networks

Authors:

Tat-Seng ChuaAuthors Info & Claims

IJCAI'17: Proceedings of the 26th International Joint Conference on Artificial Intelligence

Pages 3119 - 3125

Published: 19 August 2017 Publication History

Abstract

Factorization Machines (FMs) are a supervised learning approach that enhances the linear regression model by incorporating the second-order feature interactions. Despite effectiveness, FM can be hindered by its modelling of all feature interactions with the same weight, as not all feature interactions are equally useful and predictive. For example, the interactions with useless features may even introduce noises and adversely degrade the performance. In this work, we improve FM by discriminating the importance of different feature interactions. We propose a novel model named Attentional Factorization Machine (AFM), which learns the importance of each feature interaction from data via a neural attention network. Extensive experiments on two real-world datasets demonstrate the effectiveness of AFM. Empirically, it is shown on regression task AFM betters FM with a 8.6% relative improvement, and consistently outperforms the state-of-the-art deep learning methods Wide&Deep [Cheng et al. , 2016] and Deep-Cross [Shan et al. , 2016] with a much simpler structure and fewer model parameters. Our implementation of AFM is publicly available at: https://github. com/hexiangnan/attentional_factorization_machine

References

[1]

Linas Baltrunas, Karen Church, Alexandros Karatzoglou, and Nuria Oliver. Frappe: Understanding the usage and perception of mobile app recommendations in-the-wild. CoRR , abs/1505.03014, 2015.

[2]

Immanuel Bayer, Xiangnan He, Bhargav Kanagal, and Steffen Rendle. A generic coordinate descent framework for learning from implicit feedback. In WWW , 2017.

Digital Library

[3]

Mathieu Blondel, Akinori Fujino, Naonori Ueda, and Masakazu Ishihata. Higher-order factorization machines. In NIPS , 2016.

[4]

Tao Chen, Xiangnan He, and Min-Yen Kan. Context-aware image tweet modelling and recommendation. In MM , 2016.

Digital Library

[5]

Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, and Tat-Seng Chua. Attentive collaborative filtering: Multimedia recommendation with feature- and item-level attention. In SIGIR , 2017.

Digital Library

[6]

Long Chen, Hanwang Zhang, Jun Xiao, Liqiang Nie, Jian Shao, and Tat-Seng Chua. SCA-CNN: spatial and channel-wise attention in convolutional networks for image captioning. In CVPR , 2017.

[7]

Chen Cheng, Fen Xia, Tong Zhang, Irwin King, and Michael R Lyu. Gradient boosting factorization machines. In RecSys , 2014.

Digital Library

[8]

Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, et al. Wide & deep learning for recommender systems. In DLRS , 2016.

Digital Library

[9]

F. Maxwell Harper and Joseph A. Konstan. The movielens datasets: History and context. ACM TIIS , 2015.

Digital Library

[10]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR , 2016.

[11]

Xiangnan He, Hanwang Zhang, Min-Yen Kan, and Tat-Seng Chua. Fast matrix factorization for online recommendation with implicit feedback. In SIGIR , 2016.

Digital Library

[12]

Xiangnan He, Ming Gao, Min-Yen Kan, and Dingxian Wang. BiRank: Towards ranking on bipartite graphs. IEEE TKDE , 2017.

Digital Library

[13]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural collaborative filering. In WWW , 2017.

Digital Library

[14]

Yuchin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. Field-aware factorization machines for ctr prediction. In RecSys , 2016.

Digital Library

[15]

Yehuda Koren. Factorization meets the neighborhood: A multifaceted collaborative filtering model. In KDD , 2008.

Digital Library

[16]

Fabio Petroni, Luciano Del Corro, and Rainer Gemulla. Core: Context-aware open relation extraction with factorization machines. In EMNLP , 2015.

[17]

Steffen Rendle, Zeno Gantner, Christoph Freudenthaler, and Lars Schmidt-Thieme. Fast context-aware recommendations with factorization machines. In SIGIR , 2011.

Digital Library

[18]

Steffen Rendle. Factorization machines. In ICDM , 2010.

Digital Library

[19]

Steffen Rendle. Factorization machines with libfm. ACM TIST , 2012.

Digital Library

[20]

Xiangnan He and Tat-Seng Chua. Neural factorization machines for sparse predictive analytics. In SIGIR , 2017.

Digital Library

[21]

Xiangnan He, Min-Yen Kan, Peichu Xie, and Xiao Chen. Comment-based multi-view clustering of web 2.0 items. In WWW , 2014.

Digital Library

[22]

Ying Shan, T Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, and JC Mao. Deep crossing: Web-scale modeling without manually crafted combinatorial features. In KDD , 2016.

Digital Library

[23]

Fumin Shen, Chunhua Shen, Wei Liu, and Heng Tao Shen. Supervised discrete hashing. In CVPR , 2015.

[24]

Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. JMLR , 2014.

Digital Library

[25]

Meng Wang, Xueliang Liu, and Xindong Wu. Visual classification by l1-hypergraph modeling. IEEE TKDE , 2015.

[26]

Meng Wang, Weijie Fu, Shijie Hao, Dacheng Tao, and Xindong Wu. Scalable semi-supervised learning by efficient anchor graph regularization. IEEE TKDE , 2016.

[27]

Xiang Wang, Xiangnan He, Liqiang Nie and Tat-Seng Chua Item Silk Road: Recommending Items from Information Domains to Social Users SIGIR , 2017.

Digital Library

[28]

Meng Wang, Weijie Fu, Shijie Hao, Hengchang Liu, and Xindong Wu. Learning on big graph: Label inference and regularization with anchor hierarchy. IEEE TKDE , 2017.

Digital Library

[29]

Chenyan Xiong, Jimie Callan, and Tie-Yen Liu. Learning to attend and to rank with word-entity duets. In SIGIR , 2017.

[30]

Yang Yang, Zheng-Jun Zha, Yue Gao, Xiaofeng Zhu, and Tat-Seng Chua. Exploiting web images for semantic video indexing via robust sample-specific loss. IEEE TMM , 2014.

[31]

Yang Yang, Zhigang Ma, Yi Yang, Feiping Nie, and Heng Tao Shen. Multitask spectral clustering by exploring intertask correlation. IEEE TCYB , 2015.

[32]

Hanwang Zhang, Xindi Shang, Huanbo Luan, Meng Wang, and Tat-Seng Chua. Learning from collective intelligence: Feature learning using social images and tags. TMM , 2016.

Digital Library

[33]

Hanwang Zhang, Fumin Shen, Wei Liu, Xiangnan He, Huanbo Luan, and Tat-Seng Chua. Discrete collaborative filtering. In SIGIR , 2016.

Digital Library

[34]

Hanwang Zhang, Zawlin Kyaw, Shih-Fu Chang, and Tat-Seng Chua. Visual translation embedding network for visual relation detection. In CVPR , 2017.

[35]

Zhou Zhao, Lijun Zhang, Xiaofei He, and Wilfred Ng. Expert finding for question answering via graph regularized matrix completion. TKDE , 2015.

[36]

Zhou Zhao, Hanqing Lu, Deng Cai, Xiaofei He, and Yueting Zhuang. User Preference Learning for Online Social Recommendation. TKDE , 2016.

Digital Library

Cited By

Duan MLi KZhang WQin JXiao B(2024)Attacking Click-through Rate Predictors via Generating Realistic Fake SamplesACM Transactions on Knowledge Discovery from Data10.1145/364368518:5(1-24)Online publication date: 28-Feb-2024
https://dl.acm.org/doi/10.1145/3643685
Gao CZheng YWang WFeng FHe XLi Y(2024)Causal Inference in Recommender Systems: A Survey and Future DirectionsACM Transactions on Information Systems10.1145/363904842:4(1-32)Online publication date: 2-Jan-2024
https://dl.acm.org/doi/10.1145/3639048
Du KChen JLin JXi YWang HDai XChen BTang RZhang WBaeza-Yates RBonchi F(2024)DisCo: Towards Harmonious Disentanglement and Collaboration between Tabular and Semantic Space for RecommendationProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3672008(666-676)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3672008
Show More Cited By

Attentional factorization machines: learning the weight of feature interactions via attention networks
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches

Recommendations

Multilinear Factorization Machines for Multi-Task Multi-View Learning
WSDM '17: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining

Many real-world problems, such as web image analysis, document categorization and product recommendation, often exhibit dual-heterogeneity: heterogeneous features obtained in multiple views, and multiple tasks might be related to each other through one ...
Gradient boosting factorization machines
RecSys '14: Proceedings of the 8th ACM Conference on Recommender systems

Recommendation techniques have been well developed in the past decades. Most of them build models only based on user item rating matrix. However, in real world, there is plenty of auxiliary information available in recommendation systems. We can utilize ...
Attentional Neural Factorization Machines for Knowledge Tracing
Knowledge Science, Engineering and Management
Abstract
To promote the quality and intelligence of online education systems, knowledge tracing becomes a fundamental and crucial task. It models knowledge state and predicts performance based on student’s learning records. Recently, factorization machine ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

IJCAI'17: Proceedings of the 26th International Joint Conference on Artificial Intelligence

August 2017

5253 pages

ISBN:9780999241103

Editor:
Carles Sierra
IIIA-CSIC

Sponsors

Australian Comp Soc: Australian Computer Society
NSF: National Science Foundation
Griffith University
University of Technology Sydney
AI Journal: AI Journal

Publisher

AAAI Press

Publication History

Published: 19 August 2017

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

59
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Duan MLi KZhang WQin JXiao B(2024)Attacking Click-through Rate Predictors via Generating Realistic Fake SamplesACM Transactions on Knowledge Discovery from Data10.1145/364368518:5(1-24)Online publication date: 28-Feb-2024
https://dl.acm.org/doi/10.1145/3643685
Gao CZheng YWang WFeng FHe XLi Y(2024)Causal Inference in Recommender Systems: A Survey and Future DirectionsACM Transactions on Information Systems10.1145/363904842:4(1-32)Online publication date: 2-Jan-2024
https://dl.acm.org/doi/10.1145/3639048
Du KChen JLin JXi YWang HDai XChen BTang RZhang WBaeza-Yates RBonchi F(2024)DisCo: Towards Harmonious Disentanglement and Collaboration between Tabular and Semantic Space for RecommendationProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3672008(666-676)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3672008
Bai YZhang YLu JChang JZang XNiu YSong YFeng FAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)LabelCraft: Empowering Short Video Recommendations with Automated Label CraftingProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635816(28-37)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635816
Zhang YChen MChen RZhao CYuan MSun Z(2024)Bayesian attention‐based user behaviour modelling for click‐through rate predictionCAAI Transactions on Intelligence Technology10.1049/cit2.123439:5(1320-1330)Online publication date: 1-May-2024
https://dl.acm.org/doi/10.1049/cit2.12343
Xiao FWu YZhang MChen GOoi B(2023)MINT: Detecting Fraudulent Behaviors from Time-Series Relational DataProceedings of the VLDB Endowment10.14778/3611540.361155116:12(3610-3623)Online publication date: 12-Sep-2023
https://dl.acm.org/doi/10.14778/3611540.3611551
Zhai JGong ZWang YSun XYan ZLi FLiu XSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Revisiting Neural Retrieval on AcceleratorsProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599897(5520-5531)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599897
Lu QDu WXu WMa J(2023)KSGANInformation Systems10.1016/j.is.2023.102282119:COnline publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1016/j.is.2023.102282
Chen LShi H(2022)DexDeepFM: Ensemble Diversity Enhanced Extreme Deep Factorization Machine ModelACM Transactions on Knowledge Discovery from Data10.1145/350527216:5(1-17)Online publication date: 9-Mar-2022
https://dl.acm.org/doi/10.1145/3505272
Cheng ZLiu FMei SGuo YZhu LNie L(2022)Feature-Level Attentive ICF for RecommendationACM Transactions on Information Systems10.1145/349047740:4(1-24)Online publication date: 9-Mar-2022
https://dl.acm.org/doi/10.1145/3490477
Show More Cited By

View Options

View options

Figures

Tables

Media

View Table of Conten