[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content

Advertisement

Springer Nature Link
Log in

Click-through rate prediction based on feature interaction and behavioral sequence

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Click-through rate prediction is one of the hot topics in the recommendation and advertising systems field. The existing click-through rate prediction models can be classified into feature interactions and behavior sequences. Feature interaction models form new feature combinations by fusing different features. The behavior sequence models capture the user’s interests by considering the historical behavior and using an attention mechanism to model the relationship between the target item and the behavior sequence. However, the existing click-through rate prediction techniques either ignore both aspects or only consider one, limiting prediction performance. In order to solve the above problems, we propose a click-through prediction model (CFIBS) that combines feature interaction and behavioral sequence in this paper. Firstly, the Global-Local Gate Module and Post-LN Informer are proposed to extract the user’s interests from user behavior sequences to improve training efficiency. In addition, we introduce auxiliary losses to supervise the extraction of user interest features. Secondly, in the interest update layer, we introduce an attention mechanism based gated recurrent unit to enhance the relationship between interest representation and the target item. Finally, for non-temporal features, we propose a Multi-Cross Layer to increase the nonlinear ability of the model. Experiments show that our model can effectively improve the click-through rate prediction accuracy of advertisements. The codes will be available at https://github.com/jihuiqin2/sequence_ctr.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availibility

The datasets used during the current study are available from the corresponding author on reasonable request.

References

  1. Ren K, Zhang W, Rong Y et al (2016) User response learning for directly optimizing campaign performance in display advertising. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp 679–688

  2. Chatterjee S, Panmand M (2022) Explaining and predicting click-baitiness and click-bait virality. Ind Manag Data Syst 122(11):2485–2507

    Article  Google Scholar 

  3. Huang T, She Q, Wang Z et al (2020) Gatenet: gating-enhanced deep network for click-through rate prediction. Preprint at arXiv:2007.03519

  4. Hou M, Wu L, Chen E et al (2019) Explainable fashion recommendation: A semantic attribute region guided approach. Preprint at arXiv:1905.12862

  5. Wang X, Wang L, Wang S et al (2022) Recommending-and-grabbing: a crowdsourcing-based order allocation pattern for on-demand food delivery. IEEE Trans Intell Transp Syst 24(1):838–853

    Article  Google Scholar 

  6. Sun Z, Tang P (2021) Automatic communication error detection using speech recognition and linguistic analysis for proactive control of loss of separation. Transp Res Rec 2675(5):1–12

    Article  Google Scholar 

  7. Huijuan Z, Ning Y, Ruchuan W (2023) Improved cross-corpus speech emotion recognition using deep local domain adaptation. Chin J Electron 32(3):1–7

    Article  Google Scholar 

  8. Ali A, Chowdhury S, Afify M et al (2021) Connecting arabs: bridging the gap in dialectal speech recognition. Commun ACM 64(4):124–129

    Article  Google Scholar 

  9. Chang YW, Hsieh CJ, Chang KW et al (2010) Training and testing low-degree polynomial data mappings via linear svm. J Mach Learn Res 11(4–12)

  10. Rendle S (2012) Factorization machines with libfm. ACM Trans Intell Syst Technol (TIST) 3(3):1–22

    Article  Google Scholar 

  11. Cheng HT, Koc L, Harmsen J et al (2016) Wide & deep learning for recommender systems. In: Proceedings of the 1st workshop on deep learning for recommender systems, pp 7–10

  12. Qu Y, Cai H, Ren K et al (2016) Product-based neural networks for user response prediction. In: 2016 IEEE 16th international conference on data mining (ICDM), IEEE, pp 1149–1154

  13. Zhou G, Zhu X, Song C et al (2018) Deep interest network for click-through rate prediction. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1059–1068

  14. Zhou G, Mou N, Fan Y et al (2019) Deep interest evolution network for click-through rate prediction. In: Proceedings of the AAAI conference on artificial intelligence, pp 5941–5948

  15. Liang X, Liu Z, Ouyang C (2018) A multi-sentiment classifier based on gru and attention mechanism. In: 2018 IEEE 9th International Conference on Software Engineering and Service Science (ICSESS), IEEE, pp 527–530

  16. Young T, Hazarika D, Poria S et al (2018) Recent trends in deep learning based natural language processing. IEEE Computat Intell Mag 13(3):55–75

    Article  Google Scholar 

  17. Sun Y, Pan J, Zhang A et al (2021) Fm2: field-matrixed factorization machines for recommender systems. Proc Web Conf 2021:2828–2837

    Google Scholar 

  18. Blondel M, Fujino A, Ueda N et al (2016) Higher-order factorization machines. Advances in Neural Information Processing Systems 29

  19. Zhang W, Du T, Wang J (2016) Deep learning over multi-field categorical data: –a case study on user response prediction. In: Advances in Information Retrieval: 38th European Conference on IR Research, ECIR 2016, Padua, Italy, March 20–23, 2016. Proceedings 38, Springer, pp 45–57

  20. He X, Chua TS (2017) Neural factorization machines for sparse predictive analytics. In: Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval, pp 355–364

  21. Yan Y, Li L (2023) xdeepint: a hybrid architecture for modeling the vector-wise and bit-wise feature interactions. Preprint at arXiv:2301.01089

  22. Qu Y, Fang B, Zhang W et al (2018) Product-based neural networks for user response prediction over multi-field categorical data. ACM Trans Inform Syst (TOIS) 37(1):1–35

    Google Scholar 

  23. Zou D, Sheng M, Yu H et al (2020) Factorized weight interaction neural networks for sparse feature prediction. Neural Comput Appl 32(8):9567–9579

    Article  Google Scholar 

  24. Zou D, Wang Z, Zhang L et al (2021) Deep field relation neural network for click-through rate prediction. Inf Sci 577(36):128–139

    Article  MathSciNet  Google Scholar 

  25. Huang T, Bi L, Wang N et al (2021) Autofm: an efficient factorization machine model via probabilistic auto-encoders. Neural Comput Appl 33(15):9451–9466

    Article  Google Scholar 

  26. Song W, Shi C, Xiao Z et al (2019) Autoint: Automatic feature interaction learning via self-attentive neural networks. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp 1161–1170

  27. Lian J, Zhou X, Zhang F et al (2018) xdeepfm: Combining explicit and implicit feature interactions for recommender systems. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1754–1763

  28. Guo H, Tang R, Ye Y et al (2017) Deepfm: a factorization-machine based neural network for ctr prediction. Preprint atarXiv:1703.04247

  29. Shan Y, Hoens TR, Jiao J et al (2016) Deep crossing: Web-scale modeling without manually crafted combinatorial features. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 255–262

  30. Wang R, Fu B, Fu G et al (2017) Deep & cross network for ad click predictions. In: Proceedings of the ADKDD’17. p 1–7

  31. Chen Q, Pei C, Lv S et al (2021) End-to-end user behavior retrieval in click-through rateprediction model. Preprint at arXiv:2108.04468

  32. Chen Q, Zhao H, Li W et al (2019) Behavior sequence transformer for e-commerce recommendation in alibaba. In: Proceedings of the 1st international workshop on deep learning practice for high-dimensional sparse data, pp 1–4

  33. Rendle S, Freudenthaler C, Schmidt-Thieme L (2010) Factorizing personalized markov chains for next-basket recommendation. In: Proceedings of the 19th international conference on World wide web, pp 811–820

  34. Feng Y, Lv F, Shen W et al (2019) Deep session interest network for click-through rate prediction. Preprint at arXiv:1905.06482

  35. Pi Q, Bian W, Zhou G et al (2019) Practice on long sequential user behavior modeling for click-through rate prediction. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 2671–2679

  36. Xiao Z, Yang L, Jiang W et al (2020) Deep multi-interest network for click-through rate prediction. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp 2265–2268

  37. Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. Adv Neural Inf Process Syst 30(9):5998–6008

    Google Scholar 

  38. Xiong C, Merity S, Socher R (2016) Dynamic memory networks for visual and textual question answering. In: International conference on machine learning, PMLR, pp 2397–2406

  39. Zhu F, Wang H, Zhang Y (2023) Gru deep residual network for time series classification. In: 2023 IEEE 6th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), IEEE, pp 1289–1293

  40. Zhou Y, Chen S, Xiao D (2022) Study on natural gas price forecasting based on prophet-gru nonlinear combination. In: 2022 7th International Conference on Computational Intelligence and Applications (ICCIA), IEEE, pp 118–122

  41. Mallick R, Yebda T, Benois-Pineau J et al (2021) A gru neural network with attention mechanism for detection of risk situations on multimodal lifelog data. In: 2021 International Conference on Content-Based Multimedia Indexing (CBMI), IEEE, pp 1–6

  42. Xiao Y, He W, Zhu Y et al (2022) A click-through rate model of e-commerce based on user interest and temporal behavior. Expert Syst Appl 207(7):117–896

    Google Scholar 

  43. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  44. Zhang X, Li J, Hua Z (2022) Mffe: multi-scale feature fusion enhanced net for image dehazing. Signal Process Image Commun 105(5):116–719

    Google Scholar 

  45. Xiong R, Yang Y, He D et al (2020) On layer normalization in the transformer architecture. In: International Conference on Machine Learning, PMLR, pp 524–533

  46. McAuley J, Targett C, Shi Q et al (2015) Image-based recommendations on styles and substitutes. In: Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pp 43–52

  47. Chen T, He T, Benesty M et al (2015) Xgboost: extreme gradient boosting. R package version 04-2 1(4):1–4

  48. Richardson M, Dominowska E, Ragno R (2007) Predicting clicks: estimating the click-through rate for new ads. In: Proceedings of the 16th international conference on World Wide Web, pp 521–530

  49. Huang T, Zhang Z, Zhang J (2019) Fibinet: combining feature importance and bilinear feature interaction for click-through rate prediction. In: Proceedings of the 13th ACM Conference on Recommender Systems, pp 169–177

Download references

Acknowledgements

The research received funding from the Key Research and Promotion Projects of Henan Province under Grant Agreement No (222102210034,222102210178, 222102210229 and 232102210031), and the Key Research Projects of Henan Higher Education Institutions under Grant Agreement No 22A520020.

Author information

Authors and Affiliations

  1. College of Software, Henan University, Kaifeng, 475004, China

    Yingqi Wang, Huiqin Ji, Junyang Yu & Rui Zhai

  2. Henan Provincial Engineering Research Center of Intelligent Data Processing, Henan University, Kaifeng, 475004, China

    Yingqi Wang, Huiqin Ji, Junyang Yu & Rui Zhai

  3. School of Computer and Information Engineering, Henan University, Kaifeng, 475004, China

    Hongyu Han

  4. Henan Key Laboratory of Big Data Analysis and Processing, Henan University, Kaifeng, 475004, China

    Hongyu Han

Authors
  1. Yingqi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huiqin Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junyang Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hongyu Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rui Zhai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongyu Han.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Ji, H., Yu, J. et al. Click-through rate prediction based on feature interaction and behavioral sequence. Int. J. Mach. Learn. & Cyber. 15, 2899–2913 (2024). https://doi.org/10.1007/s13042-023-02072-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-02072-5

Keywords