More Web Proxy on the site http://driver.im/

short-paper

Reinforcement Learning for User Intent Prediction in Customer Service Bots

Authors:

Forrest Sheng BaoAuthors Info & Claims

SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 1265 - 1268

https://doi.org/10.1145/3331184.3331370

Published: 18 July 2019 Publication History

Abstract

A customer service bot is now a necessary component of an e-commerce platform. As a core module of the customer service bot, user intent prediction can help predict user questions before they ask. A typical solution is to find top candidate questions that a user will be interested in. Such solution ignores the inter-relationship between questions and often aims to maximize the immediate reward such as clicks, which may not be ideal in practice. Hence, we propose to view the problem as a sequential decision making process to better capture the long-term effects of each recommendation in the list. Intuitively, we formulate the problem as a Markov decision process and consider using reinforcement learning for the problem. With this approach, questions presented to users are both relevant and diverse. Experiments on offline real-world dataset and online system demonstrate the effectiveness of our proposed approach.

References

[1]

K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath. 2017. Deep Reinforcement Learning: A Brief Survey. IEEE Signal Processing Magazine(2017).

[2]

Heng-Tze Cheng and et al. 2016. Wide & deep learning for recommender systems. Proceedings of the 1st Workshop on Deep Learning for Recommender Systems(2016).

Digital Library

[3]

Mukund Deshpande and George Karypis. 2004. Item-based top-n recommendation algorithms. TOIS 22, 1 (2004), 143--177.

Digital Library

[4]

Yue Feng, Jun Xu, Yanyan Lan, Jiafeng Guo, Wei Zeng, and Xueqi Cheng. 2018. From Greedy Selection to Exploratory Decision-Making: Diverse Ranking with Policy-Value Networks. In SIGIR (SIGIR'18). 125--134.

Digital Library

[5]

Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. Deep FM: A Factorization-Machine based Neural Network for CTR Prediction. CoRRabs/1703.04247 (2017).

[6]

Yoon Kim. 2014. Convolutional Neural Networks for Sentence Classification. In Proceedings of the 2014 EMNLP. 1746--1751.

[7]

V. R. Konda and J. N. Tsitsiklis. 2003. On Actor-Critic Algorithms. SIAM J. Control Optim. (2003).

Digital Library

[8]

F. Li, M. Qiu, H. Chen, X. Wang, X. Gao, J. Huang, J. Ren, Z. Zhao, W. Zhao, L. Wang, and G. Jin. 2017. Ali Me Assist: An Intelligent Assistant for Creating an Innovative E-commerce Experience. In CIKM '17.

Digital Library

[9]

Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015. Continuous control with deep reinforcement learning. CoRRabs/1509.02971 (2015).

[10]

Tie-Yan Liu et al. 2009. Learning to rank for information retrieval. Foundations and Trends® in Information Retrieval3, 3 (2009), 225--331.

Digital Library

[11]

V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A.Graves, M. A. Riedmiller, A. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis. 2015. Human-level control through deep reinforcement learning. Nature(2015).

[12]

Steffen Rendle. 2010. Factorization Machines. In Proceedings of the 2010 IEEE International Conference on Data Mining (ICDM '10). 995--1000.

Digital Library

[13]

G. A. Rummery and M. Niranjan. 1994.Online Q-Learning Using Connectionist Systems. Technical Report. University of Cambridge.

[14]

Guy Shani, David Heckerman, and Ronen I Brafman. 2005. An MDP-based recommender system. JMLR 6 (2005), 1265--1295.

Digital Library

[15]

R. S. Sutton and A. G. Barto. 1998.Reinforcement Learning - An Introduction. MIT Press.

Digital Library

[16]

Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. 2017. Deep & Cross Network for Ad Click Predictions. In Proc. of the ADKDD'17. 12:1--12:7.

Digital Library

[17]

Zeng Wei, Jun Xu, Yanyan Lan, Jiafeng Guo, and Xueqi Cheng. 2017. Reinforcement Learning to Rank with Markov Decision Process. In SIGIR 17. 945--948.

Digital Library

[18]

R. J. Williams. 1992. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Machine Learning(1992).

Digital Library

[19]

Xiangyu Zhao, Liang Zhang, Zhuoye Ding, Dawei Yin, Yihong Zhao, and Jiliang Tang. 2017. Deep reinforcement learning for list-wise recommendations. arXiv preprint arXiv:1801.00209(2017).

Cited By

Hasani MPurwandari KIbrahim MArifin SSafira WPrabaswara B(2024)Utterance Intent Recognition for Online Retail2024 3rd International Conference on Digital Transformation and Applications (ICDXA)10.1109/ICDXA61007.2024.10470915(199-204)Online publication date: 29-Jan-2024
https://doi.org/10.1109/ICDXA61007.2024.10470915
Rafique WHafid AQadir J(2023)Developing smart city services using intent‐aware recommendation systems: A surveyTransactions on Emerging Telecommunications Technologies10.1002/ett.472834:4Online publication date: 12-Jan-2023
https://doi.org/10.1002/ett.4728
Zhou JChen CLi LZhang ZZheng X(2022)FinBrain 2.0: when finance meets trustworthy AI金融大脑2.0：当金融遇到可信人工智能Frontiers of Information Technology & Electronic Engineering10.1631/FITEE.220003923:12(1747-1764)Online publication date: 30-Sep-2022
https://doi.org/10.1631/FITEE.2200039
Show More Cited By

Index Terms

Reinforcement Learning for User Intent Prediction in Customer Service Bots
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Ranking
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Reinforcement learning

Recommendations

Masked-field Pre-training for User Intent Prediction
CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management

For many applications, predicting the users' intents can help the system provide the solutions or recommendations to the users. It improves the user experience, and brings economic benefits. The main challenge of user intent prediction is that we lack ...
User Intent Prediction in Information-seeking Conversations
CHIIR '19: Proceedings of the 2019 Conference on Human Information Interaction and Retrieval

Conversational assistants are being progressively adopted by the general population. However, they are not capable of handling complicated information-seeking tasks that involve multiple turns of information exchange. Due to the limited communication ...
User intent prediction search engine system based on query analysis and image recognition technologies
Abstract
With the rapid development of the Internet and the World Wide Web, and the increasing amounts and variety of information on the Internet, people can now use search engines to obtain a diverse rich range of information. This paper proposes a user ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2019

1512 pages

ISBN:9781450361729

DOI:10.1145/3331184

General Chairs:
Benjamin Piwowarski
CNRS - Sorbonne Universite, France
,
Max Chevalier
Universite de Toulouse, CNRS, France
,
Eric Gaussier
Universite Grenoble Alpes, CNRS, France
,
Program Chairs:
Yoelle Maarek
Amazon Research, Israel
,
Jian-Yun Nie
University of Montreal, Canada
,
Falk Scholer
RMIT University, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

SIGIR '19

Sponsor:

SIGIR

SIGIR '19: The 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

July 21 - 25, 2019

Paris, France

Acceptance Rates

SIGIR'19 Paper Acceptance Rate 84 of 426 submissions, 20%;

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
577
Total Downloads

Downloads (Last 12 months)41
Downloads (Last 6 weeks)7

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hasani MPurwandari KIbrahim MArifin SSafira WPrabaswara B(2024)Utterance Intent Recognition for Online Retail2024 3rd International Conference on Digital Transformation and Applications (ICDXA)10.1109/ICDXA61007.2024.10470915(199-204)Online publication date: 29-Jan-2024
https://doi.org/10.1109/ICDXA61007.2024.10470915
Rafique WHafid AQadir J(2023)Developing smart city services using intent‐aware recommendation systems: A surveyTransactions on Emerging Telecommunications Technologies10.1002/ett.472834:4Online publication date: 12-Jan-2023
https://doi.org/10.1002/ett.4728
Zhou JChen CLi LZhang ZZheng X(2022)FinBrain 2.0: when finance meets trustworthy AI金融大脑2.0：当金融遇到可信人工智能Frontiers of Information Technology & Electronic Engineering10.1631/FITEE.220003923:12(1747-1764)Online publication date: 30-Sep-2022
https://doi.org/10.1631/FITEE.2200039
Mustar ALamprier SPiwowarski BCrestani FPasi GGaussier E(2022)IRnatorProceedings of the 2022 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3539813.3545152(138-143)Online publication date: 23-Aug-2022
https://dl.acm.org/doi/10.1145/3539813.3545152
Sun HYu GZhang PZhang BWang XWang DAl Hasan MXiong L(2022)Graph Based Long-Term And Short-Term Interest Model for Click-Through Rate PredictionProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557336(1818-1826)Online publication date: 17-Oct-2022
https://dl.acm.org/doi/10.1145/3511808.3557336
Lu FXu Y(2022)Exploring Spatial UI Transition Mechanisms with Head-Worn Augmented RealityProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517723(1-16)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517723
Zhang LShen JZhang JXu JLi ZYao YYu L(2022)Multimodal Marketing Intent Analysis for Effective Targeted AdvertisingIEEE Transactions on Multimedia10.1109/TMM.2021.307326724(1830-1843)Online publication date: 2022
https://doi.org/10.1109/TMM.2021.3073267
Bhattacharyya S(2022)Monetization of customer futures through machine learning and artificial intelligence based persuasive technologiesJournal of Science and Technology Policy Management10.1108/JSTPM-09-2021-013614:4(734-757)Online publication date: 31-May-2022
https://doi.org/10.1108/JSTPM-09-2021-0136
Gupta GKatarya R(2021)A Study of Deep Reinforcement Learning Based Recommender Systems2021 2nd International Conference on Secure Cyber Computing and Communications (ICSCCC)10.1109/ICSCCC51823.2021.9478178(218-220)Online publication date: 21-May-2021
https://doi.org/10.1109/ICSCCC51823.2021.9478178
Véstias MDuarte Rde Sousa JNeto H(2020)Moving Deep Learning to the EdgeAlgorithms10.3390/a1305012513:5(125)Online publication date: 18-May-2020
https://doi.org/10.3390/a13050125
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents