[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3534678.3539137acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article
Open access

Multi-Aspect Dense Retrieval

Published: 14 August 2022 Publication History

Abstract

Prior work in Dense Retrieval usually encodes queries and documents using single-vector representations (also called embeddings) and performs retrieval in the embedding space using approximate nearest neighbor search. This paradigm enables efficient semantic retrieval. However, the single-vector representations can be ineffective at capturing different aspects of the queries and documents in relevance matching, especially for some vertical domains. For example, in e-commerce search, these aspects could be category, brand and color. Given a query ''white nike socks", a Dense Retrieval model may mistakenly retrieve some ''white adidas socks" while missing out the intended brand. We propose to explicitly represent multiple aspects using one embedding per aspect. We introduce an aspect prediction task to teach the model to capture aspect information with particular aspect embeddings. We design a lightweight network to fuse the aspect embeddings for representing queries and documents. Our evaluation using an e-commerce dataset shows impressive improvements over strong Dense Retrieval baselines. We also discover that the proposed aspect embeddings can enhance the interpretability of Dense Retrieval models as a byproduct.

References

[1]
Qingyao Ai, Yongfeng Zhang, Keping Bi, Xu Chen, and W Bruce Croft. Learning a hierarchical embedding model for personalized product search. In Proc. of SIGIR, pages 645--654, 2017.
[2]
Alexandr Andoni, Piotr Indyk, and Ilya Razenshteyn. Approximate nearest neighbor search in high dimensions. In Proceedings of the International Congress of Mathematicians: Rio de Janeiro 2018, pages 3287--3318. World Scientific, 2018.
[3]
Jing Bai, Ke Zhou, Guirong Xue, Hongyuan Zha, Gordon Sun, Belle Tseng, Zhaohui Zheng, and Yi Chang. Multi-task learning for learning to rank in web search. In Proc. of CIKM, pages 1549--1552, 2009.
[4]
Oren Barkan, Noam Razin, Itzik Malkiel, Ori Katz, Avi Caciularu, and Noam Koenigstein. Scalable attentive sentence pair modeling via distilled sentence embedding. In Proc. of AAAI, volume 34, pages 3235--3242, 2020.
[5]
Olivier Chapelle, Pannagadatta Shivaswamy, Srinivas Vadrevu, Kilian Weinberger, Ya Zhang, and Belle Tseng. Multi-task learning for boosting with application to web search ranking. In Proc. of KDD, pages 1189--1198, 2010.
[6]
Jiecao Chen, Liu Yang, Karthik Raman, Michael Bendersky, Jung-Jung Yeh, Yun Zhou, Marc Najork, Danyang Cai, and Ehsan Emadzadeh. Dipair: Fast and accurate distillation for trillion-scale text matching and pair modeling. arXiv preprint arXiv:2010.03099, 2020.
[7]
Ronan Collobert and Jason Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proc. of ICML, pages 160--167, 2008.
[8]
Zihang Dai, Guokun Lai, Yiming Yang, and Quoc V Le. Funnel-transformer: Filtering out sequential redundancy for efficient language processing. arXiv preprint arXiv:2006.03236, 2020.
[9]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
[10]
Daniel Gillick, Sayali Kulkarni, Larry Lansing, Alessandro Presta, Jason Baldridge, Eugene Ie, and Diego Garcia-Olano. Learning dense representations for entity retrieval. arXiv preprint arXiv:1909.10506, 2019.
[11]
Ruiqi Guo, Sanjiv Kumar, Krzysztof Choromanski, and David Simcha. Quantization based fast inner product search. In Artificial Intelligence and Statistics, pages 482--490. PMLR, 2016.
[12]
er et al.(2020)Hofst"atter, Zlabinger, and Hanbury]hofstatter2020interpretableSebastian Hofst"atter, Markus Zlabinger, and Allan Hanbury. Interpretable & time-budget-constrained contextualization for re-ranking. arXiv preprint arXiv:2002.01854, 2020.
[13]
Baotian Hu, Zhengdong Lu, Hang Li, and Qingcai Chen. Convolutional neural network architectures for matching natural language sentences. In Advances in neural information processing systems, pages 2042--2050, 2014.
[14]
Jui-Ting Huang, Ashish Sharma, Shuying Sun, Li Xia, David Zhang, Philip Pronin, Janani Padmanabhan, Giuseppe Ottaviano, and Linjun Yang. Embedding-based retrieval in facebook search. In Proc. of KDD, pages 2553--2561, 2020.
[15]
Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. Learning deep structured semantic models for web search using clickthrough data. In Proc. of CIKM, pages 2333--2338, 2013.
[16]
Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. Poly-encoders: Transformer architectures and pre-training strategies for fast and accurate multi-sentence scoring. arXiv preprint arXiv:1905.01969, 2019.
[17]
Ben Ltaifa Ibtihel, Hlaoua Lobna, and Ben Romdhane Lotfi. A deep learning-based ranking approach for microblog retrieval. Procedia Computer Science, 159: 352--362, 2019.
[18]
Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. Adaptive mixtures of local experts. Neural computation, 3 (1): 79--87, 1991.
[19]
z, Min, Lewis, Wu, Edunov, Chen, and Yih]karpukhin2020denseVladimir Karpukhin, Barlas Oug uz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906, 2020.
[20]
Omar Khattab and Matei Zaharia. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proc. of SIGIR, pages 39--48, 2020.
[21]
Jurek Leonhardt, Koustav Rudra, and Avishek Anand. Learnt sparsity for effective and interpretable document ranking. arXiv preprint arXiv:2106.12460, 2021.
[22]
Jimmy Lin, Rodrigo Nogueira, and Andrew Yates. Pretrained transformers for text ranking: Bert and beyond. arXiv preprint arXiv:2010.06467, 2020.
[23]
Yi Luan, Jacob Eisenstein, Kristina Toutanova, and Michael Collins. Sparse, dense, and attentional representations for text retrieval. arXiv preprint arXiv:2005.00181, 2020.
[24]
Bhaskar Mitra, Nick Craswell, et al. An introduction to neural information retrieval. Now Foundations and Trends, 2018.
[25]
Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, and Jimmy Lin. Multi-stage document ranking with bert. arXiv preprint arXiv:1910.14424, 2019.
[26]
Hamid Palangi, Li Deng, Yelong Shen, Jianfeng Gao, Xiaodong He, Jianshu Chen, Xinying Song, and R Ward. Semantic modelling with long-short-term memory for information retrieval. arXiv preprint arXiv:1412.6629, 2014.
[27]
Rohan Ramanath, Hakan Inan, Gungor Polatkan, Bo Hu, Qi Guo, Cagri Ozcaglar, Xianren Wu, Krishnaram Kenthapadi, and Sahin Cem Geyik. Towards deep and representation learning for talent search at linkedin. In Proc. of CIKM, pages 2253--2261, 2018.
[28]
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084, 2019.
[29]
Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulieu, Mike Gatford, et al. Okapi at trec-3. Nist Special Publication Sp, 109: 109, 1995.
[30]
Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. Colbertv2: Effective and efficient retrieval via lightweight late interaction. arXiv preprint arXiv:2112.01488, 2021.
[31]
Jaspreet Singh and Avishek Anand. Exs: Explainable search using local model agnostic interpretability. In Proc. of WSDM, pages 770--773, 2019.
[32]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998--6008, 2017.
[33]
Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. Approximate nearest neighbor negative contrastive learning for dense text retrieval. arXiv preprint arXiv:2007.00808, 2020.
[34]
Qian Xu, Sinno Jialin Pan, Hannah Hong Xue, and Qiang Yang. Multitask learning for protein subcellular location prediction. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 8 (3): 748--759, 2010.
[35]
Shaowei Yao, Jiwei Tan, Xi Chen, Keping Yang, Rong Xiao, Hongbo Deng, and Xiaojun Wan. Learning a product relevance model from click-through data in e-commerce. In Proc of WWW, pages 2890--2899, 2021.
[36]
Hongchun Zhang, Tianyi Wang, Xiaonan Meng, Yi Hu, and Hao Wang. Improving semantic matching via multi-task learning in e-commerce. In eCOM@ SIGIR, 2019.
[37]
Tianzhu Zhang, Bernard Ghanem, Si Liu, and Narendra Ahuja. Robust visual tracking via structured multi-task sparse learning. International journal of computer vision, 101 (2): 367--383, 2013.

Cited By

View all
  • (2024)Exploring ChatGPT for next-generation information retrieval: Opportunities and challengesWeb Intelligence10.3233/WEB-23036322:1(31-44)Online publication date: 26-Mar-2024
  • (2024)Enhancing Multi-field B2B Cloud Solution Matching via Contrastive Pre-trainingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671513(4839-4849)Online publication date: 25-Aug-2024
  • (2024)Multi-Intent Attribute-Aware Text Matching in SearchingProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635813(360-368)Online publication date: 4-Mar-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
August 2022
5033 pages
ISBN:9781450393850
DOI:10.1145/3534678
This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2022

Check for updates

Author Tags

  1. dense retrieval
  2. multi-aspect
  3. multi-task learning

Qualifiers

  • Research-article

Conference

KDD '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1,451
  • Downloads (Last 6 weeks)90
Reflects downloads up to 21 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Exploring ChatGPT for next-generation information retrieval: Opportunities and challengesWeb Intelligence10.3233/WEB-23036322:1(31-44)Online publication date: 26-Mar-2024
  • (2024)Enhancing Multi-field B2B Cloud Solution Matching via Contrastive Pre-trainingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671513(4839-4849)Online publication date: 25-Aug-2024
  • (2024)Multi-Intent Attribute-Aware Text Matching in SearchingProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635813(360-368)Online publication date: 4-Mar-2024
  • (2024)A Multi-Granularity-Aware Aspect Learning Model for Multi-Aspect Dense RetrievalProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635770(674-682)Online publication date: 4-Mar-2024
  • (2024)To Copy, or not to Copy; That is a Critical Issue of the Output Softmax Layer in Neural Sequential RecommendersProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635755(67-76)Online publication date: 4-Mar-2024
  • (2024)Going Beyond Passages: Readability Assessment for Book-Level Long TextsChinese Computational Linguistics10.1007/978-981-97-8367-0_26(434-450)Online publication date: 29-Nov-2024
  • (2024)Reproducibility Analysis and Enhancements for Multi-aspect Dense Retriever with Aspect LearningAdvances in Information Retrieval10.1007/978-3-031-56066-8_17(194-209)Online publication date: 24-Mar-2024
  • (2023)Towards Efficient Coarse-grained Dialogue Response SelectionACM Transactions on Information Systems10.1145/359760942:2(1-32)Online publication date: 27-Sep-2023
  • (2023)Pre-training with Aspect-Content Text Mutual Prediction for Multi-Aspect Dense RetrievalProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615157(4300-4304)Online publication date: 21-Oct-2023
  • (2023)Search Result Diversification Using Query Aspects as BottlenecksProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615050(3040-3051)Online publication date: 21-Oct-2023
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media