More Web Proxy on the site http://driver.im/

research-article

Open access

Multi-Aspect Dense Retrieval

Authors:

Swaraj Khadanga,

Shaleen Kumar Gupta,

Mingyang Zhang,

Michael BenderskyAuthors Info & Claims

KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Pages 3178 - 3186

https://doi.org/10.1145/3534678.3539137

Published: 14 August 2022 Publication History

Abstract

Prior work in Dense Retrieval usually encodes queries and documents using single-vector representations (also called embeddings) and performs retrieval in the embedding space using approximate nearest neighbor search. This paradigm enables efficient semantic retrieval. However, the single-vector representations can be ineffective at capturing different aspects of the queries and documents in relevance matching, especially for some vertical domains. For example, in e-commerce search, these aspects could be category, brand and color. Given a query ''white nike socks", a Dense Retrieval model may mistakenly retrieve some ''white adidas socks" while missing out the intended brand. We propose to explicitly represent multiple aspects using one embedding per aspect. We introduce an aspect prediction task to teach the model to capture aspect information with particular aspect embeddings. We design a lightweight network to fuse the aspect embeddings for representing queries and documents. Our evaluation using an e-commerce dataset shows impressive improvements over strong Dense Retrieval baselines. We also discover that the proposed aspect embeddings can enhance the interpretability of Dense Retrieval models as a byproduct.

References

[1]

Qingyao Ai, Yongfeng Zhang, Keping Bi, Xu Chen, and W Bruce Croft. Learning a hierarchical embedding model for personalized product search. In Proc. of SIGIR, pages 645--654, 2017.

Digital Library

[2]

Alexandr Andoni, Piotr Indyk, and Ilya Razenshteyn. Approximate nearest neighbor search in high dimensions. In Proceedings of the International Congress of Mathematicians: Rio de Janeiro 2018, pages 3287--3318. World Scientific, 2018.

[3]

Jing Bai, Ke Zhou, Guirong Xue, Hongyuan Zha, Gordon Sun, Belle Tseng, Zhaohui Zheng, and Yi Chang. Multi-task learning for learning to rank in web search. In Proc. of CIKM, pages 1549--1552, 2009.

Digital Library

[4]

Oren Barkan, Noam Razin, Itzik Malkiel, Ori Katz, Avi Caciularu, and Noam Koenigstein. Scalable attentive sentence pair modeling via distilled sentence embedding. In Proc. of AAAI, volume 34, pages 3235--3242, 2020.

[5]

Olivier Chapelle, Pannagadatta Shivaswamy, Srinivas Vadrevu, Kilian Weinberger, Ya Zhang, and Belle Tseng. Multi-task learning for boosting with application to web search ranking. In Proc. of KDD, pages 1189--1198, 2010.

Digital Library

[6]

Jiecao Chen, Liu Yang, Karthik Raman, Michael Bendersky, Jung-Jung Yeh, Yun Zhou, Marc Najork, Danyang Cai, and Ehsan Emadzadeh. Dipair: Fast and accurate distillation for trillion-scale text matching and pair modeling. arXiv preprint arXiv:2010.03099, 2020.

[7]

Ronan Collobert and Jason Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proc. of ICML, pages 160--167, 2008.

Digital Library

[8]

Zihang Dai, Guokun Lai, Yiming Yang, and Quoc V Le. Funnel-transformer: Filtering out sequential redundancy for efficient language processing. arXiv preprint arXiv:2006.03236, 2020.

[9]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.

[10]

Daniel Gillick, Sayali Kulkarni, Larry Lansing, Alessandro Presta, Jason Baldridge, Eugene Ie, and Diego Garcia-Olano. Learning dense representations for entity retrieval. arXiv preprint arXiv:1909.10506, 2019.

[11]

Ruiqi Guo, Sanjiv Kumar, Krzysztof Choromanski, and David Simcha. Quantization based fast inner product search. In Artificial Intelligence and Statistics, pages 482--490. PMLR, 2016.

[12]

er et al.(2020)Hofst"atter, Zlabinger, and Hanbury]hofstatter2020interpretableSebastian Hofst"atter, Markus Zlabinger, and Allan Hanbury. Interpretable & time-budget-constrained contextualization for re-ranking. arXiv preprint arXiv:2002.01854, 2020.

[13]

Baotian Hu, Zhengdong Lu, Hang Li, and Qingcai Chen. Convolutional neural network architectures for matching natural language sentences. In Advances in neural information processing systems, pages 2042--2050, 2014.

Digital Library

[14]

Jui-Ting Huang, Ashish Sharma, Shuying Sun, Li Xia, David Zhang, Philip Pronin, Janani Padmanabhan, Giuseppe Ottaviano, and Linjun Yang. Embedding-based retrieval in facebook search. In Proc. of KDD, pages 2553--2561, 2020.

Digital Library

[15]

Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. Learning deep structured semantic models for web search using clickthrough data. In Proc. of CIKM, pages 2333--2338, 2013.

Digital Library

[16]

Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. Poly-encoders: Transformer architectures and pre-training strategies for fast and accurate multi-sentence scoring. arXiv preprint arXiv:1905.01969, 2019.

[17]

Ben Ltaifa Ibtihel, Hlaoua Lobna, and Ben Romdhane Lotfi. A deep learning-based ranking approach for microblog retrieval. Procedia Computer Science, 159: 352--362, 2019.

Digital Library

[18]

Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. Adaptive mixtures of local experts. Neural computation, 3 (1): 79--87, 1991.

[19]

z, Min, Lewis, Wu, Edunov, Chen, and Yih]karpukhin2020denseVladimir Karpukhin, Barlas Oug uz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906, 2020.

[20]

Omar Khattab and Matei Zaharia. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proc. of SIGIR, pages 39--48, 2020.

Digital Library

[21]

Jurek Leonhardt, Koustav Rudra, and Avishek Anand. Learnt sparsity for effective and interpretable document ranking. arXiv preprint arXiv:2106.12460, 2021.

[22]

Jimmy Lin, Rodrigo Nogueira, and Andrew Yates. Pretrained transformers for text ranking: Bert and beyond. arXiv preprint arXiv:2010.06467, 2020.

[23]

Yi Luan, Jacob Eisenstein, Kristina Toutanova, and Michael Collins. Sparse, dense, and attentional representations for text retrieval. arXiv preprint arXiv:2005.00181, 2020.

[24]

Bhaskar Mitra, Nick Craswell, et al. An introduction to neural information retrieval. Now Foundations and Trends, 2018.

[25]

Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, and Jimmy Lin. Multi-stage document ranking with bert. arXiv preprint arXiv:1910.14424, 2019.

[26]

Hamid Palangi, Li Deng, Yelong Shen, Jianfeng Gao, Xiaodong He, Jianshu Chen, Xinying Song, and R Ward. Semantic modelling with long-short-term memory for information retrieval. arXiv preprint arXiv:1412.6629, 2014.

[27]

Rohan Ramanath, Hakan Inan, Gungor Polatkan, Bo Hu, Qi Guo, Cagri Ozcaglar, Xianren Wu, Krishnaram Kenthapadi, and Sahin Cem Geyik. Towards deep and representation learning for talent search at linkedin. In Proc. of CIKM, pages 2253--2261, 2018.

Digital Library

[28]

Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084, 2019.

[29]

Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulieu, Mike Gatford, et al. Okapi at trec-3. Nist Special Publication Sp, 109: 109, 1995.

[30]

Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. Colbertv2: Effective and efficient retrieval via lightweight late interaction. arXiv preprint arXiv:2112.01488, 2021.

[31]

Jaspreet Singh and Avishek Anand. Exs: Explainable search using local model agnostic interpretability. In Proc. of WSDM, pages 770--773, 2019.

Digital Library

[32]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998--6008, 2017.

Digital Library

[33]

Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. Approximate nearest neighbor negative contrastive learning for dense text retrieval. arXiv preprint arXiv:2007.00808, 2020.

[34]

Qian Xu, Sinno Jialin Pan, Hannah Hong Xue, and Qiang Yang. Multitask learning for protein subcellular location prediction. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 8 (3): 748--759, 2010.

[35]

Shaowei Yao, Jiwei Tan, Xi Chen, Keping Yang, Rong Xiao, Hongbo Deng, and Xiaojun Wan. Learning a product relevance model from click-through data in e-commerce. In Proc of WWW, pages 2890--2899, 2021.

Digital Library

[36]

Hongchun Zhang, Tianyi Wang, Xiaonan Meng, Yi Hu, and Hao Wang. Improving semantic matching via multi-task learning in e-commerce. In eCOM@ SIGIR, 2019.

[37]

Tianzhu Zhang, Bernard Ghanem, Si Liu, and Narendra Ahuja. Robust visual tracking via structured multi-task sparse learning. International journal of computer vision, 101 (2): 367--383, 2013.

Digital Library

Cited By

Wu YChen PDing ZYan A(2025)Zero-Shot Dense Retrieval Based on Query ExpansionArtificial Intelligence Security and Privacy10.1007/978-981-96-1148-5_12(143-155)Online publication date: 18-Jan-2025
https://doi.org/10.1007/978-981-96-1148-5_12
Huang YHuang J(2024)Exploring ChatGPT for next-generation information retrieval: Opportunities and challengesWeb Intelligence10.3233/WEB-23036322:1(31-44)Online publication date: 26-Mar-2024
https://doi.org/10.3233/WEB-230363
Chen HDou ZHao XTao YSong SSheng ZBaeza-Yates RBonchi F(2024)Enhancing Multi-field B2B Cloud Solution Matching via Contrastive Pre-trainingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671513(4839-4849)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671513
Show More Cited By

Index Terms

Multi-Aspect Dense Retrieval
1. Information systems
  1. Information retrieval

Recommendations

A Multi-Granularity-Aware Aspect Learning Model for Multi-Aspect Dense Retrieval
WSDM '24: Proceedings of the 17th ACM International Conference on Web Search and Data Mining

Dense retrieval methods have been mostly focused on unstructured text and less attention has been drawn to structured data with various aspects, e.g., products with aspects such as category and brand. Recent work has proposed two approaches to ...
Pre-training with Aspect-Content Text Mutual Prediction for Multi-Aspect Dense Retrieval
CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

Grounded on pre-trained language models (PLMs), dense retrieval has been studied extensively on plain text. In contrast, there has been little research on retrieving data with multiple aspects using dense models. In the scenarios such as product search, ...
Reproducibility Analysis and Enhancements for Multi-aspect Dense Retriever with Aspect Learning
Advances in Information Retrieval
Abstract
Multi-aspect dense retrieval aims to incorporate aspect information (e.g., brand and category) into dual encoders to facilitate relevance matching. As an early and representative multi-aspect dense retriever, MADRAL learns several extra aspect ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 2022

5033 pages

ISBN:9781450393850

DOI:10.1145/3534678

General Chairs:
Aidong Zhang
University of Virginia
,
Huzefa Rangwala
Amazon/George Mason University

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2022

Check for updates

Author Tags

Qualifiers

Research-article

Conference

KDD '22

Sponsor:

KDD '22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 14 - 18, 2022

Washington DC, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
3,996
Total Downloads

Downloads (Last 12 months)1,300
Downloads (Last 6 weeks)93

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wu YChen PDing ZYan A(2025)Zero-Shot Dense Retrieval Based on Query ExpansionArtificial Intelligence Security and Privacy10.1007/978-981-96-1148-5_12(143-155)Online publication date: 18-Jan-2025
https://doi.org/10.1007/978-981-96-1148-5_12
Huang YHuang J(2024)Exploring ChatGPT for next-generation information retrieval: Opportunities and challengesWeb Intelligence10.3233/WEB-23036322:1(31-44)Online publication date: 26-Mar-2024
https://doi.org/10.3233/WEB-230363
Chen HDou ZHao XTao YSong SSheng ZBaeza-Yates RBonchi F(2024)Enhancing Multi-field B2B Cloud Solution Matching via Contrastive Pre-trainingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671513(4839-4849)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671513
Li MChen XXiang JZhang QMa CDai CChang JLiu ZZhang GAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)Multi-Intent Attribute-Aware Text Matching in SearchingProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635813(360-368)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635813
Sun XBi KGuo JYang SZhang QLiu ZZhang GCheng XAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)A Multi-Granularity-Aware Aspect Learning Model for Multi-Aspect Dense RetrievalProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635770(674-682)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635770
Chang HAgarwal NMcCallum AAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)To Copy, or not to Copy; That is a Critical Issue of the Output Softmax Layer in Neural Sequential RecommendersProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635755(67-76)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635755
Li WSun RZhang TWu Y(2024)Going Beyond Passages: Readability Assessment for Book-Level Long TextsChinese Computational Linguistics10.1007/978-981-97-8367-0_26(434-450)Online publication date: 29-Nov-2024
https://doi.org/10.1007/978-981-97-8367-0_26
Bi KSun XGuo JCheng X(2024)Reproducibility Analysis and Enhancements for Multi-aspect Dense Retriever with Aspect LearningAdvances in Information Retrieval10.1007/978-3-031-56066-8_17(194-209)Online publication date: 24-Mar-2024
https://dl.acm.org/doi/10.1007/978-3-031-56066-8_17
Lan TMao XWei WGao XHuang H(2023)Towards Efficient Coarse-grained Dialogue Response SelectionACM Transactions on Information Systems10.1145/359760942:2(1-32)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3597609
Sun XBi KGuo JMa XFan YShan HZhang QLiu ZFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Pre-training with Aspect-Content Text Mutual Prediction for Multi-Aspect Dense RetrievalProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615157(4300-4304)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615157
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten