[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/1935826.1935922acmconferencesArticle/Chapter ViewAbstractPublication PageswsdmConference Proceedingsconference-collections
poster

On composition of a federated web search result page: using online users to provide pairwise preference for heterogeneous verticals

Published: 09 February 2011 Publication History

Abstract

Modern web search engines are federated --- a user query is sent to the numerous specialized search engines called verticals like web (text documents), News, Image, Video, etc. and the results returned by these engines are then aggregated and composed into a search result page (SERP) and presented to the user. For a specific query, multiple verticals could be relevant, which makes the placement of these vertical results within blocks of textual web results challenging: how do we represent, assess, and compare the relevance of these heterogeneous entities?
In this paper we present a machine-learning framework for SERP composition in the presence of multiple relevant verticals. First, instead of using the traditional label generation method of human judgment guidelines and trained judges, we use a randomized online auditioning system that allows us to evaluate triples of the form query, web block, vertical>. We use a pairwise click preference to evaluate whether the web block or the vertical block had a better users' engagement. Next, we use a hinged feature vector that contains features from the web block to create a common reference frame and augment it with features representing the specific vertical judged by the user. A gradient boosted decision tree is then learned from the training data. For the final composition of the SERP, we place a vertical result at a slot if the score is higher than a computed threshold. The thresholds are algorithmically determined to guarantee specific coverage for verticals at each slot.
We use correlation of clicks as our offline metric and show that click-preference target has a better correlation than human judgments based models. Furthermore, on online tests for News and Image verticals we show higher user engagement for both head and tail queries.

References

[1]
J. Arguello, J. Callan, F. Diaz, and J. F. Crespo. Source of evidence for vertical selection. In Proc. of Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, 2009.
[2]
J. Arguello, F. Diaz, and J. F. Paiement. Vertical selection in presence of unlabeled verticals. In Proc. of Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, 2010.
[3]
C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender. Learning to rank using gradient descent. pages 89--96, 2005.
[4]
O. Chapelle and Ya Zhang. A dynamic bayesian network model for web search ranking. In Proc. of Intl. Conf. on World Wide Web, 2009.
[5]
F. Diaz. Integration of news content into web results. In Proc. of Intl. Conf. on Web Search and Data Mining, 2009.
[6]
F. Diaz and J. Arguello. Adaptation of offline selection predictions in presense of user feedback. In Proc. of Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, 2009.
[7]
P. Donmez, K. M. Svore, and C. J. C. Burges. On the local optimality of LambdaRank. In Proc. of Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, 2009.
[8]
J. H. Friedman. Greedy function approximation: A graidient boosting machine. Annals of Statistics, 29:1189--1232, 2001.
[9]
J. H. Friedman. Stochastic gradient boosting. Computational Statistics and Data Analysis, 38:367--378, 2001.
[10]
T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning. Sringer-Verlag, New York, NY, 2001.
[11]
S. Ji, T. Moon, G. Dupret, C. Liao, and Z. Zheng. User behavior driven ranking without editorial judgments. In Proc. of Intl. Conf. on Information and Knowledge Management, 2010.
[12]
T. Joachims. Optimizing search engines using clickthrough data. In Proc. 8th Ann. Intl. ACM SIGKDD Conf. on Knowledge Discovery and Data Mining, pages 133--142, 2002.
[13]
T. Joachims, L. Granka, B. Pan, H. Hembrooke, F. Radlinkski, and G. Gay. Evaluating the accuracy of implicit feedback from clicks and query reformations in web search. ACM Trans. on Information Retrieval, 2007.
[14]
J. Li, S. Huffman, and A. Tokuda. Good abandonment in mobile and PC internet search. In PRoc. of Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, 2009.
[15]
P. Li, C. J. C. Burges, and Q. Wu. Mcrank: Learning to rank using multiple classification and gradient boosting. In Proc. 21st Proc. of Advances in Neural Information Processing Systems, 2007.
[16]
V. Murdock and M. Lalmas. Workshop on aggregated search, 2008. http://www.sigir.org/forum/2008D/sigirwksp/2008d_sigirforum_murdock.pdf.
[17]
F. Radlinski and T. Joachims. Minimally invasive randomization for collecting unbiased preferences from clickthrough logs. In Proc. of AAAI, 2005.
[18]
S. Robertson and S. Walker. Some simple approximations to the 2-poisson model for probabilistic weighted retrieval. In Proc. of Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, 1994.
[19]
M. Shokouhi and L. Si. Federated information retrieval. In D. W. Oard and Editors F. Sebastiani, editors, Foundations and Trends in Information Retrieval. 2010.
[20]
M. Shokouhi, J. Zobel, S. Tahaghoghi, and F. Scholer. Using query logs to establish vocabularies in distributed information retrieval. Information Processing and Management, 43(1):169--180, 2007.
[21]
L. Si and J. Callan. Modeling search engine effectiveness for federated search. In Proc. of Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, 2005.
[22]
Z. Zheng, H. Zha, K. Chen, and G. Sun. A regression framework for learning ranking functions using relative judgments. In Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, pages 287--294, 2007.
[23]
Z. Zheng, H. Zha, T. Zhang, O. Chapelle, K. Chen, and G. Sun. A general boosting method and its application to learning ranking functions for web search. In Proc. 21st Proc. of Advances in Neural Information Processing Systems, 2007.

Cited By

View all
  • (2024)Utilizing Ant Colony Optimization for Result Merging in Federated SearchEngineering, Technology & Applied Science Research10.48084/etasr.730214:4(14832-14839)Online publication date: 2-Aug-2024
  • (2024)An overview of aggregation methods for social networks analysisKnowledge and Information Systems10.1007/s10115-024-02296-zOnline publication date: 12-Dec-2024
  • (2023)Quantifying customer interactions on ML optimized page layoutsProceedings of the International Conference on Advances in Social Networks Analysis and Mining10.1145/3625007.3627519(502-508)Online publication date: 6-Nov-2023
  • Show More Cited By

Index Terms

  1. On composition of a federated web search result page: using online users to provide pairwise preference for heterogeneous verticals

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    WSDM '11: Proceedings of the fourth ACM international conference on Web search and data mining
    February 2011
    870 pages
    ISBN:9781450304931
    DOI:10.1145/1935826
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 09 February 2011

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. federated web search
    2. heterogeneous verticals
    3. machine learning
    4. pairwise preference from clicks
    5. randomized flights

    Qualifiers

    • Poster

    Conference

    Acceptance Rates

    WSDM '11 Paper Acceptance Rate 83 of 372 submissions, 22%;
    Overall Acceptance Rate 498 of 2,863 submissions, 17%

    Upcoming Conference

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)3
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 21 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Utilizing Ant Colony Optimization for Result Merging in Federated SearchEngineering, Technology & Applied Science Research10.48084/etasr.730214:4(14832-14839)Online publication date: 2-Aug-2024
    • (2024)An overview of aggregation methods for social networks analysisKnowledge and Information Systems10.1007/s10115-024-02296-zOnline publication date: 12-Dec-2024
    • (2023)Quantifying customer interactions on ML optimized page layoutsProceedings of the International Conference on Advances in Social Networks Analysis and Mining10.1145/3625007.3627519(502-508)Online publication date: 6-Nov-2023
    • (2023)Federated search techniques: an overview of the trends and state of the artKnowledge and Information Systems10.1007/s10115-023-01922-665:12(5065-5095)Online publication date: 10-Jul-2023
    • (2019)The Effects of Working Memory, Perceptual Speed, and Inhibition in Aggregated SearchACM Transactions on Information Systems10.1145/332212837:3(1-34)Online publication date: 16-May-2019
    • (2019)Aggregating E-commerce Search Results from Heterogeneous Sources via Hierarchical Reinforcement LearningThe World Wide Web Conference10.1145/3308558.3313455(1771-1781)Online publication date: 13-May-2019
    • (2019)Deep Structure Learning for Rumor Detection on Twitter2019 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN.2019.8852468(1-8)Online publication date: Jul-2019
    • (2019)Text Attention and Focal Negative Loss for Scene Text Detection2019 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN.2019.8851959(1-8)Online publication date: Jul-2019
    • (2019)CARL: Aggregated Search with Context-Aware Module Embedding Learning2019 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN.2019.8851716(1-8)Online publication date: Jul-2019
    • (2018)Optimizing Whole-Page Presentation for Web SearchACM Transactions on the Web10.1145/320446112:3(1-25)Online publication date: 17-Jul-2018
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media