More Web Proxy on the site http://driver.im/

research-article

Driving the Herd: Search Engines as Content Influencers

Authors:

Moshe Tennenholtz,

Fiana RaiberAuthors Info & Claims

CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management

Pages 586 - 595

https://doi.org/10.1145/3459637.3482334

Published: 30 October 2021 Publication History

Abstract

In competitive search settings such as the Web, many documents' authors (publishers) opt to have their documents highly ranked for some queries. To this end, they modify the documents --- specifically, their content --- in response to induced rankings. Thus, the search engine affects the content in the corpus via its ranking decisions. We present a first study of the ability of search engines to drive pre-defined, targeted, content effects in the corpus using simple techniques. The first is based on the herding phenomenon --- a celebrated result from the economics literature --- and the second is based on biasing the relevance ranking function. The types of content effects we study are either topical or touch on specific document properties --- length and inclusion of query terms. Analysis of ranking competitions we organized between incentivized publishers shows that the types of content effects we target can indeed be attained by applying our suggested techniques. These findings have important implications with regard to the role of search engines in shaping the corpus.

Supplementary Material

MP4 File (herding_video_final.mp4)

Video presentation.

Download
143.35 MB

References

[1]

AIR 2005--2009. AIRWeb - International Workshop on Adversarial Information Retrieval on the Web.

[2]

Banerjee. 1992. A simple model of herd behavior. The Quarterly Journal of Economics 107 (1992), 797--817.

[3]

Ran Ben-Basat, Moshe Tennenholtz, and Oren Kurland. 2017. A Game Theoretic Analysis of the Adversarial Retrieval Setting. J. Artif. Intell. Res. 60 (2017), 1127--1164.

Digital Library

[4]

Omer Ben-Porat and Moshe Tennenholtz. 2018. A Game-Theoretic Approach to Recommendation Systems with Strategic Content Providers. In Proceedings of NeurIPS. 1118--1128.

Digital Library

[5]

Asia J. Biega, Krishna P. Gummadi, and Gerhard Weikum. 2018. Equity of Attention: Amortizing Individual Fairness in Rankings. In Proceedings of SIGIR. 405--414.

Digital Library

[6]

S. Bikhchandani, D. Hirshleifer, and I. Welch. 1992. A theory of fads, fashion, custom and cultural change as information cascade. The Journal of Political Economy 100 (1992), 992--1026.

[7]

Nadya Bliss, Elizabeth Bradley, Joshua Garland, Filippo Menczer, Scott W. Ruston, Kate Starbird, and Chris Wiggins. 2020. An Agenda for Disinformation Research. CoRR abs/2012.08572 (2020).

[8]

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. arXiv:2005.14165

[9]

Carlos Castillo. 2018. Fairness and Transparency in Ranking. SIGIR Forum 52, 2 (2018), 64--71.

Digital Library

[10]

Carlos Castillo and Brian D. Davison. 2010. Adversarial Web Search. Foundations and Trends in Information Retrieval 4, 5 (2010), 377--486.

Digital Library

[11]

Gordon V. Cormack, Mark D. Smucker, and Charles L. A. Clarke. 2011. Efficient and effective spam filtering and re-ranking for large web datasets. Informaltiom Retrieval Journal 14, 5 (2011), 441--465.

Digital Library

[12]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. CoRR abs/1810.04805 (2018).

[13]

Hui Fang and ChengXiang Zhai. 2005. An exploration of axiomatic approaches to information retrieval. In Proceedings of SIGIR. 480--487.

Digital Library

[14]

Jamie Guillory and Jeffrey T. Hancock. 2012. The Effect of Linkedin on Deception in Resumes. Cyberpsychology Behav. Soc. Netw. 15, 3 (2012), 135--140.

[15]

Zoltán Gyöngyi and Hector Garcia-Molina. 2005. Web Spam Taxonomy. In Proceedings of AIRWeb 2005. 39--47.

[16]

Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. 2005. Accurately interpreting clickthrough data as implicit feedback. In Proceed- ings of SIGIR. 154--161.

Digital Library

[17]

Adam D. I. Kramer, Jamie Elizabeth Guillory, and Jeffrey T. Hancock. 2014. Exper- imental evidence of massive-scale emotional contagion through social networks. In Proceedings of the National Academy of Sciences of the United States of America 111 24 (2014), 8788--90.

[18]

Victor Lavrenko and W. Bruce Croft. 2003. Relevance Models in Information Retrieval. In Language modeling for information retrieval. Kluwer academic publishers, Chapter 2, 11--56.

[19]

Tie-Yan Liu. 2011. Learning to Rank for Information Retrieval. Springer. I--XVII, 1--285 pages.

Digital Library

[20]

Kira Radinsky and Paul N. Bennett. 2013. Predicting content change on the web. In Proceedings of WSDM. 415--424.

Digital Library

[21]

Kira Radinsky, Fernando Diaz, Susan T. Dumais, Milad Shokouhi, Anlei Dong, and Yi Chang. 2013. Temporal web dynamics and its application to information retrieval. In Proceedings of WSDM. 781--782.

Digital Library

[22]

Fiana Raiber and Oren Kurland. 2010. On identifying representative relevant documents. In Proceedings of CIKM. 99--108.

Digital Library

[23]

Nimrod Raifer, Fiana Raiber, Moshe Tennenholtz, and Oren Kurland. 2017. In- formation Retrieval Meets Game Theory: The Ranking Competition Between Documents' Authors. In Proceedings of SIGIR. 465--474.

Digital Library

[24]

Stephen E. Robertson. 1977. The Probability Ranking Principle in IR. Journal of Documentation 33 (1977), 294--304.

Digital Library

[25]

Aécio S. R. Santos, Bruno Pasini, and Juliana Freire. 2016. A First Study on Temporal Dynamics of Topics on the Web. In Proceedings of WWW. 849--854.

Digital Library

[26]

Ashudeep Singh and Thorsten Joachims. 2018. Fairness of Exposure in Rankings. In Proceedings of SIGKDD. 2219--2228.

Digital Library

[27]

Amit Singhal, Chris Buckley, and Mandar Mitra. 1996. Pivoted Document Length Normalization. In Proceedings of SIGIR. 21--29.

Digital Library

[28]

L. Smith and P. Sorensen. 2000. Pathalogical outcomes of observational learning. Econometrica 68 (2000), 371--398.

[29]

Moshe Tennenholtz and Oren Kurland. 2019. Rethinking search engines and recommendation systems: a game theoretic perspective. Commun. ACM 62, 12 (2019), 66--75.

Digital Library

[30]

Ryen W. White. 2016. Interactions with Search Systems. Cambridge University Press.

[31]

Qiang Wu, Christopher J. C. Burges, Krysta Marie Svore, and Jianfeng Gao. 2010. Adapting boosting for information retrieval measures. Information Retrieval 13, 3 (2010), 254--270.

Digital Library

[32]

Ke Yang and Julia Stoyanovich. 2017. Measuring Fairness in Ranked Outputs. In Proceedings of the 29th International Conference on Scientific and Statistical Database Management, Chicago, IL, USA, June 27-29, 2017. 22:1--22:6.

Digital Library

[33]

Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdi- nov, and Quoc V. Le. 2019. XLNet: Generalized Autoregressive Pretraining for Language Understanding. In Proceedings of NeurIPS. 5754--5764.

Digital Library

[34]

Elad Yom-Tov, Anat Brunstein-Klomek, Or Mandel, Arie Hadas, and Silvana Fennig. 2018. Inducing Behavioral Change in Seekers of Pro-Anorexia Content Using Internet Advertisements: Randomized Controlled Trial. JMIR Ment Health 5, 1 (22 Feb 2018), e6.

[35]

Elad Yom-Tov, Peter Muennig, and Abdulrahman M El-Sayed. 2016. Web-Based Antismoking Advertising to Promote Smoking Cessation: A Randomized Controlled Trial. J Med Internet Res 18, 11 (21 Nov 2016), e306.

[36]

Meike Zehlike, Francesco Bonchi, Carlos Castillo, Sara Hajian, Mohamed Megahed, and Ricardo A. Baeza-Yates. 2017. FA*IR: A Fair Top-k Ranking Algorithm. In Proceedings of CIKM. 1569--1578.

Digital Library

[37]

Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending Against Neural Fake News. In Proceedings of NeurIPS. 9051--9062.

Digital Library

[38]

Chengxiang Zhai and John D. Lafferty. 2001. A Study of Smoothing Methods for Language Models Applied to Ad Hoc Information Retrieval. In Proceedings of SIGIR. 334--342

Digital Library

Cited By

Robinson KGarvin MEglash RRobert LGuzdial MBennett A(2024)Making exploratory search engines using qualitative case studies: a mixed method implementation using interviews with Detroit ArtisansJournal of Integrated Global STEM10.1515/jigs-2024-00071:1(15-32)Online publication date: 15-Nov-2024
https://doi.org/10.1515/jigs-2024-0007
Nachimovsky HTennenholtz MRaiber FKurland OOosterhuis HBast HXiong C(2024)Ranking-Incentivized Document Manipulations for Multiple QueriesProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672516(61-70)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672516
Vasilisky ZKurland OTennenholtz MRaiber FYoshioka MKiseleva JAliannejadi M(2023)Content-Based Relevance Estimation in Retrieval Settings with Ranking-Incentivized Document ManipulationsProceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3578337.3605124(205-214)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.1145/3578337.3605124
Show More Cited By

Index Terms

Driving the Herd: Search Engines as Content Influencers
1. Information systems
  1. Information retrieval
    1. Search engine architectures and scalability
      1. Adversarial retrieval

Recommendations

Content-Based Relevance Estimation in Retrieval Settings with Ranking-Incentivized Document Manipulations
ICTIR '23: Proceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval

In retrieval settings such as the Web, many document authors are ranking incentivized: they opt to have their documents highly ranked for queries of interest. Consequently, they often respond to rankings by modifying their documents. These modifications ...
Ranking-Incentivized Quality Preserving Content Modification
SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

The Web is a canonical example of a competitive retrieval setting where many documents' authors consistently modify their documents to promote them in rankings. We present an automatic method for quality-preserving modification of document content --- ...
Ranking Robustness Under Adversarial Document Manipulations
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval

For many queries in the Web retrieval setting there is an on-going ranking competition: authors manipulate their documents so as to promote them in rankings. Such competitions can have unwarranted effects not only in terms of retrieval effectiveness, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management

October 2021

4966 pages

ISBN:9781450384469

DOI:10.1145/3459637

General Chairs:
Gianluca Demartini
The University of Queensland, Australia
,
Guido Zuccon
The University of Queensland, Australia
,
Program Chairs:
J. Shane Culpepper
RMIT University, Australia
,
Zi Huang
The University of Queensland, Australia
,
Hanghang Tong
University of Illinois at Urbana-Champaign, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 October 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

European Research Council (ERC)

Conference

CIKM '21

Sponsor:

CIKM '21: The 30th ACM International Conference on Information and Knowledge Management

November 1 - 5, 2021

Queensland, Virtual Event, Australia

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
129
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Robinson KGarvin MEglash RRobert LGuzdial MBennett A(2024)Making exploratory search engines using qualitative case studies: a mixed method implementation using interviews with Detroit ArtisansJournal of Integrated Global STEM10.1515/jigs-2024-00071:1(15-32)Online publication date: 15-Nov-2024
https://doi.org/10.1515/jigs-2024-0007
Nachimovsky HTennenholtz MRaiber FKurland OOosterhuis HBast HXiong C(2024)Ranking-Incentivized Document Manipulations for Multiple QueriesProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672516(61-70)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672516
Vasilisky ZKurland OTennenholtz MRaiber FYoshioka MKiseleva JAliannejadi M(2023)Content-Based Relevance Estimation in Retrieval Settings with Ranking-Incentivized Document ManipulationsProceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3578337.3605124(205-214)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.1145/3578337.3605124
Kurland OTennenholtz MAmigo ECastells PGonzalo JCarterette BCulpepper JKazai G(2022)Competitive SearchProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3532771(2838-2849)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3477495.3532771

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten