[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3589334.3645384acmconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article

Robust Decision Aggregation with Second-order Information

Published: 13 May 2024 Publication History

Abstract

We consider a decision aggregation problem with two experts who each make a binary recommendation after observing a private signal about an unknown binary world state. An agent, who does not know the joint information structure between signals and states, sees the experts' recommendations and aims to match the action with the true state. Under the scenario, we study whether supplemented additionally with second-order information (each expert's forecast on the other's recommendation) could enable a better aggregation.
We adopt a minimax regret framework to evaluate the aggregator's performance, by comparing it to an omniscient benchmark that knows the joint information structure. With general information structures, we show that second-order information provides no benefit -- no aggregator can improve over a trivial aggregator, which always follows the first expert's recommendation. However, positive results emerge when we assume experts' signals are conditionally independent given the world state. When the aggregator is deterministic, we present a robust aggregator that leverages second-order information, which can significantly outperform counterparts without it. Second, when two experts are homogeneous, by adding a non-degenerate assumption on the signals, we demonstrate that random aggregators using second-order information can surpass optimal ones without it. In the remaining settings, the second-order information is not beneficial. We also extend the above results to the setting when the aggregator's utility function is more general.

Supplemental Material

MOV File
Supplemental video

References

[1]
Itai Arieli, Yakov Babichenko, and Rann Smorodinsky. 2018. Robust forecast aggregation. Proceedings of the National Academy of Sciences, Vol. 115, 52 (2018), E12135--E12143.
[2]
Itai Arieli, Yakov Babichenko, and Rann Smorodinsky. 2020. Identifiable information structures. Games and Economic Behavior, Vol. 120 (2020), 16--27.
[3]
Itai Arieli, Yakov Babichenko, Inbal Talgam-Cohen, and Konstantin Zabarnyi. 2023. Universally Robust Information Aggregation for Binary Decisions. arXiv preprint arXiv:2302.03667 (2023).
[4]
Yakov Babichenko and Dan Garber. 2021. Learning optimal forecast aggregation in partial evidence environments. Mathematics of Operations Research, Vol. 46, 2 (2021), 628--641.
[5]
Jonathan Baron, Barbara A Mellers, Philip E Tetlock, Eric Stone, and Lyle H Ungar. 2014. Two reasons to make aggregated probability forecasts more extreme. Decision Analysis, Vol. 11, 2 (2014), 133--145.
[6]
Gabriel Carroll. 2017. Robustness and separation in multidimensional screening. Econometrica, Vol. 85, 2 (2017), 453--488.
[7]
Yi-Chun Chen, Manuel Mueller-Frank, and Mallesh M Pai. 2021. The wisdom of the crowd and higher-order beliefs. arXiv preprint arXiv:2102.02666 (2021).
[8]
Robert T Clemen and Robert L Winkler. 1986. Combining economic forecasts. Journal of Business & Economic Statistics, Vol. 4, 1 (1986), 39--46.
[9]
Henrique De Oliveira, Yuhta Ishii, and Xiao Lin. 2021. Robust merging of information. arXiv preprint arXiv:2106.00088 (2021).
[10]
Morris H DeGroot. 1974. Reaching a consensus. Journal of the American Statistical association, Vol. 69, 345 (1974), 118--121.
[11]
Yongkang Guo, Jason D Hartline, Zhihuan Huang, Yuqing Kong, Anant Shah, and Fang-Yi Yu. 2024. Algorithmic Robust Forecast Aggregation. arXiv preprint arXiv:2401.17743 (2024).
[12]
Wei He and Jiangtao Li. 2022. Correlation-robust auction design. Journal of Economic Theory, Vol. 200 (2022), 105403.
[13]
Hadi Hosseini, Debmalya Mandal, Nisarg Shah, and Kevin Shi. 2021. Surprisingly Popular Voting Recovers Rankings, Surprisingly! arXiv preprint arXiv:2105.09386 (2021).
[14]
Victor Richmond R Jose and Robert L Winkler. 2008. Simple robust averages of forecasts: Some empirical results. International journal of forecasting, Vol. 24, 1 (2008), 163--169.
[15]
Yuqing Kong. 2024. Peer Expectation in Robust Forecast Aggregation: The Possibility/Impossibility. arxiv: 2402.06062 [cs.GT]
[16]
Yuqing Kong, Yunqi Li, Yubo Zhang, Zhihuan Huang, and Jinzhao Wu. 2022. Eliciting thinking hierarchy without a prior. Advances in Neural Information Processing Systems, Vol. 35 (2022), 13329--13341.
[17]
Richard P Larrick, Albert E Mannes, and Jack B Soll. 2012. The social psychology of the wisdom of crowds. In Social judgment and decision making. Psychology Press, 227--242.
[18]
Gilat Levy and Ronny Razin. 2022. Combining forecasts in the presence of ambiguity over correlation structures. Journal of Economic Theory, Vol. 199 (2022), 105075.
[19]
Marcellin Martinie, Tom Wilkening, and Piers DL Howe. 2020. Using meta-predictions to identify experts in the crowd when past performance is unknown. Plos one, Vol. 15, 4 (2020), e0232058.
[20]
Eric Neyman and Tim Roughgarden. 2022. Are you smarter than a random expert? The robust aggregation of substitutable signals. In Proceedings of the 23rd ACM Conference on Economics and Computation. 990--1012.
[21]
Asa B Palley and Ville A Satop"a"a. 2023. Boosting the wisdom of crowds within a single judgment problem: Weighted averaging based on peer predictions. Management Science (2023).
[22]
Asa B Palley and Jack B Soll. 2019. Extracting the wisdom of crowds when information is shared. Management Science, Vol. 65, 5 (2019), 2291--2309.
[23]
Yuqi Pan, Zhaohua Chen, and Yuqing Kong. 2023. Robust Decision Aggregation with Second-order Information. arXiv preprint arXiv:2311.14094 (2023).
[24]
Drazen Prelec. 2004. A Bayesian truth serum for subjective data. science, Vol. 306, 5695 (2004), 462--466.
[25]
Dravz en Prelec, H Sebastian Seung, and John McCoy. 2017. A solution to the single-question crowd wisdom problem. Nature, Vol. 541, 7638 (2017), 532--535.
[26]
Roopesh Ranjan and Tilmann Gneiting. 2010. Combining probability forecasts. Journal of the Royal Statistical Society Series B: Statistical Methodology, Vol. 72, 1 (2010), 71--91.
[27]
David M Rothschild and Justin Wolfers. 2011. Forecasting elections: Voter intentions versus expectations. Available at SSRN 1884644 (2011).
[28]
Grant Schoenebeck and Biaoshuai Tao. 2021. Wisdom of the crowd voting: Truthful aggregation of voter information and preferences. Advances in Neural Information Processing Systems, Vol. 34 (2021), 1872--1883.
[29]
James H Stock and Mark W Watson. 2004. Combination forecasts of output growth in a seven-country data set. Journal of forecasting, Vol. 23, 6 (2004), 405--430.
[30]
Juntao Wang, Yang Liu, and Yiling Chen. 2021. Forecast aggregation via peer prediction. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 9. 131--142.
[31]
Tom Wilkening, Marcellin Martinie, and Piers DL Howe. 2022. Hidden experts in the crowd: Using meta-predictions to leverage expertise in single-question prediction problems. Management Science, Vol. 68, 1 (2022), 487--508.
[32]
Andrew Chi-Chih Yao. 1977. Probabilistic computations: Toward a unified measure of complexity. In 18th Annual Symposium on Foundations of Computer Science (sfcs 1977). IEEE Computer Society, 222--227. io

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
WWW '24: Proceedings of the ACM Web Conference 2024
May 2024
4826 pages
ISBN:9798400701719
DOI:10.1145/3589334
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 May 2024

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. decision aggregation
  2. robust aggregation
  3. second-order information

Qualifiers

  • Research-article

Funding Sources

Conference

WWW '24
Sponsor:
WWW '24: The ACM Web Conference 2024
May 13 - 17, 2024
Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 95
    Total Downloads
  • Downloads (Last 12 months)95
  • Downloads (Last 6 weeks)3
Reflects downloads up to 22 Dec 2024

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media