[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3227609.3227688acmotherconferencesArticle/Chapter ViewAbstractPublication PageswimsConference Proceedingsconference-collections
research-article

A Comparative Study of Methods for Collective Prediction Determination Using Interval Estimates

Published: 25 June 2018 Publication History

Abstract

Recently, research on the Wisdom of Crowd (WoC) has been widely expanded by supporting interval values as an additional representation of underlying predictions. Accordingly, instead of giving single values, ones can express their predictions on a given cognition problem in the form of interval values1. For such a representation, many methods have been proposed for aggregating underlying predictions based on their midpoints. In this case, of course, the outputs of the proposed methods are single values. In some situations, however, the aggregated prediction in the form of interval value can be better representation of underlying predictions. In the current study, we present a comparison of the use of different approaches for aggregating individual predictions including Interval Aggregation and MidPoint Aggregation. Experimental studies have been conducted to determine how do different aggregation methods influence the quality of the obtained collective prediction.

References

[1]
O. Arazy, N. Halfon, and D. Malkinson. 2015. Forecasting Rain Events -Meteorological Models or Collective Intelligence? In Proc. of EGU General Assembly Conference 17 (Vienna, Austria, 2015), 15611--15614
[2]
J.S. Armstrong. 2001. Combining forecasts. Principles of forecasting. Springer (2001), 417--439
[3]
J. A. Baars, and C. F. Mass. 2005. Performance of National Weather Service forecasts compared to operational, consensus, and weighted model output statistics. Weather and Forecasting 20 (2005), 1034--1047
[4]
Clemen, R. T. 1989. Combining forecasts: A review and annotated bibliography. International Journal of Forecasting 5 (1989), 559--583
[5]
F. Galton. 1907. Vox populi (The wisdom of crowds). Nature 75 (1907), 450--451
[6]
M. Glaser, T. Langer, and M. Weber. 2013. True Overconfidence in Interval Estimates: Evidence Based on a New Measure of Miscalibration. Journal of Behavioral Decision Making 26 (2013), 405--417
[7]
R. H. Kurvers, S. M. Herzog, R. Hertwig, J. Krause, P. A. Carney, A. Bogart, G. Argenziano, I. Zalaudek, and M. Wolf. 2016. Boosting medical diagnostics by pooling independent judgments. Proceedings of the National Academy of Sciences (2016), 8777--8782
[8]
R. H. Kurvers, J. Krause, G. Argenziano, I. Zalaudek, and M. Wolf. 2015. Detection accuracy of collective intelligence assessments for skin cancer diagnosis. JAMA dermatology 151 (2015), 1346--1353
[9]
M. Lang, N. Bharadwaj, and C. A. Di Benedetto. 2016. How crowdsourcing improves prediction of market-oriented outcomes. Journal of Business Research 69 (2016), 4168--4176
[10]
A. Lyon, B. C. Wintle, and M. Burgman. 2015. Collective wisdom: Methods of confidence interval aggregation. Journal of Business Research 68 (2015), 1759--1767
[11]
N. T. Nguyen (ed.). 2008. Advanced Methods for Inconsistent Knowledge Management. Springer-Verlag, London
[12]
N. T. Nguyen. 2001. Consensus-based Timestamps in Distributed Temporal Databases. The Computer Journal 44 (2001), 398--409
[13]
N. T. Nguyen. 2005. Processing Inconsistency of Knowledge on Semantic Level. Journal of Universal Computer Science 11(2005), 285--302
[14]
M. Nofer, and O. Hinz. 2014. Are crowds on the internet wiser than experts? The case of a stock prediction community. Journal of Business Economics 84 (2014), 303--338
[15]
S. Park, and D. V. Budescu. 2015. Aggregating multiple probability intervals to improve calibration. Judgment and Decision Making 10 (2015), 130--143
[16]
J. B. Soll, and J. Klayman. 2004. Overconfidence in interval estimates. Journal of Experimental Psychology: Learning, Memory, and Cognition 30 (2004), 299--314
[17]
A. Speirs-Bridge, F. Fidler, M. McBride, L. Flander, G. Cumming, and M. Burgman. 2010. Reducing Overconfidence in the Interval Judgments of Experts. Risk Analysis 30 (2010), 512--523
[18]
J. Surowiecki (ed.). 2005. The wisdom of crowds. Doubleday/Anchor, New York
[19]
M., Zgrzywa. 2007. Consensus Determining with Dependencies of Attributes with Interval Values. Journal of Universal Computer Science 13 (2007), 329--344

Index Terms

  1. A Comparative Study of Methods for Collective Prediction Determination Using Interval Estimates

      Recommendations

      Comments

      Please enable JavaScript to view thecomments powered by Disqus.

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      WIMS '18: Proceedings of the 8th International Conference on Web Intelligence, Mining and Semantics
      June 2018
      398 pages
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 25 June 2018

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Interval aggregation
      2. interval estimates
      3. wisdom of crowds

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      WIMS '18

      Acceptance Rates

      Overall Acceptance Rate 140 of 278 submissions, 50%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • 0
        Total Citations
      • 68
        Total Downloads
      • Downloads (Last 12 months)1
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 12 Dec 2024

      Other Metrics

      Citations

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media