Cited By
View all- Bier VFrench S(2020)From the EditorsDecision Analysis10.1287/deca.2020.040817:1(1-8)Online publication date: 1-Mar-2020
On the Road to Making Science of "Art": Risk Bias in Market Scoring Rules
Abstract
We study market scoring rule MSR prediction markets in the presence of risk-averse or risk-seeking agents that have unknown yet bounded risk preferences. It is well known that if agents can be prescreened, then MSRs can be corrected to elicit agents' beliefs. However, agents cannot always be screened, and instead, an online MSR mechanism is needed. We show that agents' submitted reports always deviate from their beliefs, unless their beliefs are identical to the current market estimate. This means, in most cases it is impossible for a MSR prediction market to elicit an individual agent's exact belief. To analyze this issue, we introduce a measure to calculate the deviation between an agent's reported belief and personal belief. We further derive the necessary and sufficient conditions for a MSR to yield a lower deviation relative to another MSR. We find that the deviation of a MSR prediction market is related to the liquidity provided in the MSR's corresponding cost-function prediction market. We use the relation between deviation and liquidity to present a systematic approach to help determine the amount of liquidity required for cost-function prediction markets, an activity that up to this point has been described as "art" in the literature.
References
[1]
Abernethy J, Chen Y, Vaughan JW (2011) An optimization-based framework for automated market-making. Proc. 12th ACM Conf. Electronic Commerce (Association for Computing Machinery, New York), 297-306.
[2]
Abernethy J, Chen Y, Vaughan JW (2013) Efficient market making via convex optimization, and a connection to online learning. ACM Trans. Econom. Comput. 1(2):12.
[3]
Abernethy J, Kutty S, Lahaie S, Sami R (2014) Information aggregation in exponential family markets. Proc. Fifteenth ACM Conf. Econom. Comput. (Association for Computing Machinery, New York), 395-412.
[4]
Allen F (1987) Notes-discovering personal probabilities when utility functions are unknown. Management Sci. 33(4):542-544.
[5]
Arrow KJ, Forsythe R, Gorham M, Hahn R, Hanson R, Ledyard JO, Levmore S, et al. (2008) The promise of prediction markets. Science 320(5878):877-878.
[6]
Atanasov P, Rescober P, Stone E, Swift SA, Servan-Schreiber E, Tetlock P, Ungar L, Mellers B (2016) Distilling the wisdom of crowds: Prediction markets vs. prediction polls. Management Sci. 63(3):691-706.
[7]
Babcock BA, Choi EK, Feinerman E (1993) Risk and probability premiums for CARA utility functions. J. Agricultural Resource Econom. 18(1):17-24.
[8]
Berg JE, Neumann GR, Rietz TA (2009) Searching for Google's value: Using prediction markets to forecast market capitalization prior to an initial public offering. Management Sci. 55(3):348-361.
[9]
Bossaerts P, Fine L, Ledyard J (2002) Inducing liquidity in thin financial markets through combined-value trading mechanisms. Eur. Econom. Rev. 46(9):1671-1695.
[10]
Brahma A, Chakraborty M, Das S, Lavoie A, Magdon-Ismail M (2012) A bayesian market maker. Proc. 13th ACM Conf. Electronic Commerce (Association for Computing Machinery, New York), 215-232.
[11]
Brier GW (1950) Verification of forecasts expressed in terms of probability. Monthly Weather Rev. 78(1):1-3.
[12]
Chen KY, Fine LR, Huberman BA (2004) Eliminating public knowledge biases in information-aggregation mechanisms. Management Sci. 50(7):983-994.
[13]
Chen MK, Ingersoll JE Jr, Kaplan EH (2008) Modeling a presidential prediction market. Management Sci. 54(8):1381-1394.
[14]
Chen Y, Pennock DM (2007) A utility framework for bounded-loss market makers. Proc. 23rd Conf. Uncertainty Artificial Intelligence (Association for Uncertainty in Artificial Intelligence, Portland, Oregon), 49-56.
[15]
Chen Y, Vaughan JW (2010) A new understanding of prediction markets via no-regret learning. Proc. 11th ACM Conf. Electronic Commerce (Association for Computing Machinery, New York), 189-198.
[16]
Cowgill B, Zitzewitz E (2015) Corporate prediction markets: Evidence from Google, Ford, and Firm x. Rev. Econom. Stud. 82(4):1309-1341.
[17]
Cox JC, Harrison GW (2008) Risk Aversion in Experiments, Vol. 12 (Emerald Group Publishing, Bingley, UK).
[18]
Csaszar FA, Eggers JP (2013) Organizational decision making: An information aggregation view. Management Sci. 59(10): 2257-2277.
[19]
Dimitrov S, Sami R, Epelman MA (2015) Subsidized prediction mechanisms for risk-averse agents. ACM Trans. Econom. Comput. 3(4):24.
[20]
Dudík M, Lahaie S, Rogers R, Vaughan JW (2017) A decomposition of forecast error in prediction markets. Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R, eds. Adv. Neural Inform. Processing Systems (Curran Associates, Inc., New York), 4374-4383.
[21]
Economist, The (2016) Polls versus prediction markets: Who said Brexit was a surprise? (June 24), https://www.economist.com/blogs/graphicdetail/2016/06/polls-versus-prediction-markets.
[22]
Filios M (2016) Identifying black swans. (May 9), https://www.linkedin.com/pulse/identifying-black-swans-your-organization-before-occur-matt-filios
[23]
Frongillo R, Reid MD (2015) Convergence analysis of prediction markets via randomized subspace descent. Cortes C, Lawrence ND, Lee DD, Sugiyama M, Garnett R, eds. Proc. 28th Internat. Conf. Neural Inform. Processing Systems, Vol. 2 (MIT Press, Cambridge, MA), 3034-3042.
[24]
Frongillo RM, Penna ND, Reid MD (2012) Interpreting prediction markets: A stochastic approach. Pereira F, Burges CJC, Bottou L, Weinberger KQ, eds. Proc. 25th Internat. Conf. Neural Inform. Processing Systems, Vol. 2 (Curran Associates Inc., New York), 3266-3274.
[25]
Gneiting T, Raftery AE (2007) Strictly proper scoring rules, prediction, and estimation. J. Amer. Statist. Assoc. 102(477):359-378.
[26]
Good IJ (1952) Rational decisions. J. Roy. Statist. Soc. Ser. B (Methodological) 14(1):107-114.
[27]
Hanson R (2003) Combinatorial information market design. Inform. Systems Frontiers 5(1):107-119.
[28]
Healy PJ, Linardi S, Lowery JR, Ledyard JO (2010) Prediction markets: Alternative mechanisms for complex environments with few traders. Management Sci. 56(11):1977-1996.
[29]
Heidari H, Lahaie S, Pennock DM, Vaughan JW (2015) Integrating market makers, limit orders, and continuous trade in prediction markets. Proc. Sixteenth ACM Conf. Econom. Comput. (Association for Computing Machinery, New York), 583-600.
[30]
Kadane JB, Winkler RL (1988) Separating probability elicitation from utilities. J. Amer. Statist. Assoc. 83(402):357-363.
[31]
Karni E (2009) A mechanism for eliciting probabilities. Econometrica 77(2):603-606.
[32]
Lambert N (2011) Probability elicitation for agents with arbitrary risk preferences. Working paper, Graduate School of Business, Stanford University, Stanford, CA.
[33]
Ledbetter J (2008) The fizz-dom of crowds: If prediction markets are so great, why have they been so wrong lately? Slate (April 16), http://www.slate.com/articles/news_and_politics/hey_wait_a_minute/2008/04/the_fizzdom_of_crowds.html.
[34]
Li X, Vaughan JW (2013) An axiomatic characterization of adaptive-liquidity market makers. Proc. Fourteenth ACM Conf. Electronic Commerce (Association for Computing Machinery, New York), 657-674.
[35]
Manski CF (2006) Interpreting the predictions of prediction markets. Econom. Lett. 91(3):425-429.
[36]
McCarthy J (1956) Measures of the value of information. Proc. Natl. Acad. Sci. USA 42(9):654-655.
[37]
Mellers B, Stone E, Atanasov P, Rohrbaugh N, Metz SE, Ungar L, Bishop MM, Horowitz M, Merkle E, Tetlock P (2015) The psychology of intelligence analysis: Drivers of prediction accuracy in world politics. J. Experiment. Psych.: Appl. 21(1):1-14.
[38]
Murphy AH, Winkler RL (1970) Scoring rules in probability assessment and evaluation. Acta Psychologica 34:273-286.
[39]
Offerman T, Sonnemans J, Van de Kuilen G, Wakker PP (2009) A truth serum for non-Bayesians: Correcting proper scoring rules for risk attitudes. Rev. Econom. Stud. 76(4):1461-1489.
[40]
Ortner G (1998) Forecasting markets--An industrial application. Working paper, University of Vienna, Vienna.
[41]
Ostrovsky M (2012) Information aggregation in dynamic markets with strategic traders. Econometrica 80(6):2595-2647.
[42]
Othman A, Pennock DM, Reeves DM, Sandholm T (2013) A practical liquidity-sensitive automated market maker. ACM Trans. Econom. Comput. 1(3):14-24.
[43]
Palfrey TR, Wang SW (2012) Speculative overpricing in asset markets with information flows. Econometrica 80(5):1937-1976.
[44]
Pennock DM (2010) Why automated market makers? Oddhead (blog) (July 8), blog.oddhead.com/2010/07/08/why-automated-market-makers/.
[45]
Plott CR, Chen KY (2002) Information aggregation mechanisms: Concept, design and implementation for a sales forecasting problem. Working paper, Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA.
[46]
Polgreen PM, Nelson FD, Neumann GR, Weinstein RA (2007) Use of prediction markets to forecast infectious disease activity. Clinical Infectious Diseases 44(2):272-279.
[47]
Pratt JW (1964) Risk aversion in the small and in the large. Econometrica 32(1-2):122-136.
[48]
Premachandra M, Reid M (2013) Aggregating predictions via sequential mini-trading. Ong CS, Ho TB, eds. Proc. 5th Asian Conf. Machine Learn., Vol. 29 (Microtome Publishing, Brookline, MA), 373-387.
[49]
Savage LJ (1971) Elicitation of personal probabilities and expectations. J. Amer. Statist. Assoc. 66(336):783-801.
[50]
Schlack JW (2015) Ask your customers for predictions, not preferences. Harvard Bus. Rev. (January 5), https://hbr.org/2015/01/ask-your-customers-for-predictions-not-preferences.
[51]
Schlag KH, van der Weele JJ (2013) Eliciting probabilities, means, medians, variances and covariances without assuming risk neutrality. Theoret. Econom. Lett. 3(1):38-42.
[52]
Sethi R, Vaughan JW (2016) Belief aggregation with automated market makers. Comput. Econom. 48(1):155-178.
[53]
Slamka C, Skiera B, Spann M (2013) Prediction market performance and market liquidity: A comparison of automated market makers. IEEE Trans. Engrg. Management 60(1):169-185.
[54]
Tziralis G, Tatsiopoulos I (2012) Prediction markets: An extended literature review. J. Prediction Markets 1(1):75-91.
[55]
Wah E, Lahaie S, Pennock DM (2016) An empirical game-theoretic analysis of price discovery in prediction markets. Proc. 25th Internat. Joint Conf. Artificial Intelligence (AAAI Press, New York), 510-516.
[56]
Wakker P, Thaler R, Tversky A (1997) Probabilistic insurance. J. Risk Uncertainty 15(1):7-28.
[57]
Winkler RL (1969) Scoring rules and the evaluation of probability assessors. J. Amer. Statist. Assoc. 64(327):1073-1078.
[58]
Wolfers J, Zitzewitz E (2006) Interpreting prediction market prices as probabilities. Technical report, National Bureau of Economic Research, Cambridge, MA.
- On the Road to Making Science of "Art": Risk Bias in Market Scoring Rules
Recommendations
A practical liquidity-sensitive automated market maker
EC '10: Proceedings of the 11th ACM conference on Electronic commerceCurrent automated market makers over binary events suffer from two problems that make them impractical. First, they are unable to adapt to liquidity, so trades cause prices to move the same amount in both thick and thin markets. Second, under normal ...
A Practical Liquidity-Sensitive Automated Market Maker
Automated market makers are algorithmic agents that enable participation and information elicitation in electronic markets. They have been widely and successfully applied in artificial-money settings, like some Internet prediction markets. Automated ...
Funding Liquidity and Market Liquidity: The Broker-Dealer Perspective
We provide direct evidence of how dealers’ funding liquidity affects their liquidity provision in securities markets. Worse funding liquidity (higher repo haircuts and rates) leads to larger bid-ask spreads and transaction costs in corporate bonds. We ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Publisher
INFORMS
Linthicum, MD, United States
Publication History
Published: 01 June 2018
Accepted: 11 September 2017
Received: 26 September 2016
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 06 Jan 2025
Other Metrics
Citations
Cited By
View all- Bier VFrench S(2020)From the EditorsDecision Analysis10.1287/deca.2020.040817:1(1-8)Online publication date: 1-Mar-2020