[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
article

Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can Even Slightly Modify Them

Published: 01 March 2018 Publication History

Abstract

Although evidence-based algorithms consistently outperform human forecasters, people often fail to use them after learning that they are imperfect, a phenomenon known as algorithm aversion. In this paper, we present three studies investigating how to reduce algorithm aversion. In incentivized forecasting tasks, participants chose between using their own forecasts or those of an algorithm that was built by experts. Participants were considerably more likely to choose to use an imperfect algorithm when they could modify its forecasts, and they performed better as a result. Notably, the preference for modifiable algorithms held even when participants were severely restricted in the modifications they could make Studies 1-3. In fact, our results suggest that participants' preference for modifiable algorithms was indicative of a desire for some control over the forecasting outcome, and not for a desire for greater control over the forecasting outcome, as participants' preference for modifiable algorithms was relatively insensitive to the magnitude of the modifications they were able to make Study 2. Additionally, we found that giving participants the freedom to modify an imperfect algorithm made them feel more satisfied with the forecasting process, more likely to believe that the algorithm was superior, and more likely to choose to use an algorithm to make subsequent forecasts Study 3. This research suggests that one can reduce algorithm aversion by giving people some control-even a slight amount-over an imperfect algorithm's forecast.
Data, as supplemental material, are available at <ext-link ext-link-type="uri" href="https://doi.org/10.1287/mnsc.2016.2643">https://doi.org/10.1287/mnsc.2016.2643</ext-link>.
This paper was accepted by Yuval Rottenstreich, judgment and decision making.

References

[1]
Adams ID, Chan M, Clifford PC, Cooke WM, Dallos V, De Dombal FT, McIntyre N (1986) Computer aided diagnosis of acute abdominal pain: A multicentre study. British Medical J. 293(6550):800-804.
[2]
Beck AH, Sangoi AR, Leung S, Marinelli RJ, Nielsen TO, van de Vijver MJ, Koller D (2011) Systematic analysis of breast cancer morphology uncovers stromal features associated with survival. Sci. Translational Medicine 3(108):108ra113.
[3]
Bonaccio S, Dalal RS (2006) Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences. Organ. Behav. Human Decision Processes 101(2): 127-151.
[4]
Carbone R, Andersen A, Corriveau Y, Corson PP (1983) Comparing for different time series methods the value of technical expertise individualized analysis, and judgmental adjustment. Management Sci. 29(5):559-566.
[5]
Dawes RM (1971) A case study of graduate admissions: Application of three principles of human decision making. Amer. Psychologist 26(2):180-188.
[6]
Dawes RM (1979) The robust beauty of improper linear models in decision making. Amer. Psychologist 34(7):571-582.
[7]
Dawes RM, Faust D, Meehl PE (1989) Clinical versus actuarial judgment. Science 243(4899):1668-1674.
[8]
Dietvorst BJ, Simmons JP, Massey C (2015) Algorithm aversion: People erroneously avoid algorithms after seeing them err. J. Experiment. Psych.: General 144(1):114-126.
[9]
Einhorn HJ (1986) Accepting error to make less error. J. Personality Assessment 50(3):387-395.
[10]
Fildes R, Goodwin P (2007) Against your better judgment? How organizations can improve their use of management judgment in forecasting. Interfaces 37(6):570-576.
[11]
Fildes R, Goodwin P, Lawrence M, Nikolopoulos K (2009) Effective forecasting and judgmental adjustments: An empirical evaluation and strategies for improvement in supply-chain planning. Internat. J. Forecasting 25(1):3-23.
[12]
Finucane ML, Alhakami A, Slovic P, Johnson SM (2000) The affect heuristic in judgments of risks and benefits. J. Behav. Decision Making 13(1):1-17.
[13]
Goodwin P, Fildes R (1999) Judgmental forecasts of time series affected by special events: Does providing a statistical forecast improve accuracy? J. Behav. Decision Making 12(1):37-53.
[14]
Grove WM, Zald DH, Lebow BS, Snitz BE, Nelson C (2000) Clinical versus mechanical prediction: A meta-analysis. Psych. Assessment 12(1):19-30.
[15]
Highhouse S (2008) Stubborn reliance on intuition and subjectivity in employee selection. Indust. Organ. Psych. 1(3):333-342.
[16]
Hogarth RM, Makridakis S (1981) Forecasting and planning: An evaluation. Management Sci. 27(2):115-138.
[17]
Lawrence M, Goodwin P, O'Connor M, Önkal D (2006) Judgmental forecasting: A review of progress over the last 25 years. Internat. J. Forecasting 22(3):493-518.
[18]
Lim JS, O'Connor M (1995) Judgemental adjustment of initial forecasts: Its effectiveness and biases. J. Behav. Decision Making 8(3):149-168.
[19]
Meehl PE (1954) Clinical versus Statistical Prediction: A Theoretical Analysis and Review of the Literature (University of Minnesota Press, Minneapolis).
[20]
Preacher KJ, Hayes AF (2008) Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behav. Res. Methods 40(3):879-891.
[21]
Sanders NR, Manrodt KB (2003) The efficacy of using judgmental versus quantitative forecasting methods in practice. Omega 31(6):511-522.
[22]
Schweitzer ME, Cachon GP (2000) Decision bias in the newsvendor problem with a known demand distribution: Experimental evidence. Management Sci. 46(3):404-420.
[23]
Thompson RE (1952) A validation of the Glueck Social Prediction Scale for proneness to delinquency. J. Criminal Law: Criminology Police Sci. 43(4):451-470.
[24]
Vrieze SI, Grove WM (2009) Survey on the use of clinical and mechanical prediction methods in clinical psychology. Professional Psych.: Res. Practice 40(5):525-531.
[25]
Willemain TR (1991) The effect of graphical adjustment on forecast accuracy. Internat. J. Forecasting 7(2):151-154.
[26]
Wormith JS, Goldstone CS (1984) The clinical and statistical prediction of recidivism. Criminal Justice Behav. 11(1):3-34.

Cited By

View all
  1. Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can Even Slightly Modify Them

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image Management Science
    Management Science  Volume 64, Issue 3
    March 2018
    495 pages

    Publisher

    INFORMS

    Linthicum, MD, United States

    Publication History

    Published: 01 March 2018
    Accepted: 15 August 2016
    Received: 11 June 2015

    Author Tags

    1. confidence
    2. decision aids
    3. decision making
    4. forecasting
    5. heuristics and biases

    Qualifiers

    • Article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)0
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 11 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Physician Adoption of AI AssistantManufacturing & Service Operations Management10.1287/msom.2023.009326:5(1639-1655)Online publication date: 1-Sep-2024
    • (2024)Human and MachineManagement Science10.1287/mnsc.2023.474470:2(1258-1275)Online publication date: 1-Feb-2024
    • (2024)Mirror, Mirror on the WallInformation Systems Research10.1287/isre.2023.121735:1(226-248)Online publication date: 1-Mar-2024
    • (2024)Can user engagement affect trust in audio guide agent?Proceedings of the 12th International Conference on Human-Agent Interaction10.1145/3687272.3688311(44-53)Online publication date: 24-Nov-2024
    • (2024)(De)Noise: Moderating the Inconsistency Between Human Decision-MakersProceedings of the ACM on Human-Computer Interaction10.1145/36869878:CSCW2(1-38)Online publication date: 8-Nov-2024
    • (2024)Intermediation: Algorithmic Prioritization in Practice in Homeless ServicesProceedings of the ACM on Human-Computer Interaction10.1145/36869518:CSCW2(1-24)Online publication date: 8-Nov-2024
    • (2024)Grounding with Structure: Exploring Design Variations of Grounded Human-AI Collaboration in a Natural Language InterfaceProceedings of the ACM on Human-Computer Interaction10.1145/36869028:CSCW2(1-27)Online publication date: 8-Nov-2024
    • (2024)Belief Miner: A Methodology for Discovering Causal Beliefs and Causal Illusions from General PopulationsProceedings of the ACM on Human-Computer Interaction10.1145/36372988:CSCW1(1-37)Online publication date: 26-Apr-2024
    • (2024)Impact of Model Interpretability and Outcome Feedback on Trust in AIProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642780(1-25)Online publication date: 11-May-2024
    • (2024)I lose vs. I earn: Consumer perceived price fairness toward algorithmic (vs. human) price discriminationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642280(1-17)Online publication date: 11-May-2024
    • Show More Cited By

    View Options

    View options

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media