[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
IDEAS home Printed from https://ideas.repec.org/p/feb/natura/00799.html
   My bibliography  Save this paper

Toward an Understanding of the Political Economy of Using Field Experiments in Policymaking

Author

Listed:
  • Guglielmo Briscese
  • John List
Abstract
Field experiments provide the clearest window into the true impact of many policies, allowing us to understand what works, what does not, and why. Yet, their widespread use has not been accompanied by a deep understanding of the political economy of their adoption in policy circles. This study begins with a large-scale natural field experiment that demonstrates the ineffectiveness of a widely implemented intervention. We leverage this result to understand how policymakers and a representative sample of the U.S. population update their beliefs of not only the policy itself, but the use of science and the trust they have in government. Policymakers, initially overly optimistic about the program's effectiveness, adjust their views based on evidence but show reduced demand for experimentation, suggesting experiment aversion when results defy expectations. Among the U.S. public, support for policy experiments is high and remains robust despite receiving disappointing results, though trust in the implementing institutions declines, particularly in terms of perceptions of competence and integrity. Providing additional information on the value of learning from unexpected findings partially mitigates this trust loss. These insights, from both the demand and supply side, reveal the complexities of managing policymakers' expectations and underscore the need to educate the public on the value of open-mindedness in policy experimentation.

Suggested Citation

  • Guglielmo Briscese & John List, 2024. "Toward an Understanding of the Political Economy of Using Field Experiments in Policymaking," Natural Field Experiments 00799, The Field Experiments Website.
    • Handle: RePEc:feb:natura:00799
    as

    Download full text from publisher

    File URL: http://s3.amazonaws.com/fieldexperiments-papers2/papers/00799.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. John A. List & Azeem M. Shaikh & Yang Xu, 2019. "Multiple hypothesis testing in experimental economics," Experimental Economics, Springer;Economic Science Association, vol. 22(4), pages 773-793, December.
    2. Jonas Hjort & Diana Moreira & Gautam Rao & Juan Francisco Santini, 2021. "How Research Affects Policy: Experimental Evidence from 2,150 Brazilian Municipalities," American Economic Review, American Economic Association, vol. 111(5), pages 1442-1480, May.
    3. Shaoda Wang & David Y. Yang, 2021. "Policy Experimentation in China: the Political Economy of Policy Learning," NBER Working Papers 29402, National Bureau of Economic Research, Inc.
    4. Stefano DellaVigna & Devin Pope, 2018. "Predicting Experimental Results: Who Knows What?," Journal of Political Economy, University of Chicago Press, vol. 126(6), pages 2410-2456.
    5. Stefano DellaVigna & Woojin Kim & Elizabeth Linos, 2024. "Bottlenecks for Evidence Adoption," Journal of Political Economy, University of Chicago Press, vol. 132(8), pages 2748-2789.
    6. Toma, Mattie & Bell, Elizabeth, 2024. "Understanding and increasing policymakers’ sensitivity to program impact," Journal of Public Economics, Elsevier, vol. 234(C).
    7. Ingar Haaland & Christopher Roth & Johannes Wohlfart, 2023. "Designing Information Provision Experiments," Journal of Economic Literature, American Economic Association, vol. 61(1), pages 3-40, March.
    8. Agostinelli, Francesco & Avitabile, Ciro & Bobba, Matteo, 2021. "Enhancing Human Capital in Children: A Case Study on Scaling," TSE Working Papers 21-1196, Toulouse School of Economics (TSE), revised Oct 2023.
    9. Mason, Lisa Reyes & Nam, Yunju & Clancy, Margaret & Kim, Youngmi & Loke, Vernon, 2010. "Child Development Accounts and saving for children's future: Do financial incentives matter?," Children and Youth Services Review, Elsevier, vol. 32(11), pages 1570-1576, November.
    10. Stefano Carattini & Robert Dur & John List, 2024. "Policy evaluation and the causal analysis of public support," Natural Field Experiments 00798, The Field Experiments Website.
    11. Matthew Wiswall & Basit Zafar, 2015. "Determinants of College Major Choice: Identification using an Information Experiment," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 82(2), pages 791-824.
    12. John List, 2024. "Experimental Economics: Theory and Practice," Natural Field Experiments 00792, The Field Experiments Website.
    13. Navdeep S. Sahni & S. Christian Wheeler & Pradeep Chintagunta, 2018. "Personalization in Email Marketing: The Role of Noninformative Advertising Content," Marketing Science, INFORMS, vol. 37(2), pages 236-258, March.
    14. John A. List, 2024. "Optimally generate policy-based evidence before scaling," Nature, Nature, vol. 626(7999), pages 491-499, February.
    15. Garcia-Hombrados, Jorge & Jansen, Marcel & Martínez, Ángel & Özcan, Berkay & Rey-Biel, Pedro & Roldán-Monés, Antonio, 2024. "Ideological Alignment and Evidence-Based Policy Adoption," IZA Discussion Papers 17007, Institute of Labor Economics (IZA).
    16. Vivalt, Eva & Coville, Aidan, 2023. "How do policymakers update their beliefs?," Journal of Development Economics, Elsevier, vol. 165(C).
    17. Levitt, Steven D. & List, John A., 2009. "Field experiments in economics: The past, the present, and the future," European Economic Review, Elsevier, vol. 53(1), pages 1-18, January.
    18. Omar Al-Ubaydli & John A. List & Dana L. Suskind, 2017. "What Can We Learn from Experiments? Understanding the Threats to the Scalability of Experimental Results," American Economic Review, American Economic Association, vol. 107(5), pages 282-286, May.
    19. Ahmed Mushfiq Mobarak, 2022. "Assessing social aid: the scale-up process needs evidence, too," Nature, Nature, vol. 609(7929), pages 892-894, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Robert Dur & Arjan Non & Paul Prottung & Benedetta Ricci, 2023. "Who’s Afraid of Policy Experiments?," Tinbergen Institute Discussion Papers 23-027/V, Tinbergen Institute.
    2. John List & Haruka Uchida, 2024. "Here Today, Gone Tomorrow? Toward an Understanding of Fade-out in Early Childhood Education Programs," Framed Field Experiments 00797, The Field Experiments Website.
    3. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    4. John A. List, 2024. "Optimally generate policy-based evidence before scaling," Nature, Nature, vol. 626(7999), pages 491-499, February.
    5. Ingar Haaland & Christopher Roth, 2023. "Beliefs about Racial Discrimination and Support for Pro-Black Policies," The Review of Economics and Statistics, MIT Press, vol. 105(1), pages 40-53, January.
    6. Bobba, Matteo & Frisancho, Veronica & Pariguana, Marco, 2016. "Perceived Ability and School Choices: Experimental Evidence and Scale-up Effects," TSE Working Papers 16-660, Toulouse School of Economics (TSE), revised Jul 2024.
    7. Toma, Mattie & Bell, Elizabeth, 2024. "Understanding and increasing policymakers’ sensitivity to program impact," Journal of Public Economics, Elsevier, vol. 234(C).
    8. Frohnweiler, Sarah & Adongo, Charles A. & Beber, Bernd & Lakemann, Tabea & Priebe, Jan & Lay, Jann, 2024. "Effects of skills training on employment and livelihood outcomes: A randomized controlled trial with young women in Ghana," Ruhr Economic Papers 1095, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    9. Boumans, Dorine & Gründler, Klaus & Potrafke, Niklas & Ruthardt, Fabian, 2024. "Political leaders and macroeconomic expectations: Evidence from a global survey experiment," Journal of Public Economics, Elsevier, vol. 235(C).
    10. Mechtenberg, Lydia & Perino, Grischa & Treich, Nicolas & Tyran, Jean-Robert & Wang, Stephanie W., 2024. "Self-signaling in voting," Journal of Public Economics, Elsevier, vol. 231(C).
    11. Omar Al-Ubaydli & John List & Claire Mackevicius & Min Sok Lee & Dana Suskind, 2019. "How Can Experiments Play a Greater Role in Public Policy? 12 Proposals from an Economic Model of Scaling," Artefactual Field Experiments 00679, The Field Experiments Website.
    12. Simon Jäger & Christopher Roth & Nina Roussille & Benjamin Schoefer, 2024. "Worker Beliefs About Outside Options," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 139(3), pages 1505-1556.
    13. Michele Giannola, 2024. "Parental Investments and Intra-household Inequality in Child Human Capital: Evidence from a Survey Experiment," The Economic Journal, Royal Economic Society, vol. 134(658), pages 671-727.
    14. Ingar Haaland & Christopher Roth & Johannes Wohlfart, 2023. "Designing Information Provision Experiments," Journal of Economic Literature, American Economic Association, vol. 61(1), pages 3-40, March.
    15. Andor, Mark A. & Gerster, Andreas & Peters, Jörg & Schmidt, Christoph M., 2020. "Social Norms and Energy Conservation Beyond the US," Journal of Environmental Economics and Management, Elsevier, vol. 103(C).
    16. Ester Faia & Andreas Fuster & Vincenzo Pezone & Basit Zafar, 2024. "Biases in Information Selection and Processing: Survey Evidence from the Pandemic," The Review of Economics and Statistics, MIT Press, vol. 106(3), pages 829-847, May.
    17. Henning Hermes & Philipp Lergetporer & Frauke Peter & Simon Wiederhold, 2021. "Application Barriers and the Socioeconomic Gap in Child Care Enrollment," CESifo Working Paper Series 9282, CESifo.
    18. Jan-Emmanuel De Neve & Clément Imbert & Johannes Spinnewijn & Teodora Tsankova & Maarten Luts, 2021. "How to Improve Tax Compliance? Evidence from Population-Wide Experiments in Belgium," Journal of Political Economy, University of Chicago Press, vol. 129(5), pages 1425-1463.
    19. Jesper Akesson & Sam Ashworth-Hayes & Robert Hahn & Robert Metcalfe & Itzhak Rasooly, 2022. "Fatalism, beliefs, and behaviors during the COVID-19 pandemic," Journal of Risk and Uncertainty, Springer, vol. 64(2), pages 147-190, April.
    20. Sebastian Blesse & Philipp Lergetporer & Justus Nover & Katharina Werner, 2023. "Transparency and Policy Competition: Experimental Evidence from German Citizens and Politicians," Rationality and Competition Discussion Paper Series 387, CRC TRR 190 Rationality and Competition.

    More about this item

    JEL classification:

    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • H4 - Public Economics - - Publicly Provided Goods
    • H41 - Public Economics - - Publicly Provided Goods - - - Public Goods
    • O12 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Microeconomic Analyses of Economic Development
    • O36 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Open Innovation
    • P1 - Political Economy and Comparative Economic Systems - - Capitalist Economies

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:feb:natura:00799. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Francesca Pagnotta (email available below). General contact details of provider: http://www.fieldexperiments.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.