[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/29880.html
   My bibliography  Save this paper

Does Human-algorithm Feedback Loop Lead to Error Propagation? Evidence from Zillow’s Zestimate

Author

Listed:
  • Runshan Fu
  • Ginger Zhe Jin
  • Meng Liu
Abstract
We study how home sellers and buyers interact with Zillow’s Zestimate algorithm throughout the sales cycle of residential properties, with an emphasis on the implications of such interactions. In particular, leveraging Zestimate’s algorithm updates as exogenous shocks, we find evidence for a human-algorithm feedback loop: listing and selling outcomes respond significantly to Zestimate, and Zestimate is quickly updated for the focal and comparable homes after a property is listed or sold. This raises a concern that housing market disturbances may propagate and persist because of the feedback loop. However, simulation suggests that disturbances are short-lived and diminish eventually, mainly because all marginal effects across stages of the selling process—though sizable and significant—are less than one. To further validate this insight in the real data, we leverage the COVID-19 pandemic as a natural experiment. We find consistent evidence that the initial disturbances created by the March-2020 declaration of national emergency faded away in a few months. Overall, our results identify the human-algorithm feedback loop in an important real-world setting, but dismiss the concern that such a feedback loop generates persistent error propagation.

Suggested Citation

  • Runshan Fu & Ginger Zhe Jin & Meng Liu, 2022. "Does Human-algorithm Feedback Loop Lead to Error Propagation? Evidence from Zillow’s Zestimate," NBER Working Papers 29880, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:29880
    Note: IO
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w29880.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Mitchell Hoffman & Lisa B Kahn & Danielle Li, 2018. "Discretion in Hiring," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(2), pages 765-800.
    2. Douglas W. Diamond & Raghuram G. Rajan, 2011. "Fear of Fire Sales, Illiquidity Seeking, and Credit Freezes," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(2), pages 557-591.
    3. Anja Lambrecht & Catherine Tucker, 2019. "Algorithmic Bias? An Empirical Study of Apparent Gender-Based Discrimination in the Display of STEM Career Ads," Management Science, INFORMS, vol. 65(7), pages 2966-2981, July.
    4. Douglas W. Diamond & Philip H. Dybvig, 2000. "Bank runs, deposit insurance, and liquidity," Quarterly Review, Federal Reserve Bank of Minneapolis, vol. 24(Win), pages 14-23.
    5. Jens Ludwig & Sendhil Mullainathan, 2021. "Fragile Algorithms and Fallible Decision-Makers: Lessons from the Justice System," Journal of Economic Perspectives, American Economic Association, vol. 35(4), pages 71-96, Fall.
    6. Matthew O. Jackson & Agathe Pernoud, 2021. "Systemic Risk in Financial Networks: A Survey," Annual Review of Economics, Annual Reviews, vol. 13(1), pages 171-202, August.
    7. Richard Berk, 2019. "Accuracy and Fairness for Juvenile Justice Risk Assessments," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 16(1), pages 175-194, March.
    8. Bikhchandani, Sushil & Hirshleifer, David & Welch, Ivo, 1992. "A Theory of Fads, Fashion, Custom, and Cultural Change in Informational Cascades," Journal of Political Economy, University of Chicago Press, vol. 100(5), pages 992-1026, October.
    9. Emilio Calvano & Giacomo Calzolari & Vincenzo Denicolò & Sergio Pastorello, 2020. "Artificial Intelligence, Algorithmic Pricing, and Collusion," American Economic Review, American Economic Association, vol. 110(10), pages 3267-3297, October.
    10. Anderson, Lisa R & Holt, Charles A, 1997. "Information Cascades in the Laboratory," American Economic Review, American Economic Association, vol. 87(5), pages 847-862, December.
    11. Stephanie Assad & Robert Clark & Daniel Ershov & Lei Xu, 2020. "Algorithmic Pricing and Competition: Empirical Evidence from the German Retail Gasoline Market," CESifo Working Paper Series 8521, CESifo.
    12. Lucian A. Bebchuk & Itay Goldstein, 2011. "Self-fulfilling Credit Market Freezes," The Review of Financial Studies, Society for Financial Studies, vol. 24(11), pages 3519-3555.
    13. Lones Smith & Peter Sorensen, 2000. "Pathological Outcomes of Observational Learning," Econometrica, Econometric Society, vol. 68(2), pages 371-398, March.
    14. Zach Y. Brown & Alexander MacKay, 2023. "Competition in Pricing Algorithms," American Economic Journal: Microeconomics, American Economic Association, vol. 15(2), pages 109-156, May.
    15. Matthew Gentzkow & Jesse M. Shapiro, 2011. "Ideological Segregation Online and Offline," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(4), pages 1799-1839.
    16. Bogaçhan Çelen & Shachar Kariv, 2004. "Distinguishing Informational Cascades from Herd Behavior in the Laboratory," American Economic Review, American Economic Association, vol. 94(3), pages 484-498, June.
    17. Jean-Charles Rochet & Jean Tirole, 1996. "Interbank lending and systemic risk," Proceedings, Board of Governors of the Federal Reserve System (U.S.), pages 733-765.
    18. Greg Buchak & Gregor Matvos & Tomasz Piskorski & Amit Seru, 2020. "Why is Intermediating Houses so Difficult? Evidence from iBuyers," NBER Working Papers 28252, National Bureau of Economic Research, Inc.
    19. Jon Kleinberg & Himabindu Lakkaraju & Jure Leskovec & Jens Ludwig & Sendhil Mullainathan, 2018. "Human Decisions and Machine Predictions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(1), pages 237-293.
    20. Abhijit V. Banerjee, 1992. "A Simple Model of Herd Behavior," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 107(3), pages 797-817.
    21. Paul E. Carrillo, 2012. "An Empirical Stationary Equilibrium Search Model Of The Housing Market," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 53(1), pages 203-234, February.
    22. Sushil Bikhchandani & David Hirshleifer & Ivo Welch, 1998. "Learning from the Behavior of Others: Conformity, Fads, and Informational Cascades," Journal of Economic Perspectives, American Economic Association, vol. 12(3), pages 151-170, Summer.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhu Mingxi & Song Michelle, 2024. "Design Information Disclosure under Bidder Heterogeneity in Online Advertising Auctions: Implications of Bid-Adherence Behavior," Papers 2410.05535, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hirshleifer, David & Teoh, Siew Hong, 2008. "Thought and Behavior Contagion in Capital Markets," MPRA Paper 9142, University Library of Munich, Germany.
    2. Jonathan E. Alevy & Michael S. Haigh & John List, 2006. "Information Cascades: Evidence from An Experiment with Financial Market Professionals," NBER Working Papers 12767, National Bureau of Economic Research, Inc.
    3. Drehmann, Mathias & Oechssler, Jorg & Roider, Andreas, 2007. "Herding with and without payoff externalities -- an internet experiment," International Journal of Industrial Organization, Elsevier, vol. 25(2), pages 391-415, April.
    4. Shachar Kariv, 2005. "Overconfidence and Informational Cascades," Levine's Bibliography 122247000000000406, UCLA Department of Economics.
    5. repec:awi:wpaper:0420 is not listed on IDEAS
    6. Bou{g}açhan Çelen & Shachar Kariv & Andrew Schotter, 2010. "An Experimental Test of Advice and Social Learning," Management Science, INFORMS, vol. 56(10), pages 1687-1701, October.
    7. Fahr, René & Irlenbusch, Bernd, 2011. "Who follows the crowd—Groups or individuals?," Journal of Economic Behavior & Organization, Elsevier, vol. 80(1), pages 200-209.
    8. Wu, Jiemai, 2015. "Helpful laymen in informational cascades," Journal of Economic Behavior & Organization, Elsevier, vol. 116(C), pages 407-415.
    9. Puput Tri Komalasari & Marwan Asri & Bernardinus M. Purwanto & Bowo Setiyono, 2022. "Herding behaviour in the capital market: What do we know and what is next?," Management Review Quarterly, Springer, vol. 72(3), pages 745-787, September.
    10. Stone, Daniel F. & Miller, Steven J., 2013. "Leading, learning and herding," Mathematical Social Sciences, Elsevier, vol. 65(3), pages 222-231.
    11. Lukas Meub & Till Proeger & Hendrik Hüning, 2017. "A comparison of endogenous and exogenous timing in a social learning experiment," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 12(1), pages 143-166, April.
    12. Hongbin Cai & Yuyu Chen & Hanming Fang, 2009. "Observational Learning: Evidence from a Randomized Natural Field Experiment," American Economic Review, American Economic Association, vol. 99(3), pages 864-882, June.
    13. Fishman, Arthur & Fishman, Ram & Gneezy, Uri, 2019. "A tale of two food stands: Observational learning in the field," Journal of Economic Behavior & Organization, Elsevier, vol. 159(C), pages 101-108.
    14. Feri, Francesco & Meléndez-Jiménez, Miguel A. & Ponti, Giovanni & Vega-Redondo, Fernando, 2011. "Error cascades in observational learning: An experiment on the Chinos game," Games and Economic Behavior, Elsevier, vol. 73(1), pages 136-146, September.
    15. Stephanie De Mel & Kaivan Munshi & Soenje Reiche & Hamid Sabourian, 2021. "Herding with Heterogeneous Ability: An Application to Organ Transplantation," Cowles Foundation Discussion Papers 2308, Cowles Foundation for Research in Economics, Yale University.
    16. Anna K. Edenbrandt & Christian Gamborg & Bo Jellesmark Thorsen, 2020. "Observational learning in food choices: The effect of product familiarity and closeness of peers," Agribusiness, John Wiley & Sons, Ltd., vol. 36(3), pages 482-498, June.
    17. Daron Acemoglu & Munther A. Dahleh & Ilan Lobel & Asuman Ozdaglar, 2011. "Bayesian Learning in Social Networks," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 78(4), pages 1201-1236.
    18. Penczynski, Stefan P., 2017. "The nature of social learning: Experimental evidence," European Economic Review, Elsevier, vol. 94(C), pages 148-165.
    19. Michael Seiler & Mark Lane & David Harrison, 2014. "Mimetic Herding Behavior and the Decision to Strategically Default," The Journal of Real Estate Finance and Economics, Springer, vol. 49(4), pages 621-653, November.
    20. Paul J. Healy & John Conlon & Yeochang Yoon, 2016. "Information Cascades with Informative Ratings: An Experimental Test," Working Papers 16-05, Ohio State University, Department of Economics.
    21. James C. D. Fisher & John Wooders, 2017. "Interacting information cascades: on the movement of conventions between groups," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 63(1), pages 211-231, January.

    More about this item

    JEL classification:

    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • L11 - Industrial Organization - - Market Structure, Firm Strategy, and Market Performance - - - Production, Pricing, and Market Structure; Size Distribution of Firms
    • L85 - Industrial Organization - - Industry Studies: Services - - - Real Estate Services
    • L86 - Industrial Organization - - Industry Studies: Services - - - Information and Internet Services; Computer Software

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:29880. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.