[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/26861.html
   My bibliography  Save this paper

Reasonable Doubt: Experimental Detection of Job-Level Employment Discrimination

Author

Listed:
  • Patrick M. Kline
  • Christopher R. Walters
Abstract
This paper develops methods for detecting discrimination by individual employers using correspondence experiments that send fictitious resumes to real job openings. We establish identification of higher moments of the distribution of job-level callback rates as a function of the number of resumes sent to each job and propose shape-constrained estimators of these moments. Applying our methods to three experimental datasets, we find striking job-level heterogeneity in the extent to which callback probabilities differ by race or sex. Estimates of higher moments reveal that while most jobs barely discriminate, a few discriminate heavily. These moment estimates are then used to bound the share of jobs that discriminate and the posterior probability that each individual job is engaged in discrimination. In a recent experiment manipulating racially distinctive names, we find that at least 85% of jobs that contact both of two white applications and neither of two black applications are engaged in discrimination. To assess the potential value of our methods for regulators, we consider the accuracy of decision rules for investigating suspicious callback behavior in various experimental designs under a simple two-type model that rationalizes the experimental data. Though we estimate that only 17% of employers discriminate on the basis of race, we find that an experiment sending 10 applications to each job would enable detection of 7-10% of discriminatory jobs while yielding Type I error rates below 0.2%. A minimax decision rule acknowledging partial identification of the distribution of callback rates yields only slightly fewer investigations than a Bayes decision rule based on the two-type model. These findings suggest illegal labor market discrimination can be reliably monitored with relatively small modifications to existing correspondence designs.

Suggested Citation

  • Patrick M. Kline & Christopher R. Walters, 2020. "Reasonable Doubt: Experimental Detection of Job-Level Employment Discrimination," NBER Working Papers 26861, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:26861
    Note: LE LS PE TWP
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w26861.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Eva O. Arceo-Gomez & Raymundo M. Campos-Vazquez, 2014. "Race and Marriage in the Labor Market: A Discrimination Correspondence Study in a Developing Country," American Economic Review, American Economic Association, vol. 104(5), pages 376-380, May.
    2. David Arnold & Will Dobbie & Peter Hull, 2022. "Measuring Racial Discrimination in Bail Decisions," American Economic Review, American Economic Association, vol. 112(9), pages 2992-3038, September.
    3. Raj Chetty & Nathaniel Hendren, 2018. "The Impacts of Neighborhoods on Intergenerational Mobility I: Childhood Exposure Effects," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(3), pages 1107-1162.
    4. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    5. Roland G. Fryer & Steven D. Levitt, 2004. "The Causes and Consequences of Distinctively Black Names," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(3), pages 767-805.
    6. Yoram Halevy & Dotan Persitz & Lanny Zrill, 2018. "Parametric Recoverability of Preferences," Journal of Political Economy, University of Chicago Press, vol. 126(4), pages 1558-1593.
    7. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood," American Economic Review, American Economic Association, vol. 104(9), pages 2633-2679, September.
    8. Jonathan Guryan & Kerwin Kofi Charles, 2013. "Taste‐based or Statistical Discrimination: The Economics of Discrimination Returns to its Roots," Economic Journal, Royal Economic Society, vol. 123(11), pages 417-432, November.
    9. Altonji, Joseph G. & Blank, Rebecca M., 1999. "Race and gender in the labor market," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 48, pages 3143-3259, Elsevier.
    10. Victor Chernozhukov & Whitney K. Newey & Andres Santos, 2023. "Constrained Conditional Moment Restriction Models," Econometrica, Econometric Society, vol. 91(2), pages 709-736, March.
    11. Nunley John M. & Pugh Adam & Romero Nicholas & Seals R. Alan, 2015. "Racial Discrimination in the Labor Market for Recent College Graduates: Evidence from a Field Experiment," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 15(3), pages 1093-1125, July.
    12. Joshua D. Angrist & Peter D. Hull & Parag A. Pathak & Christopher R. Walters, 2017. "Leveraging Lotteries for School Value-Added: Testing and Estimation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(2), pages 871-919.
    13. Marianne Bertrand & Sendhil Mullainathan, 2004. "Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination," American Economic Review, American Economic Association, vol. 94(4), pages 991-1013, September.
    14. Efron B. & Tibshirani R. & Storey J.D. & Tusher V., 2001. "Empirical Bayes Analysis of a Microarray Experiment," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1151-1160, December.
    15. Raj Chetty & John N. Friedman & Nathaniel Hendren & Maggie R. Jones & Sonya R. Porter, 2018. "The Opportunity Atlas: Mapping the Childhood Roots of Social Mobility," Working Papers 18-42, Center for Economic Studies, U.S. Census Bureau.
    16. John D. Storey, 2002. "A direct approach to false discovery rates," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(3), pages 479-498, August.
    17. Raj Chetty & Nathaniel Hendren, 2018. "The Impacts of Neighborhoods on Intergenerational Mobility II: County-Level Estimates," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(3), pages 1163-1228.
    18. repec:eee:labchp:v:3:y:1999:i:pc:p:3143-3259 is not listed on IDEAS
    19. Henry S. Farber & Dan Silverman & Till von Wachter, 2016. "Determinants of Callbacks to Job Applications: An Audit Study," American Economic Review, American Economic Association, vol. 106(5), pages 314-318, May.
    20. Manski, Charles F., 2000. "Identification problems and decisions under ambiguity: Empirical analysis of treatment response and normative analysis of treatment choice," Journal of Econometrics, Elsevier, vol. 95(2), pages 415-442, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Timothy B. Armstrong & Patrick Kline & Liyang Sun, 2023. "Adapting to Misspecification," Papers 2305.14265, arXiv.org, revised Aug 2024.
    2. Tenev, Nicholas H, 2020. "Social Connections and Racial Wage Inequality," SocArXiv vm82w, Center for Open Science.
    3. Di Addario, Sabrina & Kline, Patrick & Saggio, Raffaele & Sølvsten, Mikkel, 2023. "It ain’t where you’re from, it’s where you’re at: Hiring origins, firm heterogeneity, and wages," Journal of Econometrics, Elsevier, vol. 233(2), pages 340-374.
    4. Dominic Coey & Kenneth Hung, 2022. "Empirical Bayes Selection for Value Maximization," Papers 2210.03905, arXiv.org, revised Jan 2023.
    5. Almarina Gramozi & Theodore Palivos & Marios Zachariadis, 2020. "On the Degree and Consequences of Talent Misallocation for the United States," University of Cyprus Working Papers in Economics 09-2020, University of Cyprus Department of Economics.
    6. Danielle Li & Lindsey R. Raymond & Peter Bergman, 2020. "Hiring as Exploration," NBER Working Papers 27736, National Bureau of Economic Research, Inc.
    7. Nicolás Grau & Damián Vergara, "undated". "A Simple Test for Prejudice in Decision Processes: The Prediction-Based Outcome Test," Working Papers wp493, University of Chile, Department of Economics.
    8. Martinez de Lafuente, David, 2021. "Cultural Assimilation and Ethnic Discrimination: An Audit Study with Schools," Labour Economics, Elsevier, vol. 72(C).
    9. Sabrina Di Addario & Patrick Kline & Raffaele Saggio & Mikkel Soelvsten, 2022. "It ain't where you're from it's where you're at: firm effects, state dependence, and the gender wage gap," Temi di discussione (Economic working papers) 1374, Bank of Italy, Economic Research and International Relations Area.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kline, Patrick & Walters, Christopher, 2019. "Audits as Evidence: Experiments, Ensembles, and Enforcement," Institute for Research on Labor and Employment, Working Paper Series qt3z72m9kn, Institute of Industrial Relations, UC Berkeley.
    2. Patrick Kline & Evan K Rose & Christopher R Walters, 2022. "Systemic Discrimination Among Large U.S. Employers [“Teachers and Student Achievement in the Chicago Public High Schools,”]," The Quarterly Journal of Economics, Oxford University Press, vol. 137(4), pages 1963-2036.
    3. Patrick Kline & Evan K Rose & Christopher R Walters, 2023. "Systemic Discrimination Among Large U.S. Employers," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 137(4), pages 1963-2036.
    4. Mookerjee, Sulagna & Slichter, David, 2023. "Test scores, schools, and the geography of economic opportunity," Journal of Urban Economics, Elsevier, vol. 137(C).
    5. Timothy B. Armstrong & Michal Kolesár & Mikkel Plagborg‐Møller, 2022. "Robust Empirical Bayes Confidence Intervals," Econometrica, Econometric Society, vol. 90(6), pages 2567-2602, November.
    6. Raffaella Giacomini & Sokbae Lee & Silvia Sarpietro, 2023. "A Robust Method for Microforecasting and Estimation of Random Effects," Working Paper Series WP 2023-26, Federal Reserve Bank of Chicago.
    7. Joanna N. Lahey & Douglas R. Oxley, 2021. "Discrimination at the Intersection of Age, Race, and Gender: Evidence from an Eye‐Tracking Experiment," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(4), pages 1083-1119, September.
    8. Michael Geruso & Timothy J. Layton & Jacob Wallace, 2023. "What Difference Does a Health Plan Make? Evidence from Random Plan Assignment in Medicaid," American Economic Journal: Applied Economics, American Economic Association, vol. 15(3), pages 341-379, July.
    9. Baert, Stijn, 2015. "Hiring a Homosexual, Taking a Risk? A Lab Experiment on Employment Discrimination and Risk Aversion," IZA Discussion Papers 9536, Institute of Labor Economics (IZA).
    10. Christopher Walters, 2024. "Empirical Bayes Methods in Labor Economics," RF Berlin - CReAM Discussion Paper Series 2422, Rockwool Foundation Berlin (RF Berlin) - Centre for Research and Analysis of Migration (CReAM).
    11. Gaddis, S. Michael, 2018. "An Introduction to Audit Studies in the Social Sciences," SocArXiv e5hfc, Center for Open Science.
    12. Patrick Kline & Evan K. Rose & Christopher R. Walters, 2024. "A Discrimination Report Card," American Economic Review, American Economic Association, vol. 114(8), pages 2472-2525, August.
    13. Corrado Giulietti & Mirco Tonin & Michael Vlassopoulos, 2015. "Racial Discrimination in Local Public Services: A Field Experiment in the US," Working Papers 080, "Carlo F. Dondena" Centre for Research on Social Dynamics (DONDENA), Università Commerciale Luigi Bocconi.
    14. Hinrichs, Peter, 2021. "What kind of teachers are schools looking for? Evidence from a randomized field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 186(C), pages 395-411.
    15. Bär, Marlies & Bakx, Pieter & Wouterse, Bram & van Doorslaer, Eddy, 2022. "Estimating the health value added by nursing homes," Journal of Economic Behavior & Organization, Elsevier, vol. 203(C), pages 1-23.
    16. Godøy, Anna & Huitfeldt, Ingrid, 2020. "Regional variation in health care utilization and mortality," Journal of Health Economics, Elsevier, vol. 71(C).
    17. Jesse Rothstein & Till von Wachter, 2016. "Social Experiments in the Labor Market," NBER Working Papers 22585, National Bureau of Economic Research, Inc.
    18. Chowdhury, Shyamal & Ooi, Evarn & Slonim, Robert, 2017. "Racial discrimination and white first name adoption: a field experiment in the Australian labour market," Working Papers 2017-15, University of Sydney, School of Economics.
    19. Samuel Bazzi & Martin Fiszbein & Mesay Gebresilasse, 2020. "Frontier Culture: The Roots and Persistence of “Rugged Individualism” in the United States," Econometrica, Econometric Society, vol. 88(6), pages 2329-2368, November.
    20. Nicolás Grau & Damián Vergara, "undated". "A Simple Test for Prejudice in Decision Processes: The Prediction-Based Outcome Test," Working Papers wp493, University of Chile, Department of Economics.

    More about this item

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C44 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Operations Research; Statistical Decision Theory
    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • J7 - Labor and Demographic Economics - - Labor Discrimination
    • J71 - Labor and Demographic Economics - - Labor Discrimination - - - Hiring and Firing
    • K31 - Law and Economics - - Other Substantive Areas of Law - - - Labor Law
    • K42 - Law and Economics - - Legal Procedure, the Legal System, and Illegal Behavior - - - Illegal Behavior and the Enforcement of Law

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:26861. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.