- (2017). Many variables have sizeable and statistically significant coefficients that create the selection into the treatment. Figure B.1 shows the distribution of the propensity score after the manipulations described in Section 5.2.1. Figure B.1 shows that our setting does not creates problems due to extreme propensity scores or no overlap.
Paper not yet in RePEc: Add citation now
Abadie, A. (2005). Semiparametric difference-in-differences estimators. Review of Economic Studies, 72(1), 1–19. doi: 10.1111/0034-6527.00321 Abrevaya, J., Hsu, Y.-C., & Lieli, R. P. (2015). Estimating conditional average treatment effects. Journal of Business & Economic Statistics, 33(4), 485–505. doi: 975555 Andini, M., Ciani, E., de Blasio, G., D’Ignazio, A., & Salvestrini, V. (2018). Targeting with machine learning: An application to a tax rebate program in Italy. Journal of Economic Behavior & Organization, 156, 86–102.
- Ascarza, E. (2018). Retention futility: Targeting high risk customers might be ineffective. Journal of Marketing Research, 55(1), 80–98.
Paper not yet in RePEc: Add citation now
Athey, S. (2018). The impact of machine learning on economics. In A. K. Agrawal, J. Gans, & A. Goldfarb (Eds.), The economics of artificial intelligence: An agenda. University of Chicago Press.
- Athey, S., & Imbens, G. W. (2016). Recursive partitioning for heterogeneous causal effects. Proceedings of the National Academy of Sciences, 113(27), 7353–7360.
Paper not yet in RePEc: Add citation now
Athey, S., & Imbens, G. W. (2017). The state of applied econometrics: causality and policy evaluation. Journal of Economic Perspectives, 31(2), 3–32.
Athey, S., Tibshirani, J., & Wager, S. (2018). Generalized random forests. Annals of Statistics, forthcoming.
Behncke, S., Frölich, M., & Lechner, M. (2010a). A caseworker like me - Does the similarity between the unemployed and their caseworkers increase job placements? The Economic Journal, 120(549), 1430–1459.
Behncke, S., Frölich, M., & Lechner, M. (2010b). Unemployed and their caseworkers: Should they be friends or foes? Journal of the Royal Statistical Society: Series A (Statistics in Society), 173(1), 67–92.
Bell, S. H., & Orr, L. L. (2002). Screening (and creaming?) applicants to job training programs: The AFDC homemaker - home health aide demonstrations. Labour Economics, 9(2), 279–301.
Belloni, A., Chernozhukov, V., & Hansen, C. (2014a). High-dimensional methods and inference on structural and treatment effects. Journal of Economic Perspectives, 28(2), 29–50.
Belloni, A., Chernozhukov, V., & Hansen, C. (2014b). Inference on treatment effects after selection among high-dimensional controls. Review of Economic Studies, 81(2), 608–650.
Belloni, A., Chernozhukov, V., Fernández-Val, I., & Hansen, C. (2017). Program evaluation and causal inference with high-dimensional data. Econometrica, 85(1), 233–298.
- Bertrand, M., Crépon, B., Marguerie, A., & Premand, P. (2017). Contemporaneous and post-program impacts of a public works program: Evidence from Côte d’Ivoire.
Paper not yet in RePEc: Add citation now
- Breiman, L., Friedman, J., Stone, C. J., & Olshen, R. A. (1984). Classification and regression trees. CRC press.
Paper not yet in RePEc: Add citation now
- Card, D., Kluve, J., & Weber, A. (2017). What works? A meta analysis of recent active labor market program evaluations. Journal of the European Economic Association, 16(3), 894–931.
Paper not yet in RePEc: Add citation now
Chen, S., Tian, L., Cai, T., & Yu, M. (2017). A general statistical framework for subgroup identification and comparative treatment scoring. Biometrics, 73(4), 1199–1209.
- Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., & Newey, W. (2017). Double/Debiased/Neyman machine learning of treatment effects. American Economic Review Papers and Proceedings, 107(5), 261–265.
Paper not yet in RePEc: Add citation now
Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., & Robins, J. (2018). Double/Debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1), C1-C68.
Davis, J. M., & Heller, S. B. (2017). Using causal forests to predict treatment heterogeneity: An application to summer jobs. American Economic Review, 107(5), 546–550.
Djebbari, H., & Smith, J. (2008). Heterogeneous impacts in PROGRESA. Journal of Econometrics, 145(1), 64–80.
Econometric Theory, 21(1), 21–59. doi: 10.1017/S0266466605050036 Leeb, H., & Pötscher, B. M. (2008). Sparse estimators and the oracle property, or the return of Hodges’ estimator. Journal of Econometrics, 142(1), 201–211. doi: 10.1016/J.JECONOM.2007.05.017 McFowland, E., Somanchi, S., & Neill, D. B. (2018). Efficient discovery of heterogeneous treatment effects in randomized experiments via anomalous pattern detection.
Farrell, M. H. (2015). Robust inference on average treatment effects with possibly more covariates than observations. Journal of Econometrics, 189(1), 1–23.
Friedman, J., Hastie, T., & Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1), 1–22.
Gerfin, M., & Lechner, M. (2002). A microeconometric evaluation of the active labour market policy in Switzerland. Economic Journal, 112(482), 854–893.
Grimmer, J., Messing, S., & Westwood, S. J. (2017). Estimating heterogeneous treatment effects and the effects of heterogeneous treatments with ensemble methods. Political Analysis, 25(4), 413–434.
- Hahn, P. R., Murray, J. S., & Carvalho, C. (2017). Bayesian regression tree models for causal inference: Regularization, confounding, and heterogeneous effects. (2011), 1–22. Retrieved from http://arxiv.org/abs/1706.09523 Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning -Data mining, inference, and prediction (2nd ed.). Springer, New York.
Paper not yet in RePEc: Add citation now
- Hill, J. L. (2011). Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics, 20(1), 217–240.
Paper not yet in RePEc: Add citation now
Hirano, K., Imbens, G. W., & Ridder, G. (2003). Efficient estimation of average treatment effects using the estimated propensity score. Econometrica, 71(4), 1161–1189.
- Horvitz, D. G., & Thompson, D. J. (1952). A generalization of sampling without replacement from a finite universe. Journal of the American statistical Association, 47(260), 663–685.
Paper not yet in RePEc: Add citation now
Huber, M., Lechner, M., & Mellace, G. (2017). Why do tougher caseworkers increase employment? The role of program assignment as a causal mechanism. Review of Economics and Statistics, 99(1), 180–183.
Huber, M., Lechner, M., & Wunsch, C. (2013). The performance of estimators based on the propensity score. Journal of Econometrics, 175(1), 1–21.
- Imai, K., & Ratkovic, M. (2013). Estimating treatment effect heterogeneity in randomized program evaluation. Annals of Applied Statistics, 7(1), 443–470.
Paper not yet in RePEc: Add citation now
Imbens, G. W., & Wooldridge, J. M. (2009). Recent developments in the econometrics of program evaluation. Journal of Economic Literature, 47(1), 5–86.
- In total, 238,902 persons registered as being unemployed in 2003. We only consider the first unemployment registration per individual in 2003. Each registered unemployed person is assigned to a caseworker. In most cases, the same caseworker is responsible for the entire unemployment spell of her/his client. If this is not the case, we focus on the first caseworker to avoid concerns about (rare) endogenous caseworker changes (see Behncke et al., 2010a). We only consider unemployed aged between 24 and 55 years who receive unemployment insurance benefits. We omitted unemployed persons who apply for disability insurance benefits, when the responsible caseworker is not clearly defined, or when their caseworkers did not answer the questionnaire (the response rate is 84%). We drop unemployed foreigners with a residence permit that is valid for less than a year.
Paper not yet in RePEc: Add citation now
- Johansson, F., Shalit, U., & Sontag, D. (2016). Learning representations for counterfactual inference. In International conference on machine learning (pp. 3020–3029).
Paper not yet in RePEc: Add citation now
- Kang, J. D. Y., & Schafer, J. L. (2007). Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data. Statistical Science, 22(4), 523–539.
Paper not yet in RePEc: Add citation now
Knaus, M. C., Lechner, M., & Strittmatter, A. (2017). Heterogeneous employment effects of job search programmes: A machine learning approach. Retrieved from http://arxiv.org/abs/1709.10279 Künzel, S. R., Sekhon, J. S., Bickel, P. J., & Yu, B. (2017). Meta-learners for estimating heterogeneous treatment effects using machine learning. Retrieved from http:// arxiv.org/abs/1706.03461 Lalive, R., van Ours, J., & Zweimüller, J. (2008). The impact of active labor market programs on the duration of unemployment. Economic Journal, 118(525), 235–257.
- Lechner, M. (2018). Penalized causal forests for estimating heterogeneous causal effects. Working Paper.
Paper not yet in RePEc: Add citation now
Lechner, M., & Smith, J. (2007). What is the value added by caseworkers? Labour Economics, 14(2), 135–151.
Lechner, M., & Strittmatter, A. (2017). Practical procedures to deal with common support problems in matching estimation. Econometric Reviews, forthcoming.
Lechner, M., & Wunsch, C. (2013). Sensitivity of matching-based program evaluations to the availability of control variables. Labour Economics, 21, 111–121.
Lee, S., Okui, R., & Whang, Y.-J. (2017). Doubly robust uniform confidence band for the conditional average treatment effect function. Journal of Applied Econometrics.
Leeb, H., & Pötscher, B. M. (2005). Model selection and inference: Facts and fiction.
- Nie, X., & Wager, S. (2017). Quasi-oracle estimation of heterogeneous treatment effects.
Paper not yet in RePEc: Add citation now
- Qian, M., & Murphy, S. A. (2011). Performance guarantees for individualized treatment rules. Annals of statistics, 39(2), 1180.
Paper not yet in RePEc: Add citation now
- Retrieved from http://arxiv.org/abs/1712.04912 Oprescu, M., Syrgkanis, V., & Wu, Z. S. (2018). Orthogonal random forest for heterogeneous treatment effect estimation. Retrieved from http://arxiv.org/abs/1806.03467 Powers, S., Qian, J., Jung, K., Schuler, A., Shah, N. H., Hastie, T., & Tibshirani, R. (2018). Some methods for heterogeneous treatment effect estimation in high dimensions.
Paper not yet in RePEc: Add citation now
Retrieved from http://arxiv.org/abs/1803.09159 Murphy, S. (2003). Optimal dynamic treatment regimes. Journal of the Royal Statistical Society. Series B (Statistical Methodology), 65(2), 331–366.
- Robins, J. M. (2004). Optimal structural nested models for optimal sequential decisions. , 189–326.
Paper not yet in RePEc: Add citation now
- Robins, J. M., & Rotnitzky, A. (1995). Semiparametric efficiency in multivariate regression Models with missing data. Journal of the American Statistical Association, 90(429), 122–129.
Paper not yet in RePEc: Add citation now
Robinson, P. M. (1988). Root-n-consistent semiparametric regression. Econometrica, 56(4), 931.
- Rolling, C. A., Velez, D., & Yang, Y. (2018). Combining estimates of conditional treatment effects. Econometric Theory, forthcoming.
Paper not yet in RePEc: Add citation now
- Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701.
Paper not yet in RePEc: Add citation now
- Schuler, A., Baiocchi, M., Tibshirani, R., & Shah, N. (2018). A comparison of methods for model selection when estimating individual treatment effects. Retrieved from http://arxiv.org/abs/1804.05146 Schwab, P., Linhardt, L., & Karlen, W. (2018). Perfect match: A simple method for learning representations for counterfactual inference with neural networks. Retrieved from http://arxiv.org/abs/1810.00656 Shalit, U., Johansson, F. D., & Sontag, D. (2016). Estimating individual treatment effect: Generalization bounds and algorithms. Retrieved from http://arxiv.org/abs/ 1606.03976 Signorovitch, J. E. (2007). Identifying informative biological markers in high-dimensional genomic data and clinical trials (Unpublished doctoral dissertation). PhD thesis, Harvard University.
Paper not yet in RePEc: Add citation now
Strittmatter, A. (2018). What is the value added by causal machine learning methods in a welfare experiment evaluation? Working Paper.
- Su, X., Tsai, C.-L., Wang, H., Nickerson, D. M., & Li, B. (2009). Subgroup analysis via recursive partitioning. Journal of Machine Learning Research, 10(Feb), 141–158.
Paper not yet in RePEc: Add citation now
- Table D.39: Average computation time of one replication in seconds ITE0 w/o noise ITE1 w/ noise ITE2 w/ noise (1) (2) (3) 1000 observations Random Forest: Infeasible 1.1 2.6 2.8 Conditional mean regression 4.0 4.1 4.0 MOM IPW 5.2 5.1 5.2 MOM DR 8.2 8.2 8.1 Causal Forest 3.9 3.9 3.9 Causal Forest with local centering 5.2 5.2 5.2 Lasso: Infeasible - 26.8 29.5 Conditional mean regression 7.6 7.7 7.7 MOM IPW 12.4 12.3 12.3 MOM DR 17.9 17.9 17.9 MCM 11.3 11.3 11.3 MCM with efficiency augmentation 17.4 17.4 17.4 R-learning 17.4 17.4 17.4 4000 observations Random Forest: Infeasible 3.2 8.6 9.7 Conditional mean regression 11.2 11.4 11.3 MOM IPW 17.0 17.0 17.0 MOM DR 32.4 33.1 32.8 Causal Forest 11.6 11.8 11.7 Causal Forest with local centering 18.3 18.3 18.3 Lasso: Infeasible - 40.5 46.4 Conditional mean regression 24.2 24.1 24.2 MOM IPW 49.6 49.4 49.2 MOM DR 68.0 67.9 67.9 MCM 51.8 51.7 51.5 MCM with efficiency augmentation 67.4 67.2 67.2 R-learning 67.4 67.2 67.3
Paper not yet in RePEc: Add citation now
Taddy, M., Gardner, M., Chen, L., & Draper, D. (2016). A nonparametric Bayesian analysis of heterogeneous treatment effects in digital experimentation. Journal of Business & Economic Statistics, 34(4), 661–672.
- This is the standard data used for many Swiss ALMP evaluations (e.g., Gerfin & Lechner, 2002; Lalive, van Ours, & Zweimüller, 2008; Lechner & Smith, 2007). We observe (among others) residence status, qualification, education, language skills, employment history, profession, job position, industry of last job, and desired occupation and industry. The administrative data is linked with regional labour market characteristics, such as the population size of municipalities and the cantonal unemployment rate. The availability of extensive caseworker information and their subjective assessment of the employability of their clients distinguishes our data. Swiss caseworkers employed in the period 2003-2004 were surveyed based on a written questionnaire in December 2004 (see Behncke et al., 2010a, 2010b). The questionnaire contained questions about aims and strategies of the caseworker and the regional employment agency.
Paper not yet in RePEc: Add citation now
Tian, L., Alizadeh, A. A., Gentles, A. J., & Tibshirani, R. (2014). A simple method for estimating interactions between a treatment and a large number of covariates.
- Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society. Series B (Methodological), 58, 267–288.
Paper not yet in RePEc: Add citation now
Waernbaum, I., & Pazzagli, L. (2017). Model misspecification and bias for inverse probability weighting and doubly robust estimators. , 1–25. Retrieved from http:// arxiv.org/abs/1711.09388 Wager, S., & Athey, S. (2018). Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113(523), 1228–1242.
- Washington D.C. Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.
Paper not yet in RePEc: Add citation now
- Wendling, T., Jung, K., Callahan, A., Schuler, A., Shah, N. H., & Gallego, B. (2018). Comparing methods for estimation of heterogeneous treatment effects using observational data from health care databases. Statistics in Medicine.
Paper not yet in RePEc: Add citation now
Zhang, B., Tsiatis, A. A., Laber, E. B., & Davidian, M. (2012). A robust method for estimating optimal treatment regimes. Biometrics, 68(4), 1010–1018.
- Zhao, Q., Small, D. S., & Ertefaie, A. (2017). Selective inference for effect modification via the lasso. Retrieved from http://arxiv.org/abs/1705.08020 Appendices A Data A.1 Dataset The data we use includes all individuals who are registered as unemployed at Swiss regional employment agencies in the year 2003. The data contains rich information from different unemployment insurance databases (AVAM/ASAL) and social security records (AHV).
Paper not yet in RePEc: Add citation now