Abstract
Uplift modeling is a causal learning technique that estimates subgroup-level treatment effects. It is commonly used in industry and elsewhere for tasks such as targeting ads. In a typical setting, uplift models can take thousands of features as inputs, which is costly and results in problems such as overfitting and poor model interpretability. Consequently, there is a need to select a subset of the most important features for modeling. However, traditional methods for doing feature selection are not fit for the task because they are designed for standard machine learning models whose target is importantly different from uplift models. To address this, this paper introduces a set of feature selection methods explicitly designed for uplift modeling, drawing inspiration from statistics and information theory. Empirical evaluations are conducted on the proposed methods on publicly available datasets, demonstrating the advantages of the proposed methods compared to traditional feature selection. We make the proposed methods publicly available as a part of the CausalML open-source package.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Megafon Uplift Competition (2021). https://ods.ai/tracks/df21-megafon/competitions/megafon-df21-comp/data
Athey, S., Imbens, G.: Recursive partitioning for heterogeneous causal effects, April 2015
Athey, S., Tibshirani, J., Wager, S.: Generalized random forests, October 2016
Bolón-Canedo, V., Sánchez-Maroño, N., Alonso-Betanzos, A.: A review of feature selection methods on synthetic data. Knowl. Inf. Syst. 34(3), 483–519 (2013)
Bommert, A., Sun, X., Bischl, B., Rahnenführer, J., Lang, M.: Benchmark for filter methods for feature selection in high-dimensional classification data. Comput. Stat. Data Anal. 143, 106839 (2020)
Chandrashekar, G., Sahin, F.: A survey on feature selection methods. Comput. Electr. Eng. 40(1), 16–28 (2014)
Chen, H., Harinen, T., Lee, J.Y., Yung, M., Zhao, Z.: CausalML: Python package for causal machine learning. arXiv preprint arXiv:2002.11631 (2020)
Chen, X., et al.: Imbalance-aware uplift modeling for observational data (2022)
Dahl, G.E., Sainath, T.N., Hinton, G.E.: Improving deep neural networks for LVCSR using rectified linear units and dropout. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 8609–8613. IEEE (2013)
Grimmer, J., Messing, S., Westwood, S.J.: Estimating heterogeneous treatment effects and the effects of heterogeneous treatments with ensemble methods. Polit. Anal. 25(4), 413–434 (2017)
Guelman, L., Guillén, M., Pérez-Marín, A.M.: Random forests for uplift modeling: an insurance customer retention case. In: Engemann, K.J., Gil-Lafuente, A.M., Merigó, J.M. (eds.) MS 2012. LNBIP, vol. 115, pp. 123–133. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-30433-0_13
Guelman, L., Guillén, M., Pérez-Marín, A.M.: Uplift random forests. Cybern. Syst. 46(3–4), 230–248 (2015)
Gutierrez, P., Gerardy, J.Y.: Causal inference and uplift modeling a review of the literature. In: JMLR: Workshop and Conference Proceedings, vol. 67 (2016)
Hansotia, B., Rukstales, B.: Incremental value modeling. Res. Council J. 16, 35–46 (2001)
Holland, P.W.: Statistics and causal inference. J. Am. Stat. Assoc. 81(396), 945–960 (1986)
Ke, G., et al.: LightGBM: a highly efficient gradient boosting decision tree. In: Advances in Neural Information Processing Systems, pp. 3146–3154 (2017)
Kohavi, R., Tang, D., Xu, Y.: Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing. Cambridge University Press, Cambridge (2020)
Künzel, S.R., Sekhon, J.S., Bickel, P.J., Yu, B.: Meta-learners for estimating heterogeneous treatment effects using machine learning, June 2017
Larsen, K.: Data exploration with weight of evidence and information value in R (2015)
Lo, V.S.: The true lift model: a novel data mining approach to response modeling in database marketing. ACM SIGKDD Explor. Newsl. 4(2), 78–86 (2002)
Mouloud, B., Olivier, G., Ghaith, K.: Adapting neural networks for uplift models. arXiv preprint arXiv:2011.00041 (2020)
Neyman, J.: Sur les applications de la théorie des probabilités aux experiences agricoles: Essai des principes. Roczniki Nauk Rolniczych 10, 1–51 (1923)
Nie, X., Wager, S.: Quasi-Oracle estimation of heterogeneous treatment effects, December 2017
Olaya, D., Coussement, K., Verbeke, W.: A survey and benchmarking study of multitreatment uplift modeling. Data Mining Knowl. Discov. 34(2), 273–308 (2020). https://doi.org/10.1007/s10618-019-00670-y
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Radcliffe, N.J., Surry, P.D.: Real-world uplift modelling with significance-based uplift trees. In: White Paper TR-2011-1, Stochastic Solutions, pp. 1–33 (2011)
Rubin, D.B.: Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66(5), 688–701 (1974)
Rubin, D.B.: Causal inference using potential outcomes: design, modeling, decisions. J. Am. Stat. Assoc. 100(469), 322–331 (2005)
Rzepakowski, P., Jaroszewicz, S.: Decision trees for uplift modeling with single and multiple treatments. Knowl. Inf. Syst. 32(2), 303–327 (2012)
Shortreed, S.M., Ertefaie, A.: Outcome-adaptive lasso: variable selection for causal inference. Biometrics 73(4), 1111–1122 (2017)
Sołtys, M., Jaroszewicz, S., Rzepakowski, P.: Ensemble methods for uplift modeling. Data Mining Knowl. Discov. 29(6), 1531–1559 (2014). https://doi.org/10.1007/s10618-014-0383-9
Tang, J., Alelyani, S., Liu, H.: Feature selection for classification: a review. Data Classif. Algorithms Appl. 37 (2014)
Teinemaa, I., Albert, J., Goldenberg, D.: Uplift modeling: from causal inference to personalization (2021)
Wager, S., Athey, S.: Estimation and inference of heterogeneous treatment effects using random forests, October 2015
Zaniewicz, L., Jaroszewicz, S.: Support vector machines for uplift modeling. In: 2013 IEEE 13th International Conference on Data Mining Workshops, pp. 131–138, December 2013
Zhao, Y., Fang, X., Simchi-Levi, D.: Uplift modeling with multiple treatments and general response types, May 2017
Zhao, Z.: Synthetic data for uplift modeling and heterogenous treatment effect with known counterfactuals and ITE, March 2022. https://doi.org/10.5281/zenodo.6342553
Zhao, Z., Harinen, T.: Uplift modeling for multiple treatments with cost optimization. arXiv preprint arXiv:1908.05372 (2019)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 IFIP International Federation for Information Processing
About this paper
Cite this paper
Zhao, Z., Zhang, Y., Harinen, T., Yung, M. (2022). Feature Selection Methods for Uplift Modeling and Heterogeneous Treatment Effect. In: Maglogiannis, I., Iliadis, L., Macintyre, J., Cortez, P. (eds) Artificial Intelligence Applications and Innovations. AIAI 2022. IFIP Advances in Information and Communication Technology, vol 647. Springer, Cham. https://doi.org/10.1007/978-3-031-08337-2_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-08337-2_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08336-5
Online ISBN: 978-3-031-08337-2
eBook Packages: Computer ScienceComputer Science (R0)