Abstract
Uplift Modeling measures the impact of an action (marketing, medical treatment) on a person’s behavior. This allows the selection of the subgroup of persons for which the effect of the action will be most noteworthy. Uplift estimation is based on groups of people who have received different treatments. These groups are assumed to be equivalent. However, in practice, we observe biases between these groups. We propose in this paper a protocol to evaluate and study the impact of the Non-Random Assignment bias (NRA) on the performance of the main uplift methods. Then we present a weighting method to reduce the effect of the NRA bias. Experimental results show that our bias reduction method significantly improves the performance of uplift models under NRA bias.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)
Devriendt, F., Guns, T., Verbeke, W.: Learning to rank for uplift modeling (2020)
Diemert, E.: A large scale benchmark for uplift modeling (2018)
Gerber, A.S., Green, D.P., Larimer, C.W.: Social pressure and voter turnout: evidence from a large-scale field experiment. Am. Polit. Sci. Rev. 102(1), 33–48 (2008)
Guelman, L.A.: Optimal personalized treatment learning models with insurance applications (2015)
Gutierrez, P., Gérardy, J.Y.: Causal inference and uplift modelling: a review of the literature. In: PAPIs (2016)
Jacob, D.: CATE meets ML - the conditional average treatment effect and machine learning (2021)
Jaskowski, M., Jaroszewicz, S.: Uplift modeling for clinical trial data (2012)
Kennedy, E.H.: Optimal doubly robust estimation of heterogeneous causal effects (2020)
Kouw, W.M., Loog, M.: An introduction to domain adaptation and transfer learning (2019)
Li, C., et al.: Reinforcement learning for uplift modeling (2019)
Lo, V.S.Y., Pachamanova, D.A.: From predictive uplift modeling to prescriptive uplift analytics: a practical approach to treatment optimization while accounting for estimation risk. J. Mark. Anal. 3, 79–95 (2015). https://doi.org/10.1057/jma.2015.5
Lopez-Paz, D., Oquab, M.: Revisiting classifier two-sample tests (2018)
Lunceford, J.K., Davidian, M.: Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Stat. Med. 23(19), 2937–2960 (2004)
Olaya, D., Coussement, K., Verbeke, W.: A survey and benchmarking study of multitreatment uplift modeling. Data Min. Knowl. Discov. 34(2), 273–308 (2020). https://doi.org/10.1007/s10618-019-00670-y
Radcliffe, N.: Using control groups to target on predicted lift: building and assessing uplift model. Direct Mark. Anal. J. 14–21 (2007)
Radcliffe, N.J., Surry, P.D.: Differential response analysis: modeling true responses by isolating the effect of a single action (1999)
Radcliffe, N.J., Surry, P.D.: Real-world uplift modelling with significance-based uplift trees (2012)
Rafla, M., Voisine, N., Crémilleux, B.: Supplementary material. https://github.com/MinaWagdi/UpliftEvaluation_NRA/
Rubin, D.B.: Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66, 688–701 (1974)
Rubin, D.B.: Using propensity scores to help design observational studies: application to the tobacco litigation. Health Serv. Outcomes Res. Methodol. 2(3), 169–188 (2001). https://doi.org/10.1023/A:1020363010465
Rzepakowski, P., Jaroszewicz, S.: Decision trees for uplift modeling with single and multiple treatments. Knowl. Inf. Syst. 32, 303–327 (2011). https://doi.org/10.1007/s10115-011-0434-0
Wilcoxon, F.: Individual comparisons by ranking methods. Biom. Bull. 1(6), 80–83 (1945). http://www.jstor.org/stable/3001968
Zhao, Y., Fang, X., Simchi-Levi, D.: Uplift modeling with multiple treatments and general response types. In: Proceedings of the 2017 SIAM International Conference on Data Mining, pp. 588–596. SIAM (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Rafla, M., Voisine, N., Crémilleux, B. (2022). Evaluation of Uplift Models with Non-Random Assignment Bias. In: Bouadi, T., Fromont, E., Hüllermeier, E. (eds) Advances in Intelligent Data Analysis XX. IDA 2022. Lecture Notes in Computer Science, vol 13205. Springer, Cham. https://doi.org/10.1007/978-3-031-01333-1_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-01333-1_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-01332-4
Online ISBN: 978-3-031-01333-1
eBook Packages: Computer ScienceComputer Science (R0)