More Web Proxy on the site http://driver.im/

article

The Sense and Non-Sense of Holdout Sample Validation in the Presence of Endogeneity

Authors:

Dominik Papies,

Harald J. van HeerdeAuthors Info & Claims

Marketing Science, Volume 30, Issue 6

Pages 1115 - 1122

Published: 01 November 2011 Publication History

Abstract

Market response models based on field-generated data need to address potential endogeneity in the regressors to obtain consistent parameter estimates. Another requirement is that market response models predict well in a holdout sample. With both requirements combined, it may seem reasonable to subject an endogeneity-corrected model to a holdout prediction task, and this is quite common in the academic marketing literature. One may be inclined to expect that the consistent parameter estimates obtained via instrumental variables (IV) estimation predict better than the biased ordinary least squares (OLS) estimates. This paper shows that this expectation is incorrect. That is, if the holdout sample is similar to the estimation sample so that the regressors are endogenous in both samples, holdout sample validation favors regression estimates that are not corrected for endogeneity (i.e., OLS) over estimates that are corrected for endogeneity (i.e., IV estimation). We also discuss ways in which holdout samples may be used sensibly in the presence of endogeneity. A key takeaway is that if consistent parameter estimates are the primary model objective, the model should be validated with an exogenous (rather than endogenous) holdout sample.

References

[1]

Allenby, G. M., "Cross-validation, the Bayes theorem, and small-sample bias," J. Bus. Econom. Statist., v8, pp. 171-178, 1990.

[2]

Allenby, G. M. and Rossi, P. E., "Marketing models of consumer heterogeneity," J. Econometrics, v89, pp. 57-78, 1999.

[3]

Andrews, R. L. and Currim, I. S., "Multi-stage purchase decision models: Accommodating response heterogeneity, common demand shocks, and endogeneity using disaggregate data," Internat. J. Res. Marketing, v26, pp. 197-206, 2009.

[4]

Angrist, J. D. and Imbens, G. W., "Two-stage least squares estimation of average causal effects in models with variable treatment intensity," J. Amer. Statist. Assoc., v90, pp. 431-442, 1995.

[5]

Berry, S. T., "Estimating discrete-choice models of product differentiation," RAND J. Econom., v25, pp. 242-262, 1994.

[6]

Besanko, D., Gupta, S. and Jain, D., "Logit demand estimation under competitive pricing behavior: An equilibrium framework," Management Sci., v44, pp. 1533-1547, 1998.

Digital Library

[7]

Bijmolt, T. H. A., van Heerde, H. J. and Pieters, R. G. M., "New empirical generalizations on the determinants of price elasticity," J. Marketing Res., v42, pp. 141-156, 2005.

[8]

Bronnenberg, B. J. and Mahajan, V., "Unobserved retailer behavior in multimarket data: Joint spatial dependence in market shares and promotion variables," Marketing Sci., v20, pp. 284-299, 2001.

Digital Library

[9]

Chintagunta, P. K., "Endogeneity and heterogeneity in a probit demand model: Estimation using aggregate data," Marketing Sci., v20, pp. 442-456, 2001.

Digital Library

[10]

Cooil, B., Winer, R. S. and Rados, D. L., "Cross-validation for prediction," J. Marketing Res., v24, pp. 271-279, 1987.

[11]

Davidson, R. and MacKinnon, J. G., Estimation and Inference in Econometrics, Oxford University Press, Oxford, UK, 1993.

[12]

Drèze, X., Nisol, P. and Vilcassim, N. J., "Do promotions increase store expenditures? A descriptive study of household shopping behavior," Quant. Marketing Econom., v2, pp. 59-92, 2004.

[13]

Franses, P. H., "On the use of econometric models for policy simulation in marketing," J. Marketing Res., v42, pp. 4-14, 2005.

[14]

Greene, W. H., Econometric Analysis, Prentice Hall, Upper Saddle River, NJ, 2003.

[15]

Hagerty, M. R. and Srinivasan, S., "Comparing the predictive powers of alternative multiple regression models," Psychometrika, v56, pp. 77-85, 1991.

[16]

Hanssens, D. M., Parsons, L. J. and Schultz, R. L., Market Response Models, Econometric and Time Series Analysis, Kluwer Academic Publishers, Norwell, MA, 2001.

[17]

Hoch, S. J., Drèze, X. and Purk, M. E., "EDLP, hi-lo, and margin arithmetic," J. Marketing, v58, pp. 16-27, 1994.

[18]

Kennedy, P., A Guide to Econometrics, MIT Press, Cambridge, MA, 1992.

[19]

Kleibergen, F. and Zivot, E., "Bayesian and classical approaches to instrumental variables regression," J. Econometrics, v114, pp. 29-72, 2003.

[20]

Leeflang, P. S. H., Wittink, D. R., Wedel, M. and Naert, P. A., Building Models for Marketing Decisions, Kluwer Academic Publishers, Dordrecht, The Netherlands, 2000.

[21]

Leenheer, J., van Heerde, H. J., Bijmolt, T. H. A. and Smidts, A., "Do loyalty programs really enhance behavioral loyalty? An empirical analysis accounting for self-selecting members," Internat. J. Res. Marketing, v24, pp. 31-47, 2007.

[22]

Neslin, S. A., "A market response model for coupon promotions," Marketing Sci., v9, pp. 125-145, 1990.

Digital Library

[23]

Petrin, A. and Train, K., "A control function approach to endogeneity in consumer choice models," J. Marketing Res., v47, pp. 3-13, 2010.

[24]

Picard, R. R. and Cook, R. D., "Cross-validation of regression models," J. Amer. Statist. Assoc., v79, pp. 575-583, 1984.

[25]

Rossi, P. E., Allenby, G. M. and McCulloch, R., Bayesian Statistics and Marketing, John Wiley & Sons, Chichester, UK, 2005.

[26]

Shmueli, G., "To explain or to predict?," Statist. Sci., v25, pp. 289-310, 2010.

[27]

Shugan, S. M., "Endogeneity in marketing decision models," Marketing Sci., v23, pp. 1-3, 2004.

Digital Library

[28]

Shugan, S. M., "Relevancy is robust prediction, not alleged realism," Marketing Sci., v28, pp. 991-998, 2009.

Digital Library

[29]

Steckel, J. H. and Vanhonacker, W. R., "Cross-validating regression models in marketing research," Marketing Sci., v12, pp. 415-427, 1993.

Digital Library

[30]

Tsang, E. W. K., Assumptions, explanation, and prediction in marketing science: “It's the findings, stupid, not the assumptions”, Marketing Sci., v28, pp. 986-990, 2009.

Digital Library

[31]

van Dijk, A., van Heerde, H. J., Leeflang, P. S. H. and Wittink, D. R., "Similarity-based spatial methods to estimate shelf space elasticities," Quant. Marketing Econom., v2, pp. 257-277, 2004.

[32]

van Heerde, H. J., Dekimpe, M. G. and Putsis, W. P., "Marketing models and the Lucas critique," J. Marketing Res., v42, pp. 15-21, 2005.

[33]

Verbeek, M., A Guide to Modern Econometrics, John Wiley & Sons, Hoboken, NJ, 2008.

[34]

Villas-Boas, J. M. and Winer, R. S., "Endogeneity in brand choice models," Management Sci., v45, pp. 1324-1338, 1999.

Digital Library

[35]

Wooldridge, J. M., Econometric Analysis of Cross Section and Panel Data, MIT Press, Cambridge, MA, 2002.

Cited By

Jagabathula SRusmevichientong P(2019)A Nonparametric Joint Assortment and Price Choice ModelManagement Science10.1287/mnsc.2016.249163:9(3128-3145)Online publication date: 5-Jan-2019
https://dl.acm.org/doi/10.1287/mnsc.2016.2491
Phillips RŞimşek Avan Ryzin G(2019)The Effectiveness of Field Price DiscretionManagement Science10.1287/mnsc.2014.208461:8(1741-1759)Online publication date: 5-Jan-2019
https://dl.acm.org/doi/10.1287/mnsc.2014.2084
Dong XJanakiraman RXie Y(2019)The Effect of Survey Participation on Consumer BehaviorMarketing Science10.1287/mksc.2014.085233:4(567-585)Online publication date: 4-Jan-2019
https://dl.acm.org/doi/10.1287/mksc.2014.0852

The Sense and Non-Sense of Holdout Sample Validation in the Presence of Endogeneity
1. Applied computing

Recommendations

The Sense and Non-Sense of Holdout Sample Validation in the Presence of Endogeneity

Market response models based on field-generated data need to address potential endogeneity in the regressors to obtain consistent parameter estimates. Another requirement is that market response models predict well in a holdout sample. With both ...
The Sense and Non-Sense of Holdout Sample Validation in the Presence of Endogeneity

Market response models based on field-generated data need to address potential endogeneity in the regressors to obtain consistent parameter estimates. Another requirement is that market response models predict well in a holdout sample. With both ...
Beyond the Endogeneity Bias: The Effect of Unmeasured Brand Characteristics on Household-Level Brand Choice Models

We investigate the role of potential weekly brand-specific characteristics that influence consumer choices, but are unobserved or unmeasurable by the researcher. We use an empirical approach, based on the estimation methods used for standard random ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Marketing Science

Marketing Science Volume 30, Issue 6

11-12 2011

152 pages

ISSN:1526-548X

Issue’s Table of Contents

Publisher

INFORMS

Linthicum, MD, United States

Publication History

Published: 01 November 2011

Accepted: 05 June 2011

Received: 25 May 2010

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jagabathula SRusmevichientong P(2019)A Nonparametric Joint Assortment and Price Choice ModelManagement Science10.1287/mnsc.2016.249163:9(3128-3145)Online publication date: 5-Jan-2019
https://dl.acm.org/doi/10.1287/mnsc.2016.2491
Phillips RŞimşek Avan Ryzin G(2019)The Effectiveness of Field Price DiscretionManagement Science10.1287/mnsc.2014.208461:8(1741-1759)Online publication date: 5-Jan-2019
https://dl.acm.org/doi/10.1287/mnsc.2014.2084
Dong XJanakiraman RXie Y(2019)The Effect of Survey Participation on Consumer BehaviorMarketing Science10.1287/mksc.2014.085233:4(567-585)Online publication date: 4-Jan-2019
https://dl.acm.org/doi/10.1287/mksc.2014.0852

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents