Generalized Synthetic Control for TestOps at ABI: Models, Algorithms, and Infrastructure

Luis Costa (), Vivek F. Farias (), Patricio Foncea (), Jingyuan (Donna) Gan (), Ayush Garg (), Ivo Rosa Montenegro (), Kumarjit Pathak (), Tianyi Peng () and Dusan Popovic ()
Additional contact information
Luis Costa: Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139
Vivek F. Farias: Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139
Patricio Foncea: Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139
Jingyuan (Donna) Gan: Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139
Ayush Garg: Anheuser-Busch InBev, St. Louis, Missouri 63118
Ivo Rosa Montenegro: Anheuser-Busch InBev, St. Louis, Missouri 63118
Kumarjit Pathak: Anheuser-Busch InBev, St. Louis, Missouri 63118
Tianyi Peng: Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139
Dusan Popovic: Anheuser-Busch InBev, St. Louis, Missouri 63118

Interfaces, 2023, vol. 53, issue 5, 336-349

Abstract: We describe a novel optimization-based approach—generalized synthetic control (GSC)—in which we learn from experiments conducted in a physical retail environment. GSC solves a long-standing problem of learning from experiments conducted in this environment when treatment effects are small, the environment is extremely noisy and nonstationary, and interference and adherence problems are commonplace. The utilization of GSC has demonstrated a remarkable increase in statistical power, approximately one hundredfold (100×) higher than conventional inferential methods. This innovative approach forms the basis of TestOps, a pioneering large-scale experimentation platform designed specifically for physical retailers. TestOps was developed and has been broadly implemented as part of a collaboration between Anheuser Busch Inbev (ABI) and a team of operations researchers and data engineers from the Massachusetts Institute of Technology. TestOps currently runs physical experiments impacting approximately 135 million USD in revenue every month and routinely identifies innovations that result in a 1%–2% increase in sales volume. The vast majority of these innovations would have remained unidentified had we not developed our novel approach to inference. Prior to our implementation, statistically significant conclusions could be drawn on only ∼6% of all experiments, a fraction that has now increased by 10-fold. Given its success, TestOps is being rolled out globally at ABI, driving significant revenue growth and enabling the extraction of valuable insights from large-scale physical experiments.

Keywords: testing platform; retail experiments and innovation; panel data; causal inference (search for similar items in EconPapers)
Date: 2023
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/inte.2023.0028 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:orinte:v:53:y:2023:i:5:p:336-349

Access Statistics for this article

More articles in Interfaces from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().