[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3184558.3186915acmotherconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
poster
Free access

Handling Confounding for Realistic Off-Policy Evaluation

Published: 23 April 2018 Publication History

Abstract

Inverse Propensity Score estimator (IPS) is a basic, unbiased, off-policy evaluation technique to measure the impact of a user-interactive system without serving live traffic. We present our work on applying IPS to real-world settings by addressing some practical challenges, thereby enabling successful policy evaluation. In particular, we show that off-policy evaluation can be impossible in the absence of a complete context and we describe a systematic way of defining the context.

References

[1]
J. Langford, A. Strehl, and J. Wortman. 2008. Exploration Scavenging. In ICML.
[2]
L. Li, S. Chen, J. Kleban, and A. Gupta. 2015 a. Counterfactual Estimation and Optimization of Click Metrics in Search Engines: A Case Study WWW '15 Companion.
[3]
L. Li, J. Young Kim, and I. Zitouni. 2015 b. Toward Predicting the Outcome of an A/B Experiment for Search Relevance WSDM.
[4]
A. Strehl, J. Langford, L. Li, and S. Kakade. 2010. Learning from Logged Implicit Exploration Data. NIPS.
[5]
A. Swaminathan and T. Joachims. 2015. The self-normalized estimator for counterfactual learning NIPS.
[6]
J. Tang, A. Salem, and L. Huan. 2014. Feature selection for classification: A review. In Data Classification: Algorithms and Applications.

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences
WWW '18: Companion Proceedings of the The Web Conference 2018
April 2018
2023 pages
ISBN:9781450356404
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

  • IW3C2: International World Wide Web Conference Committee

In-Cooperation

Publisher

International World Wide Web Conferences Steering Committee

Republic and Canton of Geneva, Switzerland

Publication History

Published: 23 April 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. a/b tests
  2. action propensity estimation
  3. confounding
  4. inverse propensity score
  5. off-policy evaluation

Qualifiers

  • Poster

Conference

WWW '18
Sponsor:
  • IW3C2
WWW '18: The Web Conference 2018
April 23 - 27, 2018
Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 313
    Total Downloads
  • Downloads (Last 12 months)53
  • Downloads (Last 6 weeks)10
Reflects downloads up to 24 Dec 2024

Other Metrics

Citations

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media