Unbiased Learning from Biased User Feedback with Thorsten Joachims - TWiML Talk #207

Counterfactual Testing

Counterfactual A/B testing offers a promising alternative to traditional online A/B tests, allowing for faster and more efficient evaluation of new policies. Thorsten Joachims explains that by using existing log data and logging the propensity of actions, developers can perform offline tests without the need for new data collection 1. This method not only speeds up the development cycle but also reduces the reliance on expensive and time-consuming online tests. Joachims highlights the potential of this approach:

Many of the online A/B tests that you're doing today could probably be replaced by these types of offline counterfactual A/B tests.

--- Thorsten Joachims

This technique is gaining traction in both academia and industry, with companies like Microsoft and Spotify exploring its applications 1.

   

Policy Evaluation

Evaluating new policies using existing log data requires specific conditions, such as stochastic logging policies, to ensure unbiased results. Thorsten Joachims emphasizes the importance of inverse propensity scoring, which allows for unbiased estimates of a policy's performance by re-weighting distributions based on past actions 2. This method is effective when the original logging policy had a non-zero probability of selecting the actions being evaluated. Joachims explains:

You can only evaluate policies on the actions that your logging policy that collected the data had basically non-zero probability of choosing.

--- Thorsten Joachims

By ensuring that log data is collected under stochastic conditions, developers can leverage existing data to assess new policies efficiently 3.