Publications

We introduce OpportunityFinder, a code-less framework for performing a variety of causal inference studies with panel data for non-expert users. In its current state, OpportunityFinder only requires users to provide raw observational data and a configuration file. A pipeline is then triggered that inspects/processes data, chooses the suitable algorithm(s) to execute the causal study. It returns the causal

Brands usually invest in a portfolio of digital ad products for brand consideration and conversion, and their performance is commonly evaluated with ad - attributed metrics. However, these metrics limit the measurement of advertising effectiveness within a short time window, typically of two weeks. Therefore, they could underestimate the total effect if some ad products' efficacy lasts beyond the measurement

Motivated by bid recommendation in online ad auctions, this paper considers a general class of multi-level and multi-agent games, with two major characteristics: one is a large number of anonymous agents, and the other is the intricate interplay between competition and cooperation. To model such complex systems, we propose a novel and tractable bi-objective optimization formulation with mean-field approximation

This paper proposes a learning model of online ad auctions that allows for the following four key realistic characteristics of contemporary online auctions: (1) ad slots can have different values and click-through rates depending on users’ search queries, (2) the number and identity of competing advertisers are unobserved and change with each auction, (3) advertisers only receive partial, aggregated feedback

Companies offering web services routinely run randomized online experiments to estimate the “causal impact” associated with the adoption of new features and policies on key performance metrics of interest. These experiments are used to estimate a variety of effects: the increase in click rate due to the repositioning of a banner, the impact on subscription rate as a consequence of a discount or special

Causal Impact (CI) measurement is broadly used across the industry to inform both short- and long-term investment decisions of various types. In this paper, we apply the double machine learning (DML) methodology to estimate average and conditional average treatment effects across 100s of customer action types for e-commerce and digital businesses and 100s of millions of customers that can be used in decisions

The majority of students who choose to major in engineering do so to become a part of the community of practice of professional engineers (Johri & Olds, 2011), meaning that they want to have adequate exposure to what a career as a professional engineer could potentially be as part of their college experience. However, according to Jonassen (2014), engineering graduates are not well trained to contribute

Network interference, where observed outcomes are influenced by interaction with nearby units, is a fundamental issue in A/B testing and experimentation in social and economic networks. Clustered randomization is a frequently-used strategy that aims to prevent confounding by limiting interaction between treated and untreated units. We study a model of least-squares estimation under network interference,

Computer science (CS) is special among STEM subjects: it aims at an industry sector that has the most job growth but has a constant shortage in the workforce; it is a relatively young and burgeoning subject in K-12 education that has a shortage of classroom teachers; and it is one of a very few STEM subjects that large number of students can master by learning it completely out-of-school. To inspire the

Classical Randomized Controlled Trials (RCTs), or A/B tests, are designed to draw causal inferences about a population of units, for example, individuals, plots of land or visits to a website. A key assumption underlying a standard RCT is the absence of interactions between units, or the stable unit treatment value assumption (Ann. Statist. 6 (1978) 34-58). Modem experimentation, however, is often conducted

In order to identify expertise, forecasters should not be tested by their calibration score, which can always be made arbitrarily small, but rather by their Brier score. The Brier score is the sum of the calibration score and the refinement score; the latter measures how good the sorting into bins with the same forecast is, and thus attests to “expertise.” This raises the question of whether one can gain

In order to improve prices at Amazon, we created Pricing Labs, a price experimentation platform. Since we do not price discriminate, we must run product-randomized experiments. We discuss how we randomize to prevent spillovers, run different experimental designs (i.e., crossovers) to improve precision, and control for demand trends and differences in treatment groups to get more precise treatment effect

We propose a novel architecture for time series models built upon state-space methods. We jointly estimate many, potentially multivariate, distributions defined using state-space models by partially pooling their parameters across the cross-section. These joint distributions define a novel recurrent neural network. By combining state-space methods and neural networks, we leverage the interpretability of

The fundamental challenge of drawing causal inference is that counterfactual outcomes are not fully observed for any unit. Furthermore, in observational studies, treatment assignment is likely to be confounded. Many statistical methods have emerged for causal inference under unconfoundedness conditions given pre-treatment covariates, including: propensity score-based methods, prognostic score-based methods

Generation of photo-realistic images, semantic editing and representation learning are only a few of many applications of high- resolution generative models. Recent progress in GANs have established them as an excellent choice for such tasks. However, since they do not provide an inference model, downstream tasks such as classification cannot be easily applied on real images using the GAN latent space.