Adaptive multi-objective optimization for primary and guardrail criteria

Organizations increasingly face the challenge of evaluating numerous potential user experiences, particularly in digital contexts where content can be easily modified through page layouts, personalized recommendations, and automated content generation. While creating these experiences has become increasingly straightforward, especially with the advent of generative AI, determining user preferences remains a critical bottleneck. A/B/n tests are the gold standard in determining whether a new treatment is an improvement over the business as usual. A/B tests are well understood, robust to a range of modeling assumptions, and allow the experimenter to measure a variety of different success criteria when determining whether to replace the control with the new treatment experience after the experiment concludes. A/B/n tests however, scale poorly to large numbers of treatments being tested simultaneously. As the number of treatments under consideration increases, the same pool of traffic arriving to the website is split into smaller fractions leading to noisier treatment effect estimates and more uncertain decision making.

Instead, organizations increasingly turn to adaptive experimentation to improve power in settings where the number of treatments is large. Generically, adaptive experimentation techniques such as multi-armed bandits compute performance statistics of different treatments as the experiment proceeds, and use this information to reallocate traffic towards the best performing treatments. In the extreme case, compared to a standard A/B/n test this can reduce the number of samples necessary to identify the best treatment by an O(K) factor where K is the number of treatments [5].

A primary reason limiting the further adoption of adaptive experimentation techniques is that these algorithms most commonly optimize for a single success metric. By contrast, organizations frequently care about a range of different success criteria when evaluating whether to launch a treatment such as financial metrics or customer engagement. The core metric(s) a treatment seeks to improve are referred to as primary criteria and other metric(s) which the treatment seeks to not degrade are referred to as guardrail criteria. Since adaptive algorithms most often optimize based on a single metric, it is common to experiment in two phases. First, an adaptive algorithm selects a winning treatment according to a single primary launch criteria. Then the winning treatment enters a standard A/B test to measure the remaining guardrail criteria. This method is powerful but limited. If primary and guardrail criteria come at the expense of each other, then the organization may not be willing to launch the winning treatment and will need to start over. Instead, in this paper we study the problem of adaptive multi-objective optimization with primary and guardrail criteria where the experimenter wishes to use an adaptive algorithm to select a treatment that is likely to simultaneously meet both primary and guardrail criteria before proceeding to a standard A/B test to make a final launch decision.