Bayesian optimization

Eric Hans Lee, Valerio Perrone, Cédric Archambeau, Matthias Seeger

ICML 2020 Workshop on AutoML

2020

Bayesian optimization (BO) is a class of global optimization algorithms ubiquitous in hyperparameter optimization (HPO). BO budgets are typically given in iterations, which implicitly measures convergence in terms of the number of hyperparameter evaluations. In practice, evaluation costs may vary in diﬀerent regions of the search space. For example, the cost of neural network training increases quadratically

Machine learning

Fair Bayesian optimization

Valerio Perrone, Michele Donini, Krishnaram Kenthapadi, Cédric Archambeau

ICML 2020 Workshop on AutoML

2020

Given the increasing importance of machine learning (ML) in our lives, algorithmic fairness techniques have been proposed to mitigate biases that can be amplified by ML. Commonly, these specialized techniques apply to a single family of ML models and a specific definition of fairness, limiting their effectiveness in practice. We introduce a general constrained Bayesian optimization (BO) framework to optimize

Machine learning

Causal Bayesian optimization

Virginia Aglietti, Xiaoyu Lu, Andrei Paleyes, Javier González

AISTATS 2020

2020

This paper studies the problem of globally optimizing a variable of interest that is part of a causal model in which a sequence of interventions can be performed. This problem arises in biology, operational research, communications and, more generally, in all fields where the goal is to optimize an output metric of a system of interconnected nodes. Our approach combines ideas from causal inference, uncertainty

Machine learning

Deep PQR: Solving inverse reinforcement learning using anchor actions

Sinong Geng, Houssam Nassif, Carlos A. Manzanares, A. Max Reppen, Ronnie Sircar

ICML 2020

2020

We propose a reward function estimation framework for inverse reinforcement learning with deep energy-based policies. We name our method PQR, as it sequentially estimates the Policy, the Qfunction, and the Reward function. PQR does not assume that the reward solely depends on the state, instead it allows for a dependency on the choice of action. Moreover, PQR allows for stochastic state transitions. To

Economics

Emulation of physical processes with Emukit

Andrei Paleyes, Maren Mahsereci, Javier González, Mark Pullin

NeurIPS 2019 Workshop on Machine Learning and Physical Sciences

2019

Decision making in uncertain scenarios is an ubiquitous challenge in real world systems. Tools to deal with this challenge include simulations to gather information and statistical emulation to quantify uncertainty. The machine learning community has developed a number of methods to facilitate decision making, but so far they are scattered in multiple different toolkits, and generally rely on a fixed backend

Machine learning

Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning

Valerio Perrone, Huibin Shen, Matthias Seeger, Cédric Archambeau, Rodolphe Jenatton

NeurIPS 2019

2019

Bayesian optimization (BO) is a successful methodology to optimize black-box functions that are expensive to evaluate. While traditional methods optimize each black-box function in isolation, there has been recent interest in speeding up BO by transferring knowledge across multiple related black-box functions. In this work, we introduce a method to automatically design the BO search space by relying on

Machine learning

Meta-Surrogate Benchmarking for Hyperparameter Optimization

Aaron Klein, Javier González, Zhenwen Dai, Frank Hutter, Neil Lawrence

NeurIPS 2019

2019

Despite the recent progress in hyperparameter optimization (HPO), available benchmarks that resemble real-world scenarios usually consist of a few and very large problem instances that are expensive to solve. This blocks researchers and practitioners from systematically running large-scale comparisons that are needed to draw statistically significant results. This work proposes a method to alleviate these

Cloud and systems

Constrained Bayesian optimization with max-value entropy search

Valerio Perrone, Iaroslav Shcherbatyi, Rodolphe Jenatton, Cédric Archambeau, Matthias Seeger

NeurIPS 2019 Workshop on Metalearning

2019

Bayesian optimization (BO) is a model-based approach to minimize expensive black-boxes, and has been widely used to tune the hyperparameters of complex models such as deep neural networks. For many real-world black-boxes, however, the optimization is further subject to a priori unknown constraints. For example, model training may fail for certain conﬁgurations due to divergence or out of memory errors.

Machine learning

Scalable Hyperparameter Transfer Learning

Valerio Perrone, Rodolphe Jenatton, Matthias Seeger, Cédric Archambeau

NeurIPS 2018

2018

Bayesian optimization (BO) is a model-based approach for gradient-free black-box function optimization, such as hyperparameter optimization. Typically, BO relies on conventional Gaussian process (GP) regression, whose algorithmic complexity is cubic in the number of evaluations. As a result, GP-based BO cannot leverage large numbers of past function evaluations, for example, to warm-start related BO runs

Machine learning

Deep Gaussian processes for multi-fidelity modeling

Kurt Cutajar, Mark Pullin, Andreas Damianou, Javier González, Neil Lawrence

NeurIPS 2018

2018

Multi-fidelity methods are prominently used when cheaply-obtained, but possibly biased and noisy, observations must be effectively combined with limited or expensive true data in order to construct reliable models. This arises in both fundamental machine learning procedures such as Bayesian optimization, as well as more practical science and engineering applications. In this paper we develop a novel multi-fidelity

Machine learning

Bayesian optimization

Work with us