Human-in-the-loop Bayesian Optimization with No-Regret Guarantees

In recent years, Bayesian Optimization (BO) has received increasing attention due to its high sample efficiency in the global optimization of expensive-to-evaluate functions. As a principled approach to sequential decision-making under uncertainty, BO has been widely used in domain-specific problems such as hyper-parameter tuning when training machine learning models. This process, however, often ignores domain expertise on the specific problem and therefore puts human out of the learning loop. In this paper, we present a unified human-in-the-loop framework that involves human inputs in the BO procedure in the form of a prior distribution on the location of the global optimum. We propose two algorithms, namely $\pi$UCB and its variate $\pi^+$UCB, to show the generality of our framework in incorporating human preference into existing and new BO acquisition functions. We further demonstrate the performance improvement with human experience via experiments under both good and bad priors, and give theoretical justifications for the guaranteed no-regret convergence property of the proposed algorithm