This paper analyzes regret bounds for Gaussian process Thompson sampling.
problem Analyzing the performance of Gaussian process Thompson sampling (GP-TS) in Bayesian optimization.
method The paper derives several regret bounds for GP-TS, including a lower bound, upper bounds on the second moment of cumulative regret, expected lenient regret, and improved cumulative regret.
result The paper provides improved regret upper bounds for GP-TS, showing that it suffers from a polynomial dependence on 1 / δ 1/δ 1/ δ with probability δ δ δ . New algorithm SELECT minimizes satisficing regret in bandits.
problem Minimizing regret in bandit optimization with satisficing arms.
method SELECT algorithm for satisficing regret minimization.
result SELECT achieves constant expected satisficing regret.
Optimal strategy identified for minimizing regret in fixed-budget best arm selection.
problem Minimizing expected simple regret in fixed-budget best arm selection.
method Two-Stage (TS)-Hirano-Imbens-Ridder (HIR) strategy using HIR estimator.
result TS-HIR strategy is asymptotically minimax optimal.
The paper tackles lexicographic multiarmed bandit problems with bounded regret.
problem Selecting lexicographic optimal arms in multiobjective bandit problems.
method Defining lexicographic regret, considering prior information, and proposing algorithms for both settings.
result Achieves uniformly bounded regret in time for both prior settings and sublinear gap-free regret in the prior-free case.
Paper analyzes GP-EI for Bayesian optimization with no regret and provides guidance on choosing incumbents.
problem Analyzing cumulative regret of GP-EI with different incumbents in noisy Bayesian optimization.
method Analyzes GP-EI with three incumbents (BPMI, BSPMI, BOI) in both SE and Matérn kernels, proving no-regret for BPMI and BSPMI.
result GP-EI with BPMI and BSPMI is a no-regret algorithm for both SE and Matérn kernels, providing theoretical guidance for choosing incumbents.
This paper improves Thompson Sampling for complex decision-making problems.
problem Learning in infinite-horizon discounted decision processes with unknown parameters.
method Developed a general canonical probability space and new metrics for analyzing adaptive learning algorithms.
result Thompson Sampling achieves complete learning in complex decision-making problems.
This paper improves GP-UCB by using a shifted exponential distribution for confidence parameters.
problem Theoretical confidence parameter in GP-UCB increases with iterations, leading to large values.
method Introduced IRGP-UCB, a randomized variant of GP-UCB using a shifted exponential distribution for confidence parameters.
result IRGP-UCB achieves sub-linear regret without increasing the confidence parameter.
Gaptron algorithm reduces mistakes in online multiclass classification.
problem Online multiclass classification with limited information.
method Randomized first-order algorithm exploiting the gap between zero-one loss and surrogate losses.
result First linear time algorithm with O ( K T ) O(K\sqrt{T}) O ( K T ) expected regret. The paper analyzes the sliding regret of stochastic bandit algorithms.
problem Measuring the one-shot behavior of no-regret algorithms in stochastic bandits.
method Introducing sliding regret to measure the worst pseudo-regret over a time-window.
result Randomized methods have optimal sliding regret, while index policies have the worst possible sliding regret.
Study Thompson Sampling in adversarial bit prediction, finding regret bounds and optimal sequences.
problem Adversarial bit prediction with varying error weights.
method Thompson Sampling, analyzing sequences with largest and smallest regret.
result Regret bounds for adversarial bit prediction sequences, including optimal and worst-case scenarios.
Improved regret bounds for scalable bandit convex optimization.
problem Designing online algorithms for high-dimensional bandit convex optimization.
method Projection-free algorithms using a linear optimization oracle.
result First algorithm with O ( T 3 / 4 ) O(T^{3/4}) O ( T 3/4 ) expected regret in O ( T ) O(T) O ( T ) calls. Posterior sampling-based EI achieves sublinear regret bounds for expensive function optimization.
problem Theoretical analysis of expected improvement (EI) in Bayesian optimization.
method Randomized posterior sampling of EI.
result Achieves sublinear Bayesian cumulative regret bounds.
Optimized bandit algorithms have heavy-tailed regret distributions that can grow faster than expected.
problem Heavy-tailed regret distributions in optimized bandit algorithms.
method Change-of-measure ideas and UCB algorithm modifications.
result Regret distributions of optimized UCB algorithms have a heavy Cauchy tail, and can grow faster than poly-logarithmically.
Study finds optimal regret bound for multi-armed bandit problem with expert advice.
problem Optimizing decision-making in a multi-armed bandit problem with expert advice.
method Proved a tight lower bound matching the upper bound of Kale (2014) for minimax expected regret.
result The minimax optimal expected regret is Θ(√(T K log (N/K))) for the problem.
Paper analyzes nonconvex bandit problems with improved adaptive methods.
problem Continuous armed bandit problems for nonconvex cost functions.
method Simple and adaptive bin splitting methods.
result Adaptive method achieves locally minimax optimal expected cumulative regret.
Optimal algorithm for identifying best-arm with minimal regret.
problem Identifying the best arm in two treatments with limited budget.
method Neyman allocation based on outcome standard deviations.
result Neyman allocation is minimax optimal for simple regret.
CTS reduces regret in probabilistically triggered combinatorial bandits.
problem Optimizing decisions with probabilistically triggered arms in combinatorial multi-armed bandits.
method Combinatorial Thompson Sampling (CTS) with a regret bound analysis.
result Derives an O ( ∑ i = 1 m log T / ( p i Δ i ) ) O(\sum_{i =1}^m \log T / (p_i Δ_i)) O ( ∑ i = 1 m log T / ( p i Δ i )) regret bound for CTS. Optimal algorithm found for collaborative learning in bandits with optimal regret bounds.
problem Minimizing regret in collaborative multi-agent bandit problems.
method Proposed an algorithm with optimal regret bounds for collaborative multi-agent multi-armed bandit model.
result First algorithm with order optimal regret bounds for collaborative bandit model.
Paper establishes no-regret property for practical EGO optimization.
problem No theoretical bounds on cumulative regret for practical EGO.
method Introduced practical EGO with a positive nugget, analyzed its regret bounds.
result Practical EGO is a no-regret algorithm with sublinear regret bounds.
Unified framework for distributional regret in bandits and reinforcement learning.
problem Characterizing the distribution of regret in multi-armed bandits and reinforcement learning.
method Unified framework with a UCBVI-style algorithm and distributional regret bounds.
result Distributional regret bounds with optimal trade-offs between expected and distributional regret.
Algorithm minimizes regret in non-stationary dueling bandits with unknown parameters.
problem Minimizing regret in dueling bandits with time-varying preferences.
method Proposes Beat the Winner Reset algorithm and meta-algorithms DETECT and Monitored Dueling Bandits.
result Proves bounds on expected weak and strong regret for non-stationary dueling bandits.
Optimizes treatment allocation using covariates for better outcomes.
problem Improving treatment allocation in multi-armed bandit problems.
method Maximizes a functional of the conditional potential outcome distribution.
result Developed expected regret lower bounds and near minimax optimal policy.
We prove non-asymptotic lower bounds on the expectation of the maximum of d d d independent Gaussian variables and the expectation of the maximum of d d d independent symmetric random walks. Both lower bounds recover the optimal leading constant in the limit. A simple application of the lower bound for random walks is an (…
Study optimizes experimental design for best treatment arm identification.
problem Identifying the best treatment arm given contextual information.
method Adaptive Sampling-Policy Learning (PLAS) strategy for minimax rate optimality.
result PLAS strategy achieves minimax rate optimality in expected simple regret.
A learner selects subsets of choices for a user who then picks from them, aiming to minimize regret.
problem Optimizing subset selection for user choices in a stochastic setting.
method Introduces a new problem and defines regret, then proposes algorithms with matching upper and lower bounds.
result Upper and lower bounds on expected regret match up to a logarithmic term, demonstrating algorithm efficiency.
Upper and lower bounds on regret for noisy optimization of Brownian motion.
problem Optimizing a one-dimensional Brownian motion with noisy observations.
method Upper bound uses confidence bounds and Markov property; lower bound uses hypothesis testing reduction.
result Upper and lower bounds are tight up to a factor of O ( ( log T ) 1.5 ) O((\log T)^{1.5}) O (( log T ) 1.5 ) . New algorithm reduces regret in combinatorial causal bandits without graph structure.
problem Minimizing regret in combinatorial causal bandits without graph structure.
method Design of algorithms for binary general causal models and BGLMs without graph skeleton.
result Achieves O ( T ln T ) O(\sqrt{T}\ln T) O ( T ln T ) expected regret for causal models and O ( T 2 3 ln T ) O(T^{\frac{2}{3}}\ln T) O ( T 3 2 ln T ) for BGLMs. TRAiL is a linear bandit algorithm that ensures optimal regret and guarantees inference quality.
problem Optimal regret and inference quality in linear bandits with convex action sets.
method TRAiL estimates the parameter through regularized least squares and perturbs the action set along the tangent plane.
result TRAiL achieves an Ω ( T ) Ω(\sqrt{T}) Ω ( T ) upper bound on cumulative regret with high probability. Learning reward functions can lead to poor policy performance despite low error.
problem Low error in learned reward functions does not guarantee low regret in policy performance.
method Mathematical analysis of reward learning and policy optimization.
result A low expected test error of the reward model guarantees low worst-case regret, but error-regret mismatch can occur with certain data distributions.
The paper improves regret bounds for admission control in queueing systems.
problem Improving regret bounds for admission control in queueing systems.
method Proposes an algorithm inspired by UCRL2 and uses problem structure to bound regret.
result Proves an upper bound on the expected total regret of O ( S log T + m T log T ) O(S\log T + \sqrt{mT \log T}) O ( S log T + m T log T ) . Optimal learning rate schedules for SGD in changing data distributions.
problem Minimizing regret in online learning with changing data distributions.
method Characterized optimal schedules for linear regression, proposed schedules for general convex and non-convex losses, and defined a notion of regret for non-convex losses.
result Upper and lower bounds for regret with constants for convex losses, and an upper bound on total expected regret for non-convex losses.
We consider a setting where an agent's uncertainty is represented by a set of probability measures, rather than a single measure. Measure-bymeasure updating of such a set of measures upon acquiring new information is well-known to suffer from problems; agents are not always able to learn appropriately. To deal with the…
We study the regret of optimal strategies for online convex optimization games. Using von Neumann's minimax theorem, we show that the optimal regret in this adversarial setting is closely related to the behavior of the empirical minimization algorithm in a stochastic process setting: it is equal to the maximum, over jo…
The paper analyzes and optimizes recommendation systems using user-user and item-item collaborative filtering.
problem Optimizing recommendation systems to minimize disliked recommendations.
method Proposes algorithms inspired by user-user and item-item collaborative filtering, proving performance guarantees in terms of expected regret.
result Information-theoretic lower bounds on regret match upper bounds up to logarithmic factors in two model parameter regimes.
Study contextual bandits with stage-wise constraints, proving regret bounds and extending results.
problem Contextual bandits with stage-wise constraints in high probability and expectation settings.
method Upper-confidence bound algorithms for linear and non-linear reward/cost functions, extending to multiple constraints.
result Regret bounds for various settings, including non-linear reward/cost functions.
The paper addresses frequentist regret of Linear Thompson Sampling in stochastic linear bandits.
problem The frequentist regret of Linear Thompson Sampling (LinTS) is worse than its Bayesian counterpart.
method The paper proves the fundamental nature of the frequentist regret bound for LinTS and proposes a data-driven version of LinTS to achieve minimax optimal frequentist regret.
result The frequentist regret bound for LinTS is O ~ ( d d T ) \widetilde{\mathcal{O}}(d\sqrt{dT}) O ( d d T ) , which is the best possible under certain conditions. We consider the problem of learning in single-player and multiplayer multiarmed bandit models. Bandit problems are classes of online learning problems that capture exploration versus exploitation tradeoffs. In a multiarmed bandit model, players can pick among many arms, and each play of an arm generates an i.i.d. rewar…
Optimal policy for dynamic assortment planning under MNL model with O ( T ) O(\sqrt{T}) O ( T ) regret.
problem Maximizing revenue in dynamic assortment planning under MNL model with unknown parameters.
method Trisection-based policy with adaptive confidence bounds.
result Achieves O ( T ) O(\sqrt{T}) O ( T ) regret bound, independent of the number of products. New policy optimizes risk and optimality in stochastic bandits.
problem Optimizing risk in stochastic bandits with heavy-tailed risk.
method Designing policies with worst-case optimality for expected regret and light-tailed risk distribution.
result Achieves worst-case optimality for expected regret and light-tailed risk distribution.
This paper studies risk-averse online learning, showing differences from risk-neutral approaches.
problem Risk-averse online learning under mean-variance performance measure.
method Analyzes bandit and full information settings, establishes fundamental limitations.
result Worst-case regret is lower bounded by Ω ( T ) Ω(T) Ω ( T ) , contrasting with Ω ( T ) Ω(\sqrt{T}) Ω ( T ) for risk-neutral learning. New algorithm identifies best arm in rested bandit setting.
problem Best arm identification in rested bandit with decreasing losses.
method Introduced a novel best arm identification problem and analyzed an arm elimination algorithm.
result Regret vanishes as time horizon increases, with convergence rate depending on expected loss function.
LSA minimizes aggregate regret in thresholding bandit with optimal performance.
problem Finding arms with mean rewards above a threshold within a fixed budget.
method LSA, a simple and anytime algorithm aiming to minimize aggregate regret.
result LSA is instance-wise asymptotically optimal in minimizing aggregate regret.
The paper analyzes expected improvement policies for minimizing functions in RKHSs and derives regret bounds.
problem Minimizing deterministic functions in RKHSs with Gaussian-process models.
method Analyzes expected improvement policies, uses sequential separation radii, and estimates Gram determinants and Kolmogorov widths.
result Simple regret rates for Matérn and squared-exponential kernels are derived.
Study online learning with feedback graphs and switching costs, providing algorithms and optimal regret bounds.
problem Online learning with partial feedback and switching costs.
method Analysis of feedback graphs, lower bound on expected regret, new algorithms (Threshold Based EXP3, EXP3. SC).
result Order optimal algorithms for specific cases and Threshold Based EXP3 outperforms in empirical evaluations.
We study contextual bandits with budget and time constraints, referred to as constrained contextual bandits.The time and budget constraints significantly complicate the exploration and exploitation tradeoff because they introduce complex coupling among contexts over time.Such coupling effects make it difficult to obtai…
We consider stochastic multi-armed bandits where the expected reward is a unimodal function over partially ordered arms. This important class of problems has been recently investigated in (Cope 2009, Yu 2011). The set of arms is either discrete, in which case arms correspond to the vertices of a finite graph whose stru…
Study bandit problems with BMO functions, achieving poly-log δ δ δ -regret.
problem Bandit problems with discontinuous, unbounded functions.
method Developed a toolset for BMO bandits and an algorithm achieving poly-log δ δ δ -regret. result Achieved poly-log δ δ δ -regret against an optimal arm after removing a δ δ δ -sized portion of the arm space. UCRL-CMDP algorithm optimizes RL with constraints on average costs.
problem Optimizing RL in MDPs with average cost constraints.
method Model-based RL algorithms maximizing reward while keeping costs within bounds.
result UCRL-CMDP algorithm's expected regret is upper-bounded by $T^{2\slash 3}$ .