A new algorithm for better decision-making in recommendation systems.
problem Stochastic multi-armed bandit problem and cold start problem in recommender systems.
method Proposes Hellinger-UCB, a variant of UCB algorithm using squared Hellinger distance.
result Hellinger-UCB reaches the theoretical lower bound and outperforms other algorithms in practical applications.
UCB algorithm adapted for large-scale, non-sub-Gaussian problems.
problem Selecting the best alternative from a large set of options with non-sub-Gaussian performance distributions.
method Adapted UCB algorithm for non-sub-Gaussian settings, focusing on sample size and meta-UCB selection.
result UCB algorithms can achieve sample optimality in large-scale, non-sub-Gaussian problems.
UCB-V algorithm improves on UCB for MAB problems with variance estimates.
problem Optimizing arm selection in MAB problems with variance information.
method Asymptotic and high probability analysis of UCB-V algorithm.
result UCB-V can exhibit instability in arm-pulling rates but achieves refined regret bounds.
In this paper we propose and explore the k-Nearest Neighbour UCB algorithm for multi-armed bandits with covariates. We focus on a setting where the covariates are supported on a metric space of low intrinsic dimension, such as a manifold embedded within a high dimensional ambient feature space. The algorithm is concept…
A new UCB algorithm for heavy-tailed bandits with near-optimal regret.
problem Sequential decision making in uncertain environments with heavy-tailed rewards.
method Data-driven, distribution-free UCB algorithm combining resampled median-of-means and UCB.
result Near-optimal regret bound for heavy-tailed distributions.
We consider the sequential Bayesian optimization problem with bandit feedback, adopting a formulation that allows for the reward function to vary with time. We model the reward function using a Gaussian process whose evolution obeys a simple Markov model. We introduce two natural extensions of the classical Gaussian pr…
A meta-UCB method combines stochastic bandit algorithms.
problem Combining multiple stochastic bandit algorithms efficiently.
method Meta-UCB procedure solving an N-armed bandit problem.
result Final regret depends only on the best base algorithm's regret.
Fine-grained gap-dependent regret bounds for reinforcement learning.
problem Achieving optimal regret bounds for reinforcement learning with suboptimality gaps.
method Developed novel analytical frameworks and refined algorithms for UCB-based and non-UCB-based reinforcement learning.
result Established the first fine-grained gap-dependent regret bounds for both UCB-based and non-UCB-based algorithms.
FP-UCB algorithm achieves bounded regret for finitely parameterized multi-armed bandits.
problem Finitely parameterized multi-armed bandits with unknown but known parameter set.
method FP-UCB algorithm using structural information about the parameter set.
result FP-UCB achieves bounded regret under structural condition, logarithmic otherwise.
Unified proof for various bandit algorithms with logarithmic regret.
problem Achieving logarithmic regret in stochastic bandit algorithms.
method Minimal high-probability concentration condition and two deterministic lemmas.
result Unified proofs for classical and contemporary bandit algorithms.
Greedy algorithms outperform UCB in many-armed bandit problems.
problem Optimizing multi-armed bandit problems with many arms.
method Subsampled UCB and greedy algorithms comparison.
result Greedy algorithms achieve optimal regret in many-armed bandit problems.
New algorithm prevents strategic replication in multi-armed bandit problems.
problem Strategic replication by agents can exploit bandit algorithms' balance.
method Designs Hierarchical UCB (H-UCB) and Robust Hierarchical UCB (RH-UCB) algorithms.
result Achieves O ( ln T ) O(\ln T) O ( ln T ) -regret and sublinear regret in realistic scenarios. New UCB-type algorithms reduce regret bounds for stochastic bandits with heavy and super heavy noise.
problem Improving regret bounds for stochastic bandits with heavy-tailed noise.
method General convex optimization methods with an inexact oracle, Clipped-SGD-UCB algorithm.
result Achieved an O ( log T K T log T ) O(\log T\sqrt{KT\log T}) O ( log T K T log T ) regret bound for symmetric noise, better than general lower bounds. UCB algorithm's arm-sampling behavior is revealed, leading to new insights and proofs.
problem Optimizing multi-armed bandit algorithms for worst-case scenarios.
method Analysis of UCB algorithm's arm-sampling behavior and process-level characterization.
result UCB's arm-sampling rates are asymptotically deterministic, regardless of problem complexity.
New findings show many popular bandit algorithms are unstable, contradicting minimax optimality.
problem Challenges in statistical inference from bandit algorithms due to adaptive, non-i.i.d. nature.
method Analysis of stability properties of optimism-based bandit algorithms.
result Widely used minimax-optimal UCB-style algorithms are unstable.
New algorithm outperforms existing ones in multi-player bandit problems without sensing.
problem Decentralized multi-player multi-armed bandit problem without collision or sensing info.
method Randomized Selfish KL-UCB, inspired by Selfish KL-UCB, with low complexity.
result Randomized Selfish KL-UCB outperforms state-of-the-art algorithms in almost all environments.
New UCB algorithm for learning PSRs with tractable computation and accuracy.
problem Learning predictive state representations in sequential decision-making problems.
method Proposes a novel UCB-type algorithm with a bonus term to estimate PSRs accurately and efficiently.
result First known UCB-type approach for PSRs with guaranteed model accuracy and computational tractability.
UCB algorithms improve on bandit problems with precise regret analysis and adaptive inference.
problem Understanding the performance and statistical inference of UCB algorithms in multi-armed bandit problems.
method Deterministic characterization of arm pulls and precise regret analysis.
result UCB algorithms' maximal regret deviates from minimax regret by a logarithmic factor, and the Lai-Robbins formula is exact only under specific conditions.
UCB algorithm provides stable sample means for sequential data.
problem Challenges in inferential tasks with sequential data.
method Stability property of UCB algorithm for multiarmed bandit problems.
result UCB algorithm ensures asymptotically normal sample means.
Unified theory for UCB policies in total and max bandit problems.
problem Order optimality of UCB policies in max bandit problems.
method Unified definition of UCB policy using oracle quantity and failure count.
result UCB policies are order optimal in both total and max bandit problems.
New algorithm robust to outliers in Bayesian Optimization.
problem Vulnerability of Bayesian Optimization to extreme outliers.
method Introduces a new adversary with a frequency-bounded corruption budget and derives RCGP-UCB algorithm.
result Achieves sublinear regret in the presence of up to O ( T 1 / 4 ) O(T^{1/4}) O ( T 1/4 ) and O ( T 1 / 7 ) O(T^{1/7}) O ( T 1/7 ) corruptions with possibly infinite magnitude. In this work, we address the open problem of finding low-complexity near-optimal multi-armed bandit algorithms for sequential decision making problems. Existing bandit algorithms are either sub-optimal and computationally simple (e.g., UCB1) or optimal and computationally complex (e.g., kl-UCB). We propose a boosting a…
This paper improves GP-UCB by using a shifted exponential distribution for confidence parameters.
problem Theoretical confidence parameter in GP-UCB increases with iterations, leading to large values.
method Introduced IRGP-UCB, a randomized variant of GP-UCB using a shifted exponential distribution for confidence parameters.
result IRGP-UCB achieves sub-linear regret without increasing the confidence parameter.
This paper is about index policies for minimizing (frequentist) regret in a stochastic multi-armed bandit model, inspired by a Bayesian view on the problem. Our main contribution is to prove that the Bayes-UCB algorithm, which relies on quantiles of posterior distributions, is asymptotically optimal when the reward dis…
First private Bayesian optimization algorithm with provable performance.
problem Privacy-preserving Bayesian optimization in outsourced settings.
method Random projection-based transformation to preserve privacy and distances.
result Regret bound similar to non-private GP-UCB established for PO-GP-UCB.
UCB exploration improves best arm identification in fixed-budget settings.
problem Best arm identification in fixed-budget scenarios.
method Adaptive allocations based on upper confidence bounds (UCBs) with prior information learning.
result Empirically and theoretically efficient for Bayesian BAI problem with improved performance.
Adaptive KL-UCB algorithm for Markov and i.i.d. rewards.
problem Regret minimization for Markovian and i.i.d. rewards in MAB problems.
method Identifies Markovian vs. i.i.d. rewards, switches between KL-UCB variants.
result Logarithmic regret for both i.i.d. and Markovian settings.
Neural- σ 2 σ^2 σ 2 -LinearUCB improves regret in neural contextual bandits.
problem Balancing exploration and exploitation in neural contextual bandits.
method Proposes a variance-aware neural UCB algorithm using neural representations and an upper bound of reward noise variance.
result Oracle and practical versions of Neural- σ 2 σ^2 σ 2 -LinearUCB achieve better regret guarantees and performance. Algorithm improves multi-armed bandit performance by transferring reward samples.
problem Sequential multi-armed bandit problem with changing reward distributions.
method UCB algorithm with reward sample transfer.
result Significant improvement in cumulative regret over standard UCB.
A UCB algorithm reduces regret in cooperative multi-agent graph bandits.
problem Cooperative multi-agent decision-making on a graph with shared rewards.
method Upper Confidence Bound (UCB) algorithm for minimizing regret.
result The Multi-G-UCB algorithm achieves expected regret O ( γ N log ( T ) [ K T + D K ] ) O(γN\log(T)[\sqrt{KT} + DK]) O ( γ N log ( T ) [ K T + D K ]) . New algorithm reduces sample complexity for Top Two method.
problem Fixed-confidence best arm identification for Top Two methods.
method UCB-based Top Two algorithm for non-asymptotic analysis.
result First non-asymptotic upper bound on expected sample complexity.
GP-UCB resolves sublinear regret for kernelized bandits.
problem Minimizing regret in kernelized bandit problems.
method Using a new regularization technique for kernel ridge estimators, improving GP-UCB's sublinear regret rate.
result GP-UCB achieves nearly optimal sublinear regret for the Matérn kernel.
Characterizes UCB algorithm performance in bandit problems.
problem Optimizing UCB algorithm performance in stochastic bandit environments.
method Novel perturbation analysis to characterize CLT of pulls and means.
result Smooth interpolation of pseudo-regret between large and small gap regimes.
A new algorithm for competing agents in a two-sided market setting.
problem Decentralized competition between agents in a two-sided market with unknown valuations.
method UCB-D3 algorithm for UCB with Decentralized Dominant-arm Deletion.
result UCB-D3 is order optimal and achieves a new regret lower bound.
Improved UCB method for stochastic bandits using distance tuning.
problem Improving performance in stochastic bandit problems.
method Tuning confidence bounds based on bandit distance.
result Empirically shows increased performance compared to existing methods.
ET-GP-UCB optimizes time-varying functions without knowing change rates.
problem Sequentially optimizing a time-varying objective function with unknown change rates.
method Event-triggered Bayesian optimization with adaptive resets based on probabilistic uniform error bounds.
result ET-GP-UCB outperforms other GP-UCB algorithms in synthetic and real-world data.
We study how to learn optimal interventions sequentially given causal information represented as a causal graph along with associated conditional distributions. Causal modeling is useful in real world problems like online advertisement where complex causal mechanisms underlie the relationship between interventions and …
BaNk-UCB tackles batched nonparametric bandits with k-NN regression and UCB.
problem Sequential decision-making with limited online feedback in domains like medicine and marketing.
method Combines k-NN regression with UCB principle for fully nonparametric, adaptive, and simple implementation.
result Near-optimal regret guarantees under Lipschitz smoothness and margin assumptions, with minimax-optimal rates.
Optimized Sharpe Ratio for better risk-adjusted decision-making in multi-armed bandits.
problem Challenging to optimize Sharpe Ratio (SR) in multi-armed bandits (MAB) due to constant regret.
method Proposed UCB-RSSR algorithm for RSSR maximization, derived path-dependent concentration bound and regret guarantees.
result UCB-RSSR outperforms existing algorithms and finds applications in risk-aware portfolio management.
I introduce and analyse an anytime version of the Optimally Confident UCB (OCUCB) algorithm designed for minimising the cumulative regret in finite-armed stochastic bandits with subgaussian noise. The new algorithm is simple, intuitive (in hindsight) and comes with the strongest finite-time regret guarantees for a hori…
GP-UCB performs suboptimally under certain conditions, as shown by a new regret lower bound.
problem The suboptimality of GP-UCB under polynomial effective optimism.
method Analysis of effective optimism level and new regret lower bound.
result GP-UCB is not minimax optimal under polynomial growth of effective optimism.
UCB-Advantage learns MDPs with O ( H 2 S A T ) O(\sqrt{H^2SAT}) O ( H 2 S A T ) regret.
problem Model-free reinforcement learning in finite-horizon MDPs.
method Reference-Advantage decomposition for low regret.
result Achieves i l d e O ( H 2 S A T ) ilde{O}(\sqrt{H^2SAT}) i l d e O ( H 2 S A T ) regret, matching best known bounds. The paper analyzes the sliding regret of stochastic bandit algorithms.
problem Measuring the one-shot behavior of no-regret algorithms in stochastic bandits.
method Introducing sliding regret to measure the worst pseudo-regret over a time-window.
result Randomized methods have optimal sliding regret, while index policies have the worst possible sliding regret.
Multi-armed bandit (MAB) is a class of online learning problems where a learning agent aims to maximize its expected cumulative reward while repeatedly selecting to pull arms with unknown reward distributions. We consider a scenario where the reward distributions may change in a piecewise-stationary fashion at unknown …
TS-UCB improves Thompson Sampling with minimal extra computation.
problem Online decision problems with bandit feedback.
method TS-UCB uses posterior samples and upper confidence bounds to select arms.
result TS-UCB achieves lower regret on various datasets.
Optimal mixtures of generative models outperform individual models on image datasets.
problem Selecting the best single model from a group of trained generative models.
method Formulated a quadratic optimization problem and proposed the Mixture-UCB algorithm for efficient selection.
result Mixture of generative models achieves better evaluation scores than individual models on benchmark datasets.
Contrastive UCB improves RL by learning feature representations efficiently.
problem Improving feature learning in RL for online decision making.
method Proposes UCB-based contrastive learning algorithms for RL in MDPs and MGs.
result Proves sample efficiency in learning optimal policies and Nash equilibria.
A bandit algorithm reduces regret in noisy, communication-constrained feedback.
problem Distributed stochastic multi-armed bandit with noisy, communication-constrained feedback.
method Proposes a multi-phase bandit algorithm, UE-UCB++, that matches an information-theoretic lower bound.
result Matches an information-theoretic lower bound of Ω(√(KT/σ²)) on the minimax regret.