New tighter confidence bounds for sequential kernel regression.
problem Quantifying uncertainty in sequential learning algorithms.
method Martingale tail inequalities and conic programming.
result New confidence bounds are tighter than existing ones.
A new differentiable UCB algorithm for linear bandits learns adaptive confidence bounds.
problem Inability of UCB to strike optimal exploration-exploitation due to confidence bounds.
method Proposes a differentiable linear bandit algorithm and a gradient estimator for learning adaptive confidence bounds.
result Achieves a i l d e O ( β ^ d T ) ilde{\mathcal{O}}(\hatβ\sqrt{dT}) i l d e O ( β ^ d T ) upper bound of T T T -round regret. Note improves confidence bounds for random variables.
problem Improving confidence bounds for random variables with unbounded ranges and different distributions.
method PAC-Bayes-ification of a derived confidence bound.
result Streamlined proofs for existing results.
Improved confidence bounds for linear logistic model with applications to bandits.
problem Improving confidence bounds for linear logistic model.
method Self-concordant analysis of the logistic loss to avoid dependence on worst-case variance.
result Significant improvement in confidence bounds, avoiding dependence on 1 / κ 1/κ 1/ κ . The paper honors Lai's contributions to multi-armed bandits and establishes new regret bounds.
problem Improving regret bounds in multi-armed bandit problems.
method Establishes non-asymptotic regret bounds for upper confidence bound indices.
result New regret bounds match Lai-Robbins lower bound.
A new framework tightens risk measure confidence bounds.
problem Improving confidence bounds for various risk measures.
method Distribution optimization framework with two estimation schemes based on concentration bounds.
result Consistently tighter confidence bounds compared to previous methods.
Improved online confidence bounds for multinomial logistic models in bandits.
problem Achieving optimal regret in multinomial logistic bandits with bounded parameters and outcomes.
method Deriving an improved online confidence bound and proposing OFU-MNL++ and OFU-MN 2 ^2 2 L algorithms. result Achieved variance-dependent optimal regret for MNL bandits.
The paper presents a method to compute trusted confidence bounds for LECs in CPS.
problem Non-transparent predictions of LECs make CPS safety challenging.
method Inductive Conformal Prediction (ICP) and Triplet Network architecture.
result Efficient real-time computation of trusted confidence bounds.
Two batch Bayesian optimization algorithms with regret guarantees.
problem Efficiently optimizing multiple objectives in batch feedback settings.
method Gaussian process upper confidence bound and Thompson sampling approaches.
result Frequentist regret guarantees and numerical results.
This note gives a short, self-contained, proof of a sharp connection between Gittins indices and Bayesian upper confidence bound algorithms. I consider a Gaussian multi-armed bandit problem with discount factor γ γ γ . The Gittins index of an arm is shown to equal the γ γ γ -quantile of the posterior distribution of the arm'…
New principle optimizes bandit decisions with context.
problem Dealing with general function classes and large context spaces in bandits.
method Upper Counterfactual Confidence Bounds (UCCB) principle.
result Proves optimality and efficiency in complex settings.
Improved Bayesian optimisation method using randomised Gaussian process UCB.
problem Improving performance in Bayesian optimisation.
method Developed a modified Gaussian process upper confidence bound (GP-UCB) acquisition function.
result The method achieves better performance than GP-UCB in various problems.
Proposes a method to compute valid lower confidence bounds for multiple models selected based on their performance.
problem Model selection and evaluation in machine learning.
method Interprets model selection as a simultaneous inference problem, uses bootstrap tilting and maxT-type multiplicity correction.
result Yields valid lower confidence bounds that are at least as good as standard approaches and reliably reach nominal coverage probability.
Paper improves regret bounds for Gaussian process upper confidence bound in Bayesian optimization.
problem Minimizing regret in Gaussian process bandit optimization.
method Gaussian process upper confidence bound (GP-UCB) algorithm with refined analysis.
result Achieves O ( T ln 2 T ) O(\sqrt{T \ln^2 T}) O ( T ln 2 T ) cumulative regret under squared exponential kernel. The estimation of probabilities of default (PDs) for low default portfolios by means of upper confidence bounds is a well established procedure in many financial institutions. However, there are often discussions within the institutions or between institutions and supervisors about which confidence level to use for the…
One-bit feedback suffices for a bandit problem's optimal strategy.
problem Optimal strategy for multi-armed bandit problem with limited feedback.
method Coding and decoding schemes for one-bit feedback to mimic full-reward feedback.
result Regret ratio approaches 1 with one-bit feedback.
Bayesian methods improve drug discovery experiment design.
problem Optimizing drug screening experiments in high-dimensional data.
method Bayesian inference and optimisation with upper confidence bound algorithms, Thompson sampling, and sparse tree search.
result Sparse tree search techniques outperform other methods in drug toxicity screening.
Upper Confidence Bound (UCB) method is arguably the most celebrated one used in online decision making with partial information feedback. Existing techniques for constructing confidence bounds are typically built upon various concentration inequalities, which thus lead to over-exploration. In this paper, we propose a n…
Study optimizes dynamic product selection and pricing using censored preference feedback.
problem Maximizing revenue from dynamic assortment and pricing decisions.
method Proposes a censored multinomial logit model and LCB pricing strategy combined with UCB or TS product selection.
result Achieves optimal regret bounds for dynamic pricing and selection.
A new method optimizes robustness measures under input uncertainty using randomized Gaussian process upper confidence bound.
problem Optimizing robustness measures under input uncertainty.
method Randomized robustness measure GP-UCB (RRGP-UCB) that samples β from a chi-squared-based distribution.
result RRGP-UCB provides tight bounds on expected regret.
Improved UCB method for stochastic bandits using distance tuning.
problem Improving performance in stochastic bandit problems.
method Tuning confidence bounds based on bandit distance.
result Empirically shows increased performance compared to existing methods.
Berry et al. (1997) initiated the development of the infinite arms bandit problem. They derived a regret lower bound of all allocation strategies for Bernoulli rewards with uniform priors, and proposed strategies based on success runs. Bonald and Proutière (2013) proposed a two-target algorithm that achieves the regret…
New algorithm optimizes online decision-making with dynamically generated actions.
problem Balancing action generation costs with optimal decision-making in online learning.
method Doubly-optimistic algorithm using LCB for action selection and UCB for action generation.
result Achieves optimal regret bound of O ( T d d + 2 d d d + 2 + d T log T ) O(T^{\frac{d}{d+2}}d^{\frac{d}{d+2}} + d\sqrt{T\log T}) O ( T d + 2 d d d + 2 d + d T log T ) . Algorithm learns to switch control among agents in a team.
problem Learning to switch control among reinforcement learning agents.
method 2-layer Markov decision process, upper confidence bounds, shared confidence bounds.
result Sublinear total regret with shared confidence bounds.
WS-KDE provides robust confidence bounds for stochastic functions.
problem Optimizing time-consuming black-box functions with stochastic outputs.
method Wilson Score Kernel Density Estimation (WS-KDE) for Bayesian optimization.
result WS-KDE provides reliable confidence bounds for any stochastic function.
Contextual bandits are widely used in Internet services from news recommendation to advertising, and to Web search. Generalized linear models (logistical regression in particular) have demonstrated stronger performance than linear models in many applications where rewards are binary. However, most theoretical analyses …
A new UCB algorithm for heavy-tailed bandits with near-optimal regret.
problem Sequential decision making in uncertain environments with heavy-tailed rewards.
method Data-driven, distribution-free UCB algorithm combining resampled median-of-means and UCB.
result Near-optimal regret bound for heavy-tailed distributions.
This paper proposes a DGP approach with UCBs for point target tracking over WSNs.
problem Uncertainty quantification in distributed machine learning-based tracking over WSNs.
method Distributed Gaussian process (DGP) approach with upper confidence bounds (UCBs).
result UCBs provide 88% and 42% higher probability of encompassing true target states in X and Y coordinates, respectively.
We provide a pointwise confidence bound for non-linear least-squares with fixed design.
problem Confidence estimation in non-linear ℓ 2 \ell^2 ℓ 2 -regularized least squares. method Pointwise confidence bound for local minimizers, using weighted norm involving inverse-Hessian.
result The proposed confidence bound scales with the test input's similarity to the training data.
UCB algorithm provides stable sample means for sequential data.
problem Challenges in inferential tasks with sequential data.
method Stability property of UCB algorithm for multiarmed bandit problems.
result UCB algorithm ensures asymptotically normal sample means.
In this paper, the method UCB-RS, which resorts to recommendation system (RS) for enhancing the upper-confidence bound algorithm UCB, is presented. The proposed method is used for dealing with non-stationary and large-state spaces multi-armed bandit problems. The proposed method has been targeted to the problem of the …
Develops confidence bounds for off-policy evaluation in contextual bandits.
problem Evaluating policies that were not used to collect data.
method Martingale analysis for non-asymptotic, non-parametric, and valid confidence sequences.
result Empirically tight bounds on failure probability and width.
Paper proposes a new UCB approach for estimating maximum mean.
problem Estimating the maximum mean in various applications.
method Upper Confidence Bound (UCB) approach with adaptive sampling.
result LSA estimator shows faster bias decay compared to GA.
A new algorithm for better decision-making in recommendation systems.
problem Stochastic multi-armed bandit problem and cold start problem in recommender systems.
method Proposes Hellinger-UCB, a variant of UCB algorithm using squared Hellinger distance.
result Hellinger-UCB reaches the theoretical lower bound and outperforms other algorithms in practical applications.
In this paper, we study the stochastic combinatorial multi-armed bandit (CMAB) framework that allows a general nonlinear reward function, whose expected value may not depend only on the means of the input random variables but possibly on the entire distributions of these variables. Our framework enables a much larger c…
This paper improves GP-UCB by using a shifted exponential distribution for confidence parameters.
problem Theoretical confidence parameter in GP-UCB increases with iterations, leading to large values.
method Introduced IRGP-UCB, a randomized variant of GP-UCB using a shifted exponential distribution for confidence parameters.
result IRGP-UCB achieves sub-linear regret without increasing the confidence parameter.
CRB tackles rising rewards in combinatorial online learning.
problem Rising rewards in combinatorial online learning.
method CRB framework and CRUCB algorithm.
result Empirical and theoretical validation of CRUCB's effectiveness.
EBUCB framework achieves optimal regret with bounded approximate inference error.
problem Theoretical gap between practical performance and theoretical justification of Bayesian bandit algorithms with approximate inference.
method Enhanced Bayesian Upper Confidence Bound (EBUCB) framework that accommodates bandit problems with approximate inference.
result EBUCB achieves optimal regret order O ( log T ) O(\log T) O ( log T ) under certain conditions on inference error. Bayesian bandit algorithms with approximate inference improve regret bounds in stochastic linear bandits.
problem Theoretical justification for Bayesian bandit algorithms with approximate inference in stochastic linear bandits.
method Proposed a theoretical framework to analyze approximate inference impact and conducted frequentist regret analysis on LinTS and LinBUCB.
result LinTS and LinBUCB preserve their original regret upper bounds with larger constant terms in approximate inference settings.
ARTEO algorithm optimizes safety-critical systems with uncertainty.
problem Decision-making under uncertainty with safety constraints in real-time optimization.
method ARTEO algorithm uses multi-armed bandits as a mathematical programming problem subject to safety constraints, learning unknown characteristics through exploration and incorporating uncertainty quantification.
result ARTEO achieves less cumulative regret with accurate and safe decisions.
This paper is devoted to regret lower bounds in the classical model of stochastic multi-armed bandit. A well-known result of Lai and Robbins, which has then been extended by Burnetas and Katehakis, has established the presence of a logarithmic bound for all consistent policies. We relax the notion of consistence, and e…
Bayes-UCBVI tackles reinforcement learning with a new upper confidence bound method.
problem Optimizing exploration in reinforcement learning without bonuses.
method Bayes-UCBVI uses a quantile of a Q-value function posterior as an upper confidence bound.
result Proves a regret bound of order O ~ ( H 3 S A T ) \widetilde{O}(\sqrt{H^3SAT}) O ( H 3 S A T ) for tabular reinforcement learning. Optimistic algorithms and Thompson sampling use info-theory for better reinforcement learning.
problem Designing algorithms that balance exploration and exploitation in reinforcement learning.
method Integrating information-theoretic concepts into optimistic algorithms and Thompson sampling.
result Cumulative regret bound depends on uncertainty and quantifies prior information value.
In this paper we propose a multi-armed bandit inspired, pool based active learning algorithm for the problem of binary classification. By carefully constructing an analogy between active learning and multi-armed bandits, we utilize ideas such as lower confidence bounds, and self-concordant regularization from the multi…
Paper proposes MCTSPO for better reinforcement learning policy optimization.
problem Local optima and saddle points in gradient-based methods and poor initialization in gradient-free methods.
method Monte-Carlo tree search combined with gradient-free optimization.
result Improved performance on reinforcement learning tasks with deceptive or sparse reward functions.
Efficiently identifies good policies by choosing contexts for human feedback.
problem Efficiently identifying good policies in applications with high feedback costs.
method Introduces offline contextual dueling bandit setting and an upper-confidence-bound style algorithm.
result Proves a regret bound and shows superior performance over uniformly sampled contexts.
Excessive reuse of test data has become commonplace in today's machine learning workflows. Popular benchmarks, competitions, industrial scale tuning, among other applications, all involve test data reuse beyond guidance by statistical confidence bounds. Nonetheless, recent replication studies give evidence that popular…
Develops hypothesis tests for conditional distributions using learning-theoretic bounds.
problem Testing differences in conditional distributions and functionals.
method Transforming learning-theoretic bounds into hypothesis tests for conditional expectations.
result Establishes comprehensive foundation for conditional testing, including theoretical guarantees and practical implementations.