New approach combines semi-supervised learning and bandits for better predictions.
problem Online semi-supervised learning with bandit feedback for applications like clinical trials and ad recommendations.
method Adjusted Graph Convolutional Network (GCN) for contextual bandits, with semi-supervised missing rewards imputation.
result Developed multi-GCN embedded contextual bandit algorithms verified on real-world datasets.
New algorithms improve dueling bandit performance in multiplayer settings.
problem Challenges in collaborative exploration of non-informative arm pairs in multiplayer dueling bandits.
method Demonstrated Follow Your Leader approach and message-passing fully distributed protocol.
result Multiplayer algorithms outperform single-player benchmarks.
We consider an online decision making setting known as contextual bandit problem, and propose an approach for improving contextual bandit performance by using an adaptive feature extraction (representation learning) based on online clustering. Our approach starts with an off-line pre-training on unlabeled history of co…
A meta-UCB method combines stochastic bandit algorithms.
problem Combining multiple stochastic bandit algorithms efficiently.
method Meta-UCB procedure solving an N-armed bandit problem.
result Final regret depends only on the best base algorithm's regret.
Paper tackles domain adaptation for contextual bandits with sub-linear regret.
problem Adapting contextual bandit algorithms across domains with distribution shift.
method Learn a bandit model for the target domain using feedback from the source domain.
result Sub-linear regret bound maintained across domains.
New approach reduces unconstrained linear bandits to simpler optimization problems.
problem Unconstrained linear bandits problem.
method Perturbation-based approach combined with comparator-adaptive OLO algorithms.
result First high-probability guarantees for both static and dynamic regret in unconstrained linear bandits.
Balances and eliminates base algorithms in bandits and RL to bound total regret.
problem Model selection in bandits and reinforcement learning with unknown optimal regret.
method Balances and eliminates base algorithms based on candidate regret bounds.
result Total regret bound is the best valid candidate regret bound times a small multiplicative factor.
This paper improves offline contextual bandits using distributional robustness.
problem Improving offline contextual bandits with robustness.
method Extends Distributionally Robust Optimization (DRO) for offline contextual bandits, introducing a convex reformulation of Counterfactual Risk Minimization.
result Automatic calibration of asymptotic confidence intervals for policy optimization.
Reduces user feedback needed for accurate recommender systems.
problem Limited user feedback in recommender systems.
method Partial Bandit and Semi-Bandit approach for efficient user feedback retrieval.
result Similar global accuracy and learning efficiency with reduced feedback.
Study on indexability of restless multi-armed bandits and rollout policy performance.
problem Maximizing discounted rewards in finite state restless multi-armed bandit problems.
method Decouple the problem into single-armed restless bandits, analyze using value iteration, and compare with Whittle index policy.
result Demonstrates conditions for indexability and compares performance of index policy and rollout policy.
Unified approach for non-stationary and clustered bandits.
problem Solving non-stationary and clustered bandits with overlapping solutions.
method Test of homogeneity for seamless integration of non-stationary and clustered bandits.
result Unified solution framework for change detection and cluster identification.
Unified approach to correlated multi-armed bandits reduces regret significantly.
problem Correlated rewards in multi-armed bandits.
method Developed a unified approach to leverage reward correlations and presented algorithms with rigorous analysis.
result C-UCB algorithm pulls non-competitive arms only O(1) times, improving over classic algorithms.
A new approach learns to represent context for nonstationary bandits.
problem Nonstationary contextual bandits where patterns change over time.
method Combines recurrent neural networks with contextual linear bandit algorithm.
result Consistently outperforms handcrafted historical contexts and other methods.
Novel approach to contextual bandits using self-supervised learning.
problem Exploiting rich data representations in computer vision without explicit labels.
method Combining contextual bandit objective with self-supervision objective.
result Substantial gains in cumulative reward on computer vision datasets.
New algorithm reduces regret in graphical bilinear bandits.
problem Optimizing decisions in a network of agents playing bilinear games.
method Optimism in the face of uncertainty principle applied to combinatorial NP-hard problem.
result Upper bound of i l d e O ( T ) ilde{O}(\sqrt{T}) i l d e O ( T ) on α α α -regret demonstrated. New meta-learning approach for bandit policies that achieve high average reward.
problem Designing bandit policies that balance between worst-case and Bayesian assumptions.
method Differentiable parameterized policies optimized using policy gradients.
result Proposed algorithm achieves low regret and is practical for various bandit problems.
This paper is devoted to the study of the max K-armed bandit problem, which consists in sequentially allocating resources in order to detect extreme values. Our contribution is twofold. We first significantly refine the analysis of the ExtremeHunter algorithm carried out in Carpentier and Valko (2014), and next propose…
New method for contextual bandits with corrupted context.
problem Contextual bandits with corrupted context in online settings.
method Combining contextual bandit and multi-armed bandit approaches.
result Improved learning from all iterations, including corrupted ones.
Optimizes algorithms for non-concave bandit problems.
problem Optimizing algorithms for non-concave bandit problems.
method Unified zeroth-order optimization paradigm.
result Minimax-optimal algorithms in the dimension for low-rank generalized linear bandit problems.
We consider a finite-armed structured bandit problem in which mean rewards of different arms are known functions of a common hidden parameter θ ∗ θ^* θ ∗ . Since we do not place any restrictions of these functions, the problem setting subsumes several previously studied frameworks that assume linear or invertible reward funct…
Paper solves Bayesian bandit problem with continuous-time limit and approximate policy.
problem Finding optimal policy in Bayesian bandit problems with large horizons.
method Reformulates Bayesian bandit problem as continuous Hamilton-Jacobi-Bellman (HJB) equation and proposes approximate Bayes-optimal policy.
result Approximate Bayes-optimal policy for large horizons with constant computational cost.
Efficiently handles contextual bandits with diffusion models.
problem Challenges in online decision-making with contextual bandits.
method Leverage pre-trained diffusion models as priors to capture action dependencies.
result Developed an algorithm for efficient posterior approximation.
We address the problem of regret minimization in logistic contextual bandits, where a learner decides among sequential actions or arms given their respective contexts to maximize binary rewards. Using a fast inference procedure with Polya-Gamma distributed augmentation variables, we propose an improved version of Thomp…
Improved control approach for correlated bandits with better performance.
problem General multi-armed bandit problem with correlated elements.
method Introducing entropy regularisation to obtain a smooth asymptotic approximation of the value function, leading to a semi-index approximation of the optimal decision process.
result Performance of Asymptotic Randomised Control (ARC) algorithm compares favorably with other approaches.
New approach for off-policy learning in contextual bandits with performance guarantees.
problem Improving performance of logging policies in contextual bandits.
method PAC-Bayesian analysis of policy mixtures, providing tighter generalization bounds and tractable optimization algorithms.
result Proved tighter generalization bounds and demonstrated effectiveness in practical scenarios.
Polynomial-time method solves complex combinatorial semi-bandits.
problem Optimal strategies for combinatorial semi-bandits with uncorrelated Gaussian rewards.
method Proposes a polynomial-time method to solve the Graves-Lai optimization problem for various combinatorial structures.
result First known approach to implement asymptotically optimal algorithms in polynomial time for combinatorial semi-bandits.
New algorithm reduces regret in contextual bandits with many near-boundary contexts.
problem High regret in contextual bandits with many near-boundary contexts.
method Hierarchical nearest neighbour approach, holding out contexts for computation.
result Eradicates high regret in adversarial contextual bandits.
New definition resolves ambiguity in non-stationary bandit classification.
problem Ambiguity in classifying non-stationary bandits using existing definitions.
method Introducing a formal definition that resolves ambiguity and provides a unified approach.
result Unified approach applicable to both Bayesian and frequentist formulations, resolves classification issues.
New bandit algorithm maximizes information gain.
problem Optimizing decision-making in uncertain environments.
method Approximates information maximization using entropy and free energy principles.
result Asymptotic optimality proven for two-armed bandit problem.
New algorithm for nonstationary multi-armed bandits with optimal performance.
problem Nonstationary multi-armed bandits with changing model parameters over time.
method Adaptive Resetting Bandit (ADR-bandit) algorithm using adaptive windowing techniques.
result ADR-bandit achieves nearly optimal performance in both abrupt and gradual changes.
Study invariant Lipschitz bandits, improving regret bounds.
problem Optimizing decisions under symmetry in online settings.
method Integrates side observations using group orbits into UniformMesh algorithm.
result Improved regret bound for invariant Lipschitz bandit class.
Bayesian optimization improved for biased data.
problem Adversarial bias in observations, especially hidden confounders.
method Reduction to dueling bandits, information-directed sampling (IDS).
result First efficient kernelized algorithm with regret guarantees.
The paper tackles lifelong learning in multi-armed bandits, aiming to minimize average regret over multiple tasks.
problem Minimizing average regret in multi-armed bandits over multiple tasks.
method Confidence interval tuning of UCB algorithms and greedy algorithms applied to a bandit over bandit approach.
result Empirical improvement over previous work in the mortal bandit problem.
Meta-learning bandits by reducing dimensionality with PCA.
problem Learning multiple bandit tasks with shared structure.
method Online Principal Component Analysis (PCA) for dimensionality reduction, combined with optimistic and Thompson sampling strategies.
result Significant reduction in expected regret compared to existing methods.
Paper tackles noisy bandit feedback for multiclass classification.
problem Learning multiclass classifier with corrupted feedback.
method Proposes an unbiased estimator technique to estimate noise rates and an end-to-end framework.
result Algorithm achieves mistake bounds of O ( T ) O(\sqrt{T}) O ( T ) in high noise and O ( T i c e f r a c 23 ) O(T^{
icefrac{2}{3}}) O ( T i ce f r a c 2 3 ) in worst case. New bandit algorithms improve sparse reward learning.
problem Sparse rewards hinder learning efficiency in real-world bandit applications.
method Developed algorithms based on Upper Confidence Bound and Thompson Sampling for zero-inflated distributions.
result Empirical performance of new algorithms is superior to existing methods.
New method for contextual bandit with missing rewards.
problem Contextual bandit with missing rewards in online settings.
method Combining contextual bandit approach with unsupervised learning (clustering) to estimate missing rewards.
result Promising empirical results on real-life datasets.
Optimistic covariance-adaptive algorithms improve combinatorial semi-bandits regret.
problem Optimal regret in stochastic combinatorial semi-bandits with adaptive covariance estimation.
method Design of OLS-UCB-C and COS-V algorithms leveraging online covariance estimation.
result Improved gap-free regret with T^1/2 complexity for COS-V.
In this paper we propose a flexible and efficient framework for handling multi-armed bandits, combining sequential Monte Carlo algorithms with hierarchical Bayesian modeling techniques. The framework naturally encompasses restless bandits, contextual bandits, and other bandit variants under a single inferential model. …
New algorithm optimizes smooth functions with Hölder exponent > 1.
problem Optimizing smooth functions with unknown Hölder exponent > 1.
method Two-layer algorithms using misspecified linear/polynomial bandit algorithms in bins.
result Regret bound of O ~ ( T d + α d + 2 α ) \tilde{O}(T^{\frac{d+\alpha}{d+2\alpha}}) O ~ ( T d + 2 α d + α ) for α > 1 \alpha > 1 α > 1 . Contextual multi-armed bandit problems arise frequently in important industrial applications. Existing solutions model the context either linearly, which enables uncertainty driven (principled) exploration, or non-linearly, by using epsilon-greedy exploration policies. Here we present a deep learning framework for cont…
Paper solves stochastic contextual linear bandits using linear bandit algorithms.
problem Stochastic contextual linear bandits with unknown context distribution.
method Establishes a reduction framework to convert to linear bandit problems.
result Achieves nearly optimal regret bound of O ( d T log T ) O(d\sqrt{T\log T}) O ( d T log T ) . New algorithm reduces regret for single-index bandits to nearly optimal.
problem Optimizing rewards from unknown projections of high-dimensional contexts.
method Two-phase algorithm: estimate projection direction, reduce to 1D bandit, use UCB.
result Achieved optimal regret of i l d e O ( T 2 / 3 ) ilde{\mathcal{O}}(T^{2/3}) i l d e O ( T 2/3 ) . New method for semiparametric bandits reduces regret to optimal levels.
problem Complex reward structures in semiparametric bandits.
method Experimental-design approach with sharp regret bound and PAC bound.
result Minimax regret of i l d e O ( d T ) ilde{O}(\sqrt{dT}) i l d e O ( d T ) and logarithmic regret under positive suboptimality gap. Improved regret for single-index bandits with optimal algorithm.
problem Optimizing rewards from unknown one-dimensional projections of high-dimensional contexts.
method Two-phase algorithm: first estimate projection direction, then discretize and use UCB.
result Achieved optimal regret of i l d e O ( T 2 / 3 ) ilde{\mathcal{O}}(T^{2/3}) i l d e O ( T 2/3 ) . This paper applies Thompson Sampling to asymmetric α \alpha α -stable bandits for financial and wireless data.
problem Optimizing exploration-exploitation in multi-armed bandits with asymmetric α \alpha α -stable distributions. method Thompson Sampling applied to unknown asymmetric α \alpha α -stable reward distributions. result Demonstrates effectiveness of Thompson Sampling for asymmetric α \alpha α -stable bandits. New algorithm eliminates arms to minimize regret in complex bandit problems.
problem Minimizing regret in combinatorial bandit problems with explicit exploration.
method Introduces a novel arm elimination scheme that partitions arms into three categories and incorporates explicit exploration.
result Achieves near-optimal regret in combinatorial multi-armed and linear contextual bandit problems.
Adapts bandit algorithms for online survival analysis under Cox PH model.
problem Online survival analysis challenges in a bandit framework.
method Adapts three bandit algorithms to balance exploration and exploitation.
result Demonstrates sublinear regret bounds and effective learning of treatment policies.