Algorithm identifies best arm in linked bandits with reduced feedback.
problem Best arm identification in linked bandits with reduced feedback.
method Combines uniform sampling with regular bandit algorithm.
result Almost matching upper and lower bounds on sample complexity.
Con-TS optimizes wireless link throughput with latency constraints.
problem Optimizing rate selection for wireless links with latency constraints.
method Proposes Con-TS, a constrained Thompson sampling algorithm for stochastic MAB problems.
result Con-TS achieves upper bounds on expected constraint violations and throughput loss.
Bayesian optimization improved for biased data.
problem Adversarial bias in observations, especially hidden confounders.
method Reduction to dueling bandits, information-directed sampling (IDS).
result First efficient kernelized algorithm with regret guarantees.
Kernelized bandit algorithm tackles adaptive contextual bandits with single-index models.
problem Adaptive contextual bandits with single-index models and unknown link functions.
method Kernelized ε-greedy algorithm combining Stein-based index estimation and kernel ridge regression for reward functions.
result Unified framework for simultaneous learning and inference in single-index contextual bandits.
The web link selection problem is to select a small subset of web links from a large web link pool, and to place the selected links on a web page that can only accommodate a limited number of links, e.g., advertisements, recommendations, or news feeds. Despite the long concerned click-through rate which reflects the at…
Paper presents adaptive Lipschitz bandit framework for efficient optimization.
problem Optimizing rewards and minimizing regret in stochastic Lipschitz bandit problems.
method Adaptive learning of partitions in context- and arm-space using hierarchical Bayesian models.
result Achieves state-of-the-art performance in real-world tasks like neural network hyperparameter tuning.
New algorithm minimizes regret in multi-agent bandit problem with probabilistic communication.
problem Minimizing group regret in multi-agent multi-armed bandit problem with probabilistic communication.
method Proposes a new UCB-based algorithm for decentralized multi-agent multi-armed bandit problem on d d d -regular graphs with probabilistic communication. result The proposed algorithm outperforms state-of-the-art algorithms in minimizing group regret.
New algorithms for generalized linear bandits with unknown reward functions.
problem Misspecification of reward functions in existing bandit algorithms.
method Introducing single index bandits, proposing STOR, ESTOR, and GSTOR algorithms.
result Achieved nearly optimal regret bound of i l d e O T ( T ) ilde{O}_T(\sqrt{T}) i l d e O T ( T ) . New method uses reinforcement learning to accurately estimate available network bandwidth.
problem Accurate and fast estimation of available bandwidth in networks with varying cross-traffic.
method Employed reinforcement learning, specifically the ε ε ε -greedy algorithm in a multi-armed bandit approach. result Proposed method identifies available bandwidth with high precision and converges under various challenging conditions.
Transfer learning for bandits with latent Lipschitz continuity.
problem Learning to transfer structural information from prior tasks to new tasks.
method Proposes a framework to estimate Lipschitz constant from prior tasks and apply it to new tasks.
result Regret bound close to oracle algorithm with full knowledge of Lipschitz constant under mild assumptions.
Greedy algorithms outperform UCB in many-armed bandit problems.
problem Optimizing multi-armed bandit problems with many arms.
method Subsampled UCB and greedy algorithms comparison.
result Greedy algorithms achieve optimal regret in many-armed bandit problems.
A new algorithm reduces regret in bandit problems with adversarial corruptions.
problem Optimizing decision-making in bandit problems with variable uncertainties and adversarial interference.
method Proposes HCW-GLB-OMD, an OMD-based estimator with Hessian-based confidence weights for robustness.
result Achieves instance-wise minimax optimality with a κ κ κ -factor in the corruption term. An algorithm tackles low-rank linear bandit problems with improved regret bounds.
problem Low-rank linear bandit problems where rewards are inner products with an unknown low-rank matrix.
method Combines online-to-confidence-set conversion and exponentially weighted average forecaster with a covering of low-rank matrices.
result Achieves O ~ ( ( d 1 + d 2 ) 3 / 2 r T ) \widetilde{O}((d_1+d_2)^{3/2}\sqrt{rT}) O (( d 1 + d 2 ) 3/2 r T ) regret, improving over standard bounds when r ≪ min { d 1 , d 2 } r \ll \min\{d_1,d_2\} r ≪ min { d 1 , d 2 } . New algorithms reduce regret in neural logistic bandits.
problem Learning unknown reward functions in neural networks.
method Introduced a Bernstein-type inequality for self-normalized vector-valued martingales.
result Regret bounds improved to O ~ ( d ~ κ T ) \widetilde{O}(\widetilde{d}\sqrt{κT}) O ( d κ T ) and O ~ ( d ~ T / κ ) \widetilde{O}(\widetilde{d}\sqrt{T/κ}) O ( d T / κ ) . New algorithm for identifying Condorcet team in noisy comparisons.
problem Online learning with noisy comparisons of teams.
method Formalized dueling teams problem, developed algorithms for stochastic and deterministic settings.
result Identifies Condorcet winning team with reduced number of duels.
New metric measures variability in bandit algorithms, linking regret and variability.
problem Variability in multi-armed bandit allocations harms modern applications.
method Introduces allocation variability as a new metric and establishes a trade-off with regret.
result Any minimax regret-optimal algorithm must incur worst-case allocation variability Θ(T).
Bayesian algorithms minimize cumulative regret in decentralized multi-agent bandits.
problem Minimizing cumulative regret in a decentralized multi-agent multi-armed bandit problem.
method Proposed decentralized Bayesian multi-armed bandit framework, including Thompson Sampling and Bayes-UCB algorithms.
result Regret scales logarithmically with constants matching those of an optimal centralized agent.
New algorithm achieves nearly optimal regret with one-pass updates for GLB problems.
problem Generalized linear bandits with non-linear reward distributions.
method Jointly efficient algorithm using OMD estimator with one-pass updates.
result Nearly optimal regret bound with O ( 1 ) \mathcal{O}(1) O ( 1 ) time and space complexities per round. We propose combinatorial cascading bandits, a class of partial monitoring problems where at each step a learning agent chooses a tuple of ground items subject to constraints and receives a reward if and only if the weights of all chosen items are one. The weights of the items are binary, stochastic, and drawn independe…
Wireless systems perform rate adaptation to transmit at highest possible instantaneous rates. Rate adaptation has been increasingly granular over generations of wireless systems. The base-station uses SINR and packet decode feedback called acknowledgement/no acknowledgement (ACK/NACK) to perform rate adaptation. SINR i…
SGD shows distinct phases in learning single-index models, achieving optimal sample complexity and regret.
problem Learning single-index models with SGD in adaptive data settings.
method Stochastic gradient descent (SGD) with an optimal learning rate schedule.
result SGD achieves near-optimal sample complexity and regret guarantees across both burn-in and learning phases.
Algorithm discovers edges between pairs of nodes with limited queries.
problem Discovering good matches between pairs of entities with limited queries.
method Pair-matching problem as a multi-armed bandit with constraints, focusing on Stochastic Block Model.
result Sublinear regret achievable in Stochastic Block Model with two communities, with phase transition related to community detection.
We stabilize the Kumaraswamy distribution for efficient sampling and differentiation.
problem Numerical instabilities in the Kumaraswamy distribution's inverse CDF and log-pdf.
method Identified and resolved numerical issues, introduced a stabilized KS distribution.
result Stabilized Kumaraswamy distribution supports efficient sampling and differentiation.
We address the cold start problem in recommendation systems assuming no contextual information is available neither about users, nor items. We consider the case in which we only have access to a set of ratings of items by users. Most of the existing works consider a batch setting, and use cross-validation to tune param…
We consider an adversarial online learning setting where a decision maker can choose an action in every stage of the game. In addition to observing the reward of the chosen action, the decision maker gets side observations on the reward he would have obtained had he chosen some of the other actions. The observation str…
No-regret optimization for time-varying functions using uncertainty injection.
problem Optimizing time-varying functions with no-regret in bandit feedback.
method W-SparQ-GP-UCB, incorporating uncertainty injection and additional queries.
result Achieves no-regret with a vanishing number of additional queries per iteration.
GL-LowPopArt improves minimax-optimal estimation for trace regression.
problem Minimizing estimation error in generalized low-rank trace regression.
method Two-stage approach: nuclear norm regularization followed by matrix Catoni estimation.
result Achieves instance-wise optimal error bounds up to condition number.
Describes state variables in sequential decision problems, linking them to Markovian and non-Markovian models.
problem Sequential decision problems, especially in active learning and POMDPs, where decisions affect what is observed and learned.
method Canonical framework and novel two-agent perspective of POMDPs, defining state variables to claim Markovian or non-Markovian models.
result Properly modeled sequential decision problems are Markovian, while real decision problems are often non-Markovian.
Sharp analysis improves RLHF sample complexity with KL-regularization.
problem Improving RLHF sample complexity with KL-regularization.
method Sharp analysis of KL-regularized contextual bandits and RLHF.
result Achieved an O(1/ε) sample complexity when ε is sufficiently small.
New algorithms tackle RKHS bandits with reduced complexity and improved performance.
problem Adversarial and stochastic RKHS bandit problems with high computational complexity.
method Combining approximation theory with misspecified linear bandit methods.
result First general algorithm for adversarial RKHS bandit problem.
Unified formulation bridges adversarial and nonstationary bandits.
problem Handling time-varying reward distributions in multi-armed bandit problems.
method Unified oracle that switches between adversarial and nonstationary bandit oracles based on window size.
result Optimal regret achieved with matching lower bound.
Paper studies attacks on bandit algorithms and shows how attackers can manipulate data to hijack behavior.
problem Potential attacks on bandit algorithms can cause catastrophic loss in real-world applications.
method Proposes a framework of offline and online attacks on bandit algorithms using convex optimization and adaptive strategies.
result Attackers can force bandit algorithms to pull target arms with high probability by manipulating data.
Paper solves stochastic contextual linear bandits using linear bandit algorithms.
problem Stochastic contextual linear bandits with unknown context distribution.
method Establishes a reduction framework to convert to linear bandit problems.
result Achieves nearly optimal regret bound of O ( d T log T ) O(d\sqrt{T\log T}) O ( d T log T ) . Paper tackles LDP bandits learning with improved results and sub-linear regret.
problem Contextual bandits learning with LDP privacy constraints.
method Simple black-box reduction frameworks for context-free bandits, extended to GLB.
result First result for BCO with multi-point feedback under LDP, sub-linear regret for GLB.
Graph-Triggered Bandits unify rested and restless bandits with graph-defined arm interactions.
problem Modeling sequential decision-making problems with evolving arm rewards.
method Graph-Triggered Bandits (GTBs) framework that generalizes rested and restless bandits using a graph.
result Rested and restless bandits are special cases of GTBs for suitable graphs.
New definition resolves ambiguity in non-stationary bandit classification.
problem Ambiguity in classifying non-stationary bandits using existing definitions.
method Introducing a formal definition that resolves ambiguity and provides a unified approach.
result Unified approach applicable to both Bayesian and frequentist formulations, resolves classification issues.
A new algorithm for social network recommendations using side-observations.
problem Designing recommendation algorithms for users influenced by their social network.
method Contextual bandits with side-observations modeled by a social network graph.
result The proposed algorithm achieves asymptotically optimal regret, matching the lower-bound as T o ∞ T o \infty T o ∞ . Unified approach for non-stationary and clustered bandits.
problem Solving non-stationary and clustered bandits with overlapping solutions.
method Test of homogeneity for seamless integration of non-stationary and clustered bandits.
result Unified solution framework for change detection and cluster identification.
A framework for auto-tuning hyper-parameters in contextual bandit algorithms.
problem Auto-tuning hyper-parameters in real-time for contextual bandit algorithms.
method Proposes a Syndicated Bandits framework to learn multiple hyper-parameters dynamically.
result Achieves optimal regret bounds under certain scenarios and handles multiple contextual bandit algorithms.
New algorithm learns optimal exploration parameters for contextual bandits.
problem Learning optimal exploration in contextual bandits.
method Proposes two algorithms that learn optimal exploration parameters online based on context and reward.
result Demonstrates improved performance in learning optimal exploration compared to traditional methods.
Investigates sequential problems on graph structures and large action spaces.
problem Sequential decision-making on graph structures and large action spaces.
method Spectral bandits, side observations, influence maximization, kernel bandits, polymatroid bandits, function optimization, infinitely many-arms bandits.
result Contributions to graph and structured bandits.
A new method for dueling bandits improves performance.
problem Improving decision-making in dueling bandits.
method Sup-KLUCB method for K-armed dueling bandits, converting Copeland dueling bandits into standard MAB problems.
result Sup-KLUCB outperforms state-of-the-art methods in Copeland dueling bandits.
Study on indexability of restless multi-armed bandits and rollout policy performance.
problem Maximizing discounted rewards in finite state restless multi-armed bandit problems.
method Decouple the problem into single-armed restless bandits, analyze using value iteration, and compare with Whittle index policy.
result Demonstrates conditions for indexability and compares performance of index policy and rollout policy.
New insights into multi-armed bandits with budget constraints.
problem Multi-armed bandits with supply/budget constraints.
method Characterization of logarithmic regret rates, simple regret, and reduction to other bandit problems.
result Full characterization of logarithmic, instance-dependent regret rates for BwK.
Mirror descent linked to information ratio via Bayesian regret bounds.
problem Understanding stability in mirror descent and its relation to information ratio.
method Developed a connection between mirror descent and information ratio using Bayesian regret bounds.
result Mirror descent with suitable estimators and distributions achieves bounds similar to information-directed sampling.
Optimal algorithm for identifying best arm in stochastic linear bandits with fixed confidence.
problem Identifying the best arm in stochastic linear bandits with fixed confidence.
method Extending an algorithm designed for Best Arm Identification to the ε ε ε -Thresholding Bandit Problem (TBP). result Asymptotically optimal algorithm for TBP.
New algorithm for nonstationary GLBs reduces computation and memory costs.
problem Nonstationary generalized linear bandits with unknown time-varying parameters.
method Discounted Online Mirror Descent (DOMD) for parameter estimation.
result Dynamic regret bounds of order O ( 1 ) O(1) O ( 1 ) per round in drifting and piecewise-stationary environments. A meta-UCB method combines stochastic bandit algorithms.
problem Combining multiple stochastic bandit algorithms efficiently.
method Meta-UCB procedure solving an N-armed bandit problem.
result Final regret depends only on the best base algorithm's regret.