Optimistic algorithms achieve logarithmic regret bounds for MDPs without diameter dependence.
problem Achieving logarithmic regret bounds for episodic MDPs without relying on diameter-like quantities.
method Novel 'clipped' regret decomposition applied to optimistic algorithms.
result Smooth interpolation between gap-dependent and minimax rates of convergence.
The paper analyzes Q-learning in 2-player Markov games and provides gap-dependent logarithmic regret bounds.
problem Analyzing the cumulative regret of Nash Q-learning in 2-player turn-based stochastic Markov games.
method Proposed gap-dependent logarithmic upper bounds for cumulative regret in episodic tabular setting and discounted game setting.
result The proposed bounds match theoretical lower bounds up to a logarithmic term.
New bounds on minimax regret for sequential probability assignment using logarithmic loss.
problem Minimizing regret in sequential probability assignment against arbitrary experts.
method Using self-concordance property of logarithmic loss to derive tight bounds.
result Tight bounds on minimax regret for various expert classes.
Logarithmic regret for continuous-time reinforcement learning.
problem Continuous-time Markov decision processes with unknown transition probabilities and holding times.
method Upper confidence reinforcement learning, mean holding time estimation, stochastic comparison of point processes.
result Logarithmic regret bound achieved in finite time.
Logarithmic regret achieved in Q-learning with positive gap.
problem Achieving logarithmic cumulative regret in Q-learning with positive sub-optimality gap.
method Optimistic Q-learning with logarithmic regret bound.
result Logarithmic cumulative regret bound proven for optimistic Q-learning.
New framework reduces minimax regret for high-dimensional data.
problem Minimizing regret in high-dimensional data with logarithmic loss.
method Developed envelope complexity framework and spike-and-tails prior.
result Achieves minimax regret within a factor of two over high-dimensional ℓ 1 \ell_1 ℓ 1 -balls. Logarithmic regret achieved in RL with linear function approximation.
problem Achieving logarithmic regret in reinforcement learning with linear function approximation.
method LSVI-UCB for linear MDP assumption, UCRL-VTR for linear mixture MDP assumption.
result Logarithmic regret bounds established for RL with linear function approximation.
Optimizes quantile and semi-adversarial regret with novel root-logarithmic regularizers.
problem Minimizes regret in adversarial and semi-adversarial online learning.
method FTRL with root-logarithmic regularizers for quantile and semi-adversarial settings.
result Achieves minimax optimal regret bounds in both paradigms.
New algorithm achieves logarithmic regret for adversarial online control.
problem Online linear-quadratic control in systems with adversarial disturbances.
method Characterization of optimal offline control law, reduced to online learning with approximate advantage functions.
result First algorithm with logarithmic regret for arbitrary adversarial disturbance sequences.
New algorithm reduces regret in bandits with occasional free observations.
problem Reducing regret in bandit problems with occasional free observations.
method Developed an algorithm with a regret bound of Σ_i (log(1/ε) / Δ_i) up to constants and loglog terms.
result Proved that the algorithm's regret is optimal, matching lower bounds.
Paper analyzes and improves adaptive gradient methods for optimization.
problem Improving optimization methods for deep neural networks.
method Analyzes and proposes variants of RMSProp and Adagrad for online convex optimization.
result Proposes SC-Adagrad and SC-RMSProp with logarithmic regret bounds for strongly convex functions.
New algorithms achieve logarithmic regret in learning linear quadratic control systems.
problem Learning in Linear Quadratic Control systems with unknown parameters.
method Efficient algorithms for two scenarios: unknown A A A or B B B with certain conditions. result Regret scales logarithmically with the number of steps, not square root.
New bounds for Bayesian bandits show prior improves performance.
problem Improving regret bounds for Bayesian bandits.
method Upper confidence bound algorithm with finite-time logarithmic regret bounds.
result Derives O ( c Δ log n ) O(c_Δ\log n) O ( c Δ log n ) and O ( c h log 2 n ) O(c_h \log^2 n) O ( c h log 2 n ) upper bounds for Bayesian bandits. New bounds for online portfolio selection without smoothness assumptions.
problem Online portfolio selection with non-Lipschitz, non-smooth losses.
method Data-dependent bounds using novel smoothness characterizations and FTRL with self-concordant regularizers.
result Achieves logarithmic regrets when data is 'easy' and sublinear worst-case regrets.
New Thompson sampling algorithm for stochastic partial monitoring achieves logarithmic regret.
problem Limited feedback in sequential learning problems.
method Developed a novel Thompson-sampling-based algorithm to sample from the posterior distribution exactly.
result Achieved logarithmic regret bound of O(log T) for a linearized variant of the problem.
Paper proposes FedQ-Advantage for federated Q-learning with near-optimal regret and low communication cost.
problem Near-optimal federated Q-learning with low communication cost.
method Reference-advantage decomposition for variance reduction, synchronization between agents and server, policy update.
result Achieves almost optimal regret and near-linear regret speedup compared to single-agent learning.
Near-optimal regret in distributed bandit learning with efficient communication protocols.
problem Minimizing total regret in collaborative bandit learning with limited communication.
method Proposed communication protocols for distributed multi-armed and linear bandits with near-optimal regret and efficient communication costs.
result Achieved near-optimal regret with communication costs independent of time horizon and number of arms.
Study on regret minimization in deterministic MDPs.
problem Minimizing regret in deterministic reinforcement learning.
method Logarithmic regret lower bounds, leveraging graph theory and cycles.
result Explicitly quantifies the fundamental limit of performance achievable by any learning algorithm.
Logarithmic regret achieved in continuous-time linear-quadratic reinforcement learning.
problem Optimizing control actions in unknown continuous-time systems over a finite time horizon.
method Least-squares algorithm based on continuous-time observations and controls, with perturbation analysis and parameter estimation error analysis.
result Logarithmic regret bound of order O ( ( ln M ) ( ln ln M ) ) O((\ln M)(\ln\ln M)) O (( ln M ) ( ln ln M )) . New method reduces multi-armed bandit regret to near-optimal levels.
problem Improving regret bounds for KL-regularized multi-armed bandits.
method Sharp analysis of KL-UCB with peeling argument.
result First high-probability regret bound with linear dependence on K.
Study on individual regret in cooperative MAB with agents communicating over a graph.
problem Individual regret in cooperative stochastic multi-armed bandits with communication constraints.
method Analyzed COOP-SE algorithm, derived individual regret bounds under various communication constraints.
result First to show an individual regret bound in cooperative stochastic MAB independent of graph diameter.
Algorithm achieves logarithmic regret with sublinear hints.
problem Online linear optimization with limited hints.
method Using logarithmic hints to improve regret from sqrt(T) to log(T).
result O(log T) regret with O(sqrt(T)) hints, and O(sqrt(T)) regret with o(sqrt(T)) hints.
Paper analyzes and improves KL-regularized RL for LLMs with logarithmic regret.
problem Improving efficiency of RL fine-tuning for large language models.
method Optimism-based KL-regularized online contextual bandit algorithm with novel regret analysis.
result Achieves an O ( η log ( N R T ) ⋅ d R ) \mathcal{O}\big(η\log (N_{\mathcal R} T)\cdot d_{\mathcal R}\big) O ( η log ( N R T ) ⋅ d R ) logarithmic regret bound. This paper is devoted to regret lower bounds in the classical model of stochastic multi-armed bandit. A well-known result of Lai and Robbins, which has then been extended by Burnetas and Katehakis, has established the presence of a logarithmic bound for all consistent policies. We relax the notion of consistence, and e…
New algorithm reduces bandit problem's regret bound to logarithmic in dimension.
problem Sparse linear bandit problem with sparse reward structure.
method Proposes an algorithm that uses compatibility condition on optimal arm.
result Achieves regret bound of O(poly log dT) without additional diversity assumptions.
New algorithm reduces regret from sqrt(T) to polylog(T) in stochastic contextual linear bandits.
problem Achieving logarithmic regret in stochastic contextual linear bandits.
method Low Regret Stochastic Contextual Bandits ( exttt{LR-SCB}) algorithm, exploiting stochastic contexts and parameter estimation.
result Logarithmic regret (polylog(T)) achieved, improving over sqrt(T) lower bound.
FP-UCB algorithm achieves bounded regret for finitely parameterized multi-armed bandits.
problem Finitely parameterized multi-armed bandits with unknown but known parameter set.
method FP-UCB algorithm using structural information about the parameter set.
result FP-UCB achieves bounded regret under structural condition, logarithmic otherwise.
Unified analysis of online optimization with self-concordant barriers, improving regret bounds.
problem Online convex optimization with specific loss functions.
method Online mirror descent with self-concordant barriers and logarithmic loss.
result Improved regret bounds for online portfolio selection and quantum state learning.
New algorithms for batched dueling bandits with improved regret bounds.
problem Batched dueling bandits with noisy pairwise comparisons.
method Developed algorithms for two settings: Condorcet winner and strong stochastic transitivity.
result Regret bounds match sequential bounds using only a logarithmic number of batches.
Study minimax regret in sequential probability assignment with and without side information.
problem Minimax regret analysis in sequential probability assignment.
method Upper and lower bounds on minimax regret using square-root entropy.
result Lower bound matches upper bound for Donsker classes, up to log factors.
New algorithm reduces regret in linear bandits by nearly optimal factors.
problem Optimizing regret in linear contextual bandits with limited actions.
method Variable-Confidence-Level (VCL) SupLinUCB algorithm.
result Regret matches minimax lower bound with iterated logarithmic factors.
The paper sets bounds on how much regret is unavoidable in adaptive LQR with unknown B-matrix.
problem Understanding the limits of adaptive LQR with unknown B-matrix.
method Local asymptotic minimax regret lower bounds using van Trees' inequality and Bellman error representation.
result Logarithmic regret is impossible if the parametrization induces an uninformative optimal policy.
Study sparsity benefits in infinite feature contextual bandits.
problem Minimizing regret in infinite feature contextual bandits.
method Novel reduction to multi-armed bandits, Feel-Good Thompson Sampling algorithm.
result Regret bounds match lower bounds up to logarithmic factors, logarithmic dependence on effective features.
Study noise-free kernel bandits, finding upper bounds on regret.
problem Optimizing unknown functions without noise.
method Upper bounds on regret for noise-free kernel-based bandits.
result No order optimal regret bounds are established, conjecture on optimal bound.
Proposes a general method to derive regret bounds for multi-armed bandit algorithms.
problem Deriving regret bounds for randomized multi-armed bandit algorithms.
method Checking sufficient conditions on sampling probabilities and distributions.
result Proves logarithmic regret bounds for various bandit algorithms and new models.
Logarithmic regret strategies for safe multi-armed bandits with safety risk constraints.
problem Maximizing reward while avoiding unsafe arms under safety risk constraints.
method Doubly optimistic strategies with pseudo-regret formulation.
result Logarithmic regret bounds for safe multi-armed bandits.
Bandit algorithms struggle with consistent performance and robustness.
problem Achieving consistent and robust performance in stochastic multi-armed bandit settings.
method Analyzing regret minimization trade-offs and proposing distribution-oblivious algorithms.
result Logarithmic regret is inconsistent and super-logarithmic regret is necessary for consistent learning.
Study proves a tight lower bound for MNL-Bandit assortment selection problems.
problem Dynamic assortment planning under MNL bandit model with capacity constraints.
method Proved a tight lower bound on accumulated regret for all parameters.
result Tight lower bound matches existing upper bounds up to logarithmic factors.
Paper achieves logarithmic regret for online Kalman filter learning.
problem Predicting observations from an unknown, partially observed linear system with stochastic noise.
method Online least-squares algorithm exploiting the approximate linearity of Kalman filter predictions.
result Achieves regret of order poly(log(N)) with high probability.
New algorithm reduces regret in asynchronous multiplayer bandits to constant or logarithmic levels.
problem Asynchronous multiplayer bandits in cognitive radio networks.
method Cautious Greedy algorithm with O ( T log ( T ) ) \mathcal{O}(\sqrt{T\log(T)}) O ( T log ( T ) ) minimax regret. result Cautious Greedy yields constant instance-dependent regret under certain conditions.
Greedy algorithm achieves sublinear regret for various distributions.
problem Efficient performance of greedy algorithms in linear contextual bandit problems.
method Introduced Local Anti-Concentration (LAC) condition to ensure sublinear regret.
result Greedy algorithm achieves O ( poly log T ) O(\operatorname{poly} \log T) O ( poly log T ) cumulative expected regret. A decentralized policy achieves logarithmic regret for multi-agent MAB problems with communication constraints.
problem Decentralized policy for multi-agent MAB problems with option availability and communication constraints.
method Upper Confidence Bound (UCB) algorithms with non-stationary stochastic communication protocol.
result Guaranteed logarithmic regret for non-fully connected spatial graphs with communication constraints.
New insights into multi-armed bandits with budget constraints.
problem Multi-armed bandits with supply/budget constraints.
method Characterization of logarithmic regret rates, simple regret, and reduction to other bandit problems.
result Full characterization of logarithmic, instance-dependent regret rates for BwK.
Paper proposes DG-ETC for online submodular maximization with stochastic bandit feedback.
problem Online unconstrained submodular maximization with stochastic bandit feedback.
method Double-Greedy - Explore-then-Commit (DG-ETC) approach.
result DG-ETC achieves logarithmic regret O ( d log ( d T ) ) O(d\log(dT)) O ( d log ( d T )) for 1 / 2 1/2 1/2 -approximate pseudo-regret. Improved regret bounds for bandits with expert advice.
problem Optimizing decision-making in environments with expert advice.
method Proved lower and upper bounds for regret in restricted and standard feedback models.
result Proved a new upper bound of order K T ln ( N / K ) \sqrt{K T \ln(N/K)} K T ln ( N / K ) for the worst-case regret, matching a previously known lower bound. Study gap-dependent regret bounds for risk-sensitive RL.
problem Risk-sensitive reinforcement learning with entropic risk measure.
method Propose cascaded gaps to adapt to problem structures, derive regret bounds.
result Exponential improvement over existing bounds in appropriate settings.
New algorithm exploits curvature of feasible sets for fast online convex optimization.
problem Online convex optimization with fast rates.
method Adapting FTL algorithm to curvature of feasible sets.
result Achieves logarithmic regret bound of O ( ρ log T ) O(ρ\log T) O ( ρ log T ) in stochastic environments. The problem of distributed learning and channel access is considered in a cognitive network with multiple secondary users. The availability statistics of the channels are initially unknown to the secondary users and are estimated using sensing decisions. There is no explicit information exchange or prior agreement amon…