Improved regret bound for linear ensemble sampling.
problem Closing the gap between theory and practice in linear ensemble sampling.
method General regret analysis framework for linear bandit algorithms, revealing a relationship with LinPHE.
result Achieves a frequentist regret bound of i l d e O ( d 3 / 2 T ) ilde{O}(d^{3/2}\sqrt{T}) i l d e O ( d 3/2 T ) for linear ensemble sampling. New algorithm reduces regret from sqrt(T) to polylog(T) in stochastic contextual linear bandits.
problem Achieving logarithmic regret in stochastic contextual linear bandits.
method Low Regret Stochastic Contextual Bandits ( exttt{LR-SCB}) algorithm, exploiting stochastic contexts and parameter estimation.
result Logarithmic regret (polylog(T)) achieved, improving over sqrt(T) lower bound.
Logarithmic regret achieved in RL with linear function approximation.
problem Achieving logarithmic regret in reinforcement learning with linear function approximation.
method LSVI-UCB for linear MDP assumption, UCRL-VTR for linear mixture MDP assumption.
result Logarithmic regret bounds established for RL with linear function approximation.
We establish linear regret bounds for convex smooth losses using Fenchel-Young losses.
problem Establishing linear regret bounds for convex smooth losses.
method Constructing a convex smooth surrogate loss using Fenchel-Young losses generated by the convolutional negentropy.
result We derive a smooth loss with a linear surrogate regret bound.
The paper addresses frequentist regret of Linear Thompson Sampling in stochastic linear bandits.
problem The frequentist regret of Linear Thompson Sampling (LinTS) is worse than its Bayesian counterpart.
method The paper proves the fundamental nature of the frequentist regret bound for LinTS and proposes a data-driven version of LinTS to achieve minimax optimal frequentist regret.
result The frequentist regret bound for LinTS is O ~ ( d d T ) \widetilde{\mathcal{O}}(d\sqrt{dT}) O ( d d T ) , which is the best possible under certain conditions. Unified approach for non-stationary linear bandits with dynamic regret.
problem Non-stationary linear bandits with round-specific feasible actions and drifting reward models.
method Unified misspecification-reduction viewpoint, restarting algorithms with misspecification-dependent regret guarantees.
result Optimal \(T^{2/3}P_T^{1/3}\) dynamic-regret dependence for both linear bandits and contextual linear bandits.
IDS improves sparse linear bandits by balancing information and regret.
problem Sparse linear bandits in high-dimensional decision-making.
method Information-directed sampling (IDS) with Bayesian regret bounds and empirical Bayesian sparse posterior sampling.
result IDS nearly matches existing lower bounds and significantly reduces regret.
The paper tightens the regret rate for linear bandit problems.
problem Bayesian regret in linear bandit problems.
method Information-theoretic framework and chaining argument.
result Established a new bound with a tight rate of O ( d T ) O(d\sqrt{T}) O ( d T ) . Algorithm reduces regret in SSP problems with LFA.
problem Finding shortest paths in stochastic environments with linear approximations.
method Uses linear function approximation and stationary policies to minimize regret.
result Achieves sublinear regret under minimal assumptions.
Randomized exploration in linear bandits achieves optimal regret bounds.
problem Optimizing exploration in high-dimensional linear bandit problems.
method Analysis of Thompson sampling without forced optimism.
result Randomized exploration algorithms achieve an O ( d n log ( n ) ) O(d\sqrt{n} \log(n)) O ( d n log ( n )) regret bound in smooth, strongly convex action spaces. This paper achieves optimal regret bounds for locally private linear contextual bandit.
problem Designing locally private linear contextual bandit algorithms with optimal regret bounds.
method New algorithmic and analytical ideas, including mean absolute deviation analysis and layered principal component regression.
result Achieves an i l d e O ( T ) ilde O(\sqrt{T}) i l d e O ( T ) regret upper bound for locally private linear contextual bandit. Study shows certainty equivalent policy minimizes regret in continuous-time systems.
problem Minimizing regret in continuous-time stochastic linear-quadratic systems.
method Theoretical analysis of randomized certainty equivalent policy.
result Establishes square-root of time regret bounds and linear scaling with parameters.
Significant improvements in regret analysis for adaptive online learning problems.
problem Exploiting low variance in online learning problems without known variances.
method Novel peeling-based regret analysis leveraging elliptical potential `count` lemma.
result Significant improvements in regret bounds for linear bandits and linear mixture MDPs.
New algorithm reduces regret in delayed feedback generalised linear bandits.
problem Regret in delayed feedback generalised linear bandits.
method Adaptation of optimistic algorithm to delayed feedback.
result Achieves a regret bound independent of the horizon's delay penalty.
This paper addresses robust CBs for linear SEMs with model fluctuations.
problem Designing interventions in causal systems with linear SEMs that are robust to model fluctuations.
method Develops a robust CB algorithm and analyzes its regret under model deviation.
result The proposed algorithm achieves nearly optimal i l d e O ( T ) ilde{\mathcal{O}}(\sqrt{T}) i l d e O ( T ) regret when C C C is o ( T ) o(\sqrt{T}) o ( T ) and maintains sub-linear regret for a broader range of C C C . We study the problem of regret minimization in partially observable linear quadratic control systems when the model dynamics are unknown a priori. We propose ExpCommit, an explore-then-commit algorithm that learns the model Markov parameters and then follows the principle of optimism in the face of uncertainty to desig…
New model-free algorithm achieves similar LQR regret guarantees.
problem Model-free control of linear dynamical systems under quadratic costs.
method Online policy gradient scheme with policy space cost analysis.
result Achieves regret scaling with √T, matching model-based methods.
TRAiL is a linear bandit algorithm that ensures optimal regret and guarantees inference quality.
problem Optimal regret and inference quality in linear bandits with convex action sets.
method TRAiL estimates the parameter through regularized least squares and perturbs the action set along the tangent plane.
result TRAiL achieves an Ω ( T ) Ω(\sqrt{T}) Ω ( T ) upper bound on cumulative regret with high probability. New approach reduces unconstrained linear bandits to simpler optimization problems.
problem Unconstrained linear bandits problem.
method Perturbation-based approach combined with comparator-adaptive OLO algorithms.
result First high-probability guarantees for both static and dynamic regret in unconstrained linear bandits.
New method reduces linear regret in high-dimensional bandit problems.
problem Heavy spectral tails in streaming matrices lead to linear regret in sketch-based linear bandits.
method Dyadic Block Sketching, a multi-scale matrix sketching approach.
result Achieves sublinear regret bounds without prior knowledge of streaming matrix properties.
New algorithms for learning MDPs with linear approximations in infinite-horizon settings.
problem Learning infinite-horizon average-reward MDPs with linear function approximation.
method Optimism principle, adversarial linear bandits, Natural Policy Gradient.
result Efficient algorithms with optimal or near-optimal regret bounds.
Safe RL in linear systems achieves T \sqrt{T} T -regret.
problem Efficiently learning in safety-constrained online reinforcement learning.
method Study of linear quadratic regulator with safety constraints.
result First safe algorithm with i l d e O T ( T ) ilde{O}_T(\sqrt{T}) i l d e O T ( T ) -regret. New algorithm learns optimal path in reinforcement learning with linear approximations.
problem Optimal path learning in reinforcement learning with linear approximations.
method Proposes novel algorithm with Hoeffding-type and Bernstein-type confidence sets.
result Achieves near-optimal regret guarantee for linear mixture SSP.
E 4 ^4 4 algorithm optimizes batched linear bandits with minimal regret and batches.
problem Optimizing batched linear bandits for minimal regret.
method Explore-Estimate-Eliminate-Exploit framework with optimal exploration rate.
result Achieves minimax and asymptotic optimality in regret and batch complexity.
New algorithm reduces regret for linear bandits with unknown noise variance.
problem Finding optimal actions in linear bandits with varying noise variance.
method Adaptive algorithm with Freedman-type concentration inequality and multi-layer structure.
result Achieves i l d e O ( d ∑ k = 1 K σ k 2 + d ) ilde{O}(d \sqrt{\sum_{k = 1}^K σ_k^2} + d) i l d e O ( d ∑ k = 1 K σ k 2 + d ) regret for linear bandits. New algorithms for linear bandits avoid norm knowledge, reducing regret.
problem Linear bandits require knowledge of norm bound S S S on parameter θ ∗ θ^* θ ∗ , leading to high regret. method Proposes two novel algorithms for changing and fixed arm sets, analyzing their regret bounds.
result Regret bounds show no significant price for not knowing S S S , with no price for fixed arm sets. We consider adaptive control of the Linear Quadratic Regulator (LQR), where an unknown linear system is controlled subject to quadratic costs. Leveraging recent developments in the estimation of linear systems and in robust controller synthesis, we present the first provably polynomial time algorithm that provides high…
New algorithm reduces regret in CBs with time-varying models.
problem Designing robust interventions in CBs with unknown, fluctuating causal models.
method Proposes a robust CB algorithm with upper and lower bounds on regret.
result Achieves nearly optimal i l d e O ( T ) ilde{\mathcal{O}}(\sqrt{T}) i l d e O ( T ) regret under certain conditions. Greedy algorithm achieves sublinear regret for various distributions.
problem Efficient performance of greedy algorithms in linear contextual bandit problems.
method Introduced Local Anti-Concentration (LAC) condition to ensure sublinear regret.
result Greedy algorithm achieves O ( poly log T ) O(\operatorname{poly} \log T) O ( poly log T ) cumulative expected regret. Paper eliminates warm-up phase for PO in linear MDPs, achieving optimal regret.
problem Costly warm-up phase in PO algorithms for linear MDPs.
method Simple contraction mechanism replaces warm-up phase.
result Achieves rate-optimal regret with improved dependence on problem parameters.
This paper improves online learning algorithms for LP problems, achieving better regret bounds.
problem Achieving optimal regret bounds in online linear programming.
method Develops a new framework for first-order online learning algorithms under certain error bound conditions.
result First-order learning algorithms achieve o ( T ) o(\sqrt{T}) o ( T ) regret in continuous support and O ( log T ) \mathcal{O}(\log T) O ( log T ) regret in finite support, improving over O ( T ) \mathcal{O}(\sqrt{T}) O ( T ) . New algorithm reduces bandit regret to log^3(T).
problem Noise model for linear stochastic bandits with vanishing noise.
method Weighted least-squares estimation, leveraging eigenvalue relation.
result Minimax regret scaling as log^3(T) for time horizon T.
Algorithm balances online and offline data for linear bandits.
problem Online learning with an offline dataset in linear bandits.
method Proposes a linear bandit algorithm that uses offline data early and increasingly favors exploration as the horizon grows.
result Establishes regret bounds showing competitive performance with both purely online and offline solutions.
A new linear contextual bandit algorithm with improved regret bound.
problem Efficiently solving linear contextual bandit problems with reduced regret.
method Proposes a novel estimator embedded with exploration and a self-normalized bound.
result Regret bound matches lower bound of Ω ( d T ) Ω(\sqrt{dT}) Ω ( d T ) up to logarithmic factors. Bayesian bandit algorithms with approximate inference improve regret bounds in stochastic linear bandits.
problem Theoretical justification for Bayesian bandit algorithms with approximate inference in stochastic linear bandits.
method Proposed a theoretical framework to analyze approximate inference impact and conducted frequentist regret analysis on LinTS and LinBUCB.
result LinTS and LinBUCB preserve their original regret upper bounds with larger constant terms in approximate inference settings.
We study optimal regret bounds for control in linear dynamical systems under adversarially changing strongly convex cost functions, given the knowledge of transition dynamics. This includes several well studied and fundamental frameworks such as the Kalman filter and the linear quadratic regulator. State of the art met…
New method reduces ensemble size for linear bandits, achieving near optimal regret.
problem Achieving near optimal regret in linear bandits with limited ensemble size.
method Ensemble sampling with a size of order d log T d \log T d log T for a d d d -dimensional stochastic linear bandit. result Regret is at most ( d log T ) 5 / 2 T (d \log T)^{5/2} \sqrt{T} ( d log T ) 5/2 T , improving over linear scaling with T T T . The paper minimizes Borda regret in dueling bandits models.
problem Minimizing Borda regret in dueling bandits models.
method Proposes explore-then-commit and EXP3-type algorithms for stochastic and adversarial settings respectively.
result Achieves nearly matching regret upper bounds of O ( d 2 / 3 T 2 / 3 ) O(d^{2/3} T^{2/3}) O ( d 2/3 T 2/3 ) for both settings. New bounds for high-dimensional sparse linear bandits, balancing information and regret.
problem Stochastic linear bandits with high-dimensional sparse features.
method Derivation of minimax regret lower and upper bounds for explore-then-commit algorithm.
result Optimal rate of Θ ( n 2 / 3 ) Θ(n^{2/3}) Θ ( n 2/3 ) for data-poor regime, complemented by O ( n ) O(\sqrt{n}) O ( n ) under signal magnitude assumption. New algorithms minimize regret in multi-task and lifelong linear bandits with shared representation.
problem Minimizing regret in multi-task and lifelong linear bandits with shared representation.
method Novel algorithms using efficient estimator for low-rank linear feature extractor and novel analysis.
result Achieved regret bounds matching minimax lower bound up to logarithmic factors.
New algorithm reduces control error in systems with changing dynamics.
problem Online control of systems with time-varying linear dynamics.
method Introduces adaptive regret metric and a novel meta-algorithm.
result First adaptive regret bound for online convex optimization with memory.
Algorithm finds optimal regularizers for online linear optimization.
problem Finding optimal regularizers to minimize regret in online linear optimization.
method Algorithm takes input sets and outputs an optimal regularizer for FTRL.
result Algorithm guarantees regret within a constant factor of the best possible learning algorithm.
New method achieves near-optimal regret without simulator.
problem Adversarial linear contextual bandits with unknown loss vectors.
method Near-optimally reduces regret to sqrt(T) without simulator.
result Achieves regret of sqrt(T) without simulator, improving existing methods.
New algorithm reduces regret with diverse contexts in bandits.
problem Impact of context diversity on stochastic linear contextual bandits.
method Design of LinUCB-d algorithm and analysis of its regret performance.
result Cumulative expected regret is bounded by a constant under diverse context assumption.
Balances and eliminates base algorithms in bandits and RL to bound total regret.
problem Model selection in bandits and reinforcement learning with unknown optimal regret.
method Balances and eliminates base algorithms based on candidate regret bounds.
result Total regret bound is the best valid candidate regret bound times a small multiplicative factor.
Two algorithms improve online reinforcement learning in adversarial linear MDPs with bandit feedback.
problem Online reinforcement learning in linear MDPs with adversarial losses and bandit feedback.
method Two algorithms: one computationally inefficient with $\widetilde{\mathcal{O}}\left(\sqrt{K}
ight)$ regret, and one computationally efficient with $\widetilde{\mathcal{O}}\left(K^{\frac{3}{4}}
ight)$ regret.
result Achieved improved regret performance compared to existing approaches.
Federated Q-Learning achieves linear regret speedup with low communication cost.
problem Achieving linear regret speedup in federated reinforcement learning without high communication costs.
method Proposed two federated Q-Learning algorithms: FedQ-Hoeffding and FedQ-Bernstein, using event-triggered synchronization, novel step size selection, and concentration inequalities.
result Total regrets achieve linear speedup compared to single-agent counterparts with logarithmic communication cost.
Study linear contextual bandits with confounded offline data, improving regret bounds.
problem Linear contextual bandits with confounded offline data.
method Construct a linear bandit algorithm that utilizes projected information.
result Proved regret bounds that improve current bounds by a factor related to visible dimensionality.