IDS improves sparse linear bandits by balancing information and regret.
problem Sparse linear bandits in high-dimensional decision-making.
method Information-directed sampling (IDS) with Bayesian regret bounds and empirical Bayesian sparse posterior sampling.
result IDS nearly matches existing lower bounds and significantly reduces regret.
New bounds for high-dimensional sparse linear bandits, balancing information and regret.
problem Stochastic linear bandits with high-dimensional sparse features.
method Derivation of minimax regret lower and upper bounds for explore-then-commit algorithm.
result Optimal rate of Θ ( n 2 / 3 ) Θ(n^{2/3}) Θ ( n 2/3 ) for data-poor regime, complemented by O ( n ) O(\sqrt{n}) O ( n ) under signal magnitude assumption. New method reduces regret for sparse adversarial SSP problems.
problem Sparse adversarial Stochastic Shortest Path problem.
method Proposed ℓ r \ell_r ℓ r -norm regularizers for adaptive sparsity. result Regret scales with log M \sqrt{\log M} log M instead of log S A \sqrt{\log SA} log S A . New algorithm minimizes regret in sparse reinforcement learning.
problem Sparse reinforcement learning with unknown sparsity.
method Doubly robust approach combining feature vectors of all actions and novel analysis.
result Regret bound of i l d e O ( σ min − 1 s ⋆ H N ) ilde{O}(σ^{-1}_{\min} s_{\star} H \sqrt{N}) i l d e O ( σ m i n − 1 s ⋆ H N ) . This paper considers portfolio construction in a dynamic setting. We specify a loss function comprised of utility and complexity components with an unknown tradeoff parameter. We develop a novel regret-based criterion for selecting the tradeoff parameter to construct optimal sparse portfolios over time.
Paper improves sparse linear bandits by accounting for noise variance.
problem Sparse linear bandits with unknown noise variance.
method Develops a general framework to convert variance-aware algorithms to sparse linear bandits.
result Achieves $\widetilde{\mathcal O}\left(\sqrt{d\sum_{t=1}^T σ_t^2} + 1
ight)$ regret, interpolating between worst-case and benign settings.
New algorithm reduces bandit problem's regret bound to logarithmic in dimension.
problem Sparse linear bandit problem with sparse reward structure.
method Proposes an algorithm that uses compatibility condition on optimal arm.
result Achieves regret bound of O(poly log dT) without additional diversity assumptions.
FLIPHAT addresses joint differential privacy for high-dimensional sparse linear bandits.
problem Efficient sequential decision-making with high-dimensional sparse features and privacy concerns.
method FLIPHAT combines iterative forgetting and N-IHT for sparse linear regression, achieving optimal regret.
result FLIPHAT achieves optimal regret in terms of privacy parameters, context dimension, and time horizon.
VO Q Q Q L optimizes RL with sparse rewards using weighted bounds.
problem Sparse rewards and non-linear function approximation in RL.
method VO Q Q Q L combines Q Q Q -learning with weighted bounds for optimal regret. result Achieves asymptotically optimal regret for linear function approximation.
Improved online Lasso reduces regret in sparse linear contextual bandits.
problem Sparse linear contextual bandit problem with inefficient sampling.
method Perturbed adversary approach to alleviate sampling inefficiency.
result Online Lasso achieves O ( k T log d ) \mathcal{O}(\sqrt{kT\log d}) O ( k T log d ) regret bound. New method for linear bandits with unknown sparsity, improving sparse regret bounds.
problem Sparse regret bounds for unknown sparsity and adversarial action sets.
method Combines online to confidence set conversions with randomized model selection over nested confidence sets.
result First sparse regret bounds for unknown sparsity and adversarial action sets.
Thresholded Lasso bandit minimizes regret in sparse linear bandits.
problem Sparse stochastic contextual linear bandits with large feature vectors.
method Uses Lasso framework with thresholding to estimate reward function and its sparse support.
result Non-asymptotic regret upper bounds scaling as O ( log d + T ) \mathcal{O}( \log d + \sqrt{T}) O ( log d + T ) . Paper addresses regret minimization and inference in high-dimensional online decision-making.
problem Regret minimization and statistical inference in high-dimensional online decision-making.
method Integrates ε-greedy bandit algorithm with hard thresholding for sparse bandit parameters and debiasing method for inference.
result Achieves either O ( T 1 / 2 ) O(T^{1/2}) O ( T 1/2 ) regret or O ( T 1 / 2 ) O(T^{1/2}) O ( T 1/2 ) -consistent inference, with trade-off between exploration and exploitation. A new algorithm reduces frequentist regret in multi-agent bandit problems with sparse hypergraphs.
problem Deriving a frequentist regret bound for Thompson sampling in multi-agent settings with sparse hypergraphs.
method Proposed ε ε ε -exploring Multi-Agent Thompson Sampling ( ε ε ε -MATS) algorithm that combines exploration and exploitation strategies. result Achieves a worst-case frequentist regret bound sublinear in time horizon and local arm size, optimal up to constants and logarithms for sparse hypergraphs.
Study shows high-dimensional sparse RL hardness and Lasso Q-iteration's nearly dimension-free regret.
problem Hardness of online sparse reinforcement learning in high-dimensional MDPs.
method Lower bound construction and Lasso fitted Q-iteration analysis.
result Lasso Q-iteration achieves nearly dimension-free regret of O ~ ( s 2 / 3 N 2 / 3 ) \tilde{O}(s^{2/3}N^{2/3}) O ~ ( s 2/3 N 2/3 ) with oracle access to a good exploratory policy. New algorithm reduces dynamic regret by adapting to comparator complexity.
problem Nonstationary sequential decision making with unbounded domains.
method Sparse coding framework to adapt to comparator complexity.
result Improves dynamic regret bounds by adapting to comparator energy and sparsity.
New algorithm reduces semi-bandit regret using covariance estimates.
problem Complexity of semi-bandits due to joint distribution of outcomes.
method Develops a new sub-exponential distribution family and an algorithm using covariance estimates.
result Proves a new lower bound on expected regret and constructs an algorithm with asymptotic analysis.
New algorithms minimize regret in SSP with optimal sparse updates.
problem Minimizing regret in Stochastic Shortest Path models.
method Implicit finite-horizon approximation for analysis, model-free and model-based algorithms developed.
result Minimax optimal regret for both model-free and model-based algorithms.
Scalable TS using sparse GPs improves efficiency without sacrificing performance.
problem Efficiently applying TS to complex, multi-modal problems.
method Sparse Gaussian Process models for scalable TS.
result Theoretical and empirical validation of scalable TS's effectiveness.
PopArt efficiently solves sparse linear bandits with tighter recovery guarantees.
problem Sparse linear bandits where rewards depend on a few covariates.
method PopArt: a simple, computationally efficient sparse linear estimation method.
result Improved regret bounds compared to state-of-the-art algorithms.
Improved Thompson Sampling for high-dimensional sparse bandits.
problem Stochastic linear contextual bandits with high-dimensional features.
method Thompson Sampling with spike-and-slab priors and variational inference.
result Nearly optimal upper bound on expected cumulative regret.
Develops a method to tackle high-dimensional linear bandits with knapsacks using online sparse estimation and dual variables.
problem High-dimensional linear bandits with knapsacks.
method Online hard thresholding algorithm for sparse estimation, integrated with primal-dual scheme.
result Achieves sub-linear regret that scales logarithmically with feature dimension, improving on prior work.
New algorithm minimizes regret in multi-armed bandits with network interference.
problem Minimizing regret in online experiments with network interference.
method Sparse network interference model, discrete Fourier analysis, linear regression-based algorithms.
result Provable low regret algorithms for sparse interference networks.
A novel approach tackles sparse linear bandits with reduced communication costs and minimal cumulative regret.
problem Sparse linear bandits with high-dimensional feature vectors and limited relevant features.
method Cooperative Thresholded Lasso using Lasso and ridge regression for dimension reduction and aggregation.
result Regret bound of O ( s 0 log d + s 0 T ) \mathcal{O}(s_0 \log d + s_0 \sqrt{T}) O ( s 0 log d + s 0 T ) with high probability. Optimal algorithm for high-dimensional stochastic linear bandits with sparse parameters.
problem High-dimensional stochastic linear bandits with sparse parameters.
method Three-stage arm selection algorithm using thresholded Lasso for estimation.
result Achieves exact minimax optimality in cumulative regret.
Algorithm achieves comparable performance to fully dynamic data with only a few batches.
problem High-dimensional multi-armed contextual bandits with batched feedback.
method Provable sample-efficient algorithm using batch allocation method.
result Achieves regret bounds comparable to fully sequential setting with only L = O(log T) batches.
Improves content allocation in educational platforms with sparse data.
problem Imbalanced content allocation and delayed convergence in adaptive strategies.
method Introduces WAPTS, an algorithm that refines Thompson Sampling for data-sparse environments.
result Demonstrates earlier and more reliable identification of promising treatments.
Study dynamic batch learning in high-dimensional sparse linear bandits.
problem Dynamic batch learning in high-dimensional sparse linear contextual bandits under batch constraints.
method Characterized fundamental learning limits via regret lower bound and provided matching upper bound.
result Prescribed an optimal scheme for dynamic batch learning in high-dimensional sparse linear contextual bandits.
New algorithm learns from sparse data without knowing sparsity index.
problem Sparse bandit problem where only a subset of features affects reward.
method Sparsity-agnostic Lasso Bandit algorithm that doesn't require prior sparsity index knowledge.
result Established tight regret bounds and outperforms existing methods.
A new method for sparse linear bandits reduces exploration-exploitation tradeoff.
problem Sparse linear bandits in high-dimensional settings with finite actions.
method Best subset selection for parameter estimation and doubly growing epochs for regret minimization.
result Achieves nearly dimension-independent regret of i l d e O ( s T ) ilde{\mathcal{O}}(s\sqrt{T}) i l d e O ( s T ) with high probability. New research shows no-regret learning is impossible in Markov games under certain assumptions.
problem Achieving no-regret learning in decentralized Markov games.
method Novel application of aggregation techniques from online learning to prove lower bounds.
result No polynomial-time algorithm exists for independent no-regret learning in general-sum Markov games.
AIHT improves online high-dimensional quantile regression by separating support discovery and refinement.
problem Online high-dimensional quantile regression with structural sparsity.
method Adaptive Iterative Hard Thresholding (AIHT) alternates stochastic updates with adaptive hard-thresholding steps.
result AIHT achieves logarithmic regret for the sliding-window objective in high-dimensional settings.
Optimal multitask learning method for sparse heterogeneous datasets.
problem Efficiently learning from multiple related datasets with sparse task-specific differences.
method MOLAR estimator, combining weighted median and shrinkage.
result Improves estimation error dependence on data dimension compared to task-wise least squares.
The paper tackles adaptive targeting in networks with interference effects.
problem Adaptive targeting under network interference in a bandit setting.
method Linear model in a sparse regime, analyzing different levels of knowledge of the interference structure.
result Unified view of how knowledge of the interference structure affects online learning efficiency.
We propose the first reduction-based approach to obtaining long-term memory guarantees for online learning in the sense of Bousquet and Warmuth, 2002, by reducing the problem to achieving typical switching regret. Specifically, for the classical expert problem with K K K actions and T T T rounds, using our framework we dev…
New algorithm tackles multi-agent bandits with heavy-tailed data.
problem Maximizing system performance in multi-agent settings with heavy-tailed data.
method Algorithm exploits hub-like structures and synchronization among clients.
result Regret bound of O ( M 1 − 1 α log T ) O(M^{1 -\frac{1}α} \log{T}) O ( M 1 − α 1 log T ) for homogeneous settings, O ( M log T ) O(M \log{T}) O ( M log T ) for heterogeneous. New bounds show polyhedral surrogates are optimal for generalization.
problem Proving generalization rates for polyhedral loss functions.
method Developed two general results for polyhedral surrogates.
result Polyhedral surrogates provide linear surrogate regret bounds, translating directly to target rates.
SGD performs optimally on dependent data, achieving both estimation and regret.
problem Performance of SGD on temporally dependent data.
method Stochastic gradient descent with martingale-type and sequential dependence.
result SGD achieves optimal estimation and regret under various dependence structures.
New algorithms for generalized linear bandits with unknown reward functions.
problem Misspecification of reward functions in existing bandit algorithms.
method Introducing single index bandits, proposing STOR, ESTOR, and GSTOR algorithms.
result Achieved nearly optimal regret bound of i l d e O T ( T ) ilde{O}_T(\sqrt{T}) i l d e O T ( T ) . A new metric, Weighted Regret, unifies FDR and power evaluation in online multiple testing.
problem The asymmetric costs of false positives and false negatives in automated pipelines.
method Introducing Weighted Regret and Decoupled-OMT (DOMT) to unify FDR and power evaluation.
result DOMT achieves an order-optimal sublinear mitigation of threshold depletion in bursty environments.
A new algorithm reduces communication costs for collaborative decision-making across clients.
problem Collaborative decision-making with sparse rewards and heterogeneous contexts.
method Federated Lasso algorithm for sparse linear contextual bandits.
result Achieves near-optimal regret with logarithmic communication costs.
No-regret optimization for time-varying functions using uncertainty injection.
problem Optimizing time-varying functions with no-regret in bandit feedback.
method W-SparQ-GP-UCB, incorporating uncertainty injection and additional queries.
result Achieves no-regret with a vanishing number of additional queries per iteration.
We demonstrate that, in the classical non-stochastic regret minimization problem with d d d decisions, gains and losses to be respectively maximized or minimized are fundamentally different. Indeed, by considering the additional sparsity assumption (at each stage, at most s s s decisions incur a nonzero outcome), we derive…
A new algorithm STE for model-based RL improves learning rates.
problem Sparse rewards and computational intractability of estimating information gain.
method Developed a novel algorithm based on Stein Information Directed Exploration (STE)E.
result Achieves sublinear Bayesian regret, outperforming prior approaches.
Algorithm reduces regret in distributed kernel bandits with shared randomness.
problem Minimizing regret in collaborative function maximization.
method Uniform exploration at local agents and shared randomness with central server.
result Achieves optimal regret order with sublinear communication cost.
Study sparsity benefits in infinite feature contextual bandits.
problem Minimizing regret in infinite feature contextual bandits.
method Novel reduction to multi-armed bandits, Feel-Good Thompson Sampling algorithm.
result Regret bounds match lower bounds up to logarithmic factors, logarithmic dependence on effective features.
Algorithm optimizes collaborative learning among distributed clients using kernel-based bandits.
problem Optimizing personalized objectives in a distributed system with limited global information.
method Kernel-based bandit framework with surrogate Gaussian process models, sparse approximations.
result Order-optimal regret performance (up to polylogarithmic factors) and reduced communication overhead.
BONSAI optimizes parameters while respecting a default configuration, reducing unnecessary changes.
problem Standard BO pushes weakly relevant parameters to the boundary, making it hard to distinguish between important and spurious changes.
method BONSAI is a default-aware BO policy that prunes low-impact deviations from a default configuration while controlling acquisition value loss.
result BONSAI matches the GP-UCB regret rate while recovering the minimal- ℓ 0 \ell_0 ℓ 0 solution, reducing the number of non-default parameters in recommended configurations.