Unified framework for high-dimensional bandit problems with low-dimensional structures.
problem Stochastic high-dimensional bandit problems with low-dimensional structures.
method Proposed a simple unified algorithm and a general analysis framework for the regret upper bound.
result Unified algorithm achieves comparable regret bounds in various high-dimensional bandit problems.
Improved Thompson Sampling for high-dimensional sparse bandits.
problem Stochastic linear contextual bandits with high-dimensional features.
method Thompson Sampling with spike-and-slab priors and variational inference.
result Nearly optimal upper bound on expected cumulative regret.
A privacy-preserving algorithm for high-dimensional bandits.
problem High-dimensional stochastic contextual linear bandits with sparse parameters under privacy constraints.
method PrivateLASSO algorithm based on sparse hard-thresholding and episodic thresholding.
result Minimax private lower bounds and utility guarantees for PrivateLASSO.
Study dynamic batch learning in high-dimensional sparse linear bandits.
problem Dynamic batch learning in high-dimensional sparse linear contextual bandits under batch constraints.
method Characterized fundamental learning limits via regret lower bound and provided matching upper bound.
result Prescribed an optimal scheme for dynamic batch learning in high-dimensional sparse linear contextual bandits.
FLIPHAT addresses joint differential privacy for high-dimensional sparse linear bandits.
problem Efficient sequential decision-making with high-dimensional sparse features and privacy concerns.
method FLIPHAT combines iterative forgetting and N-IHT for sparse linear regression, achieving optimal regret.
result FLIPHAT achieves optimal regret in terms of privacy parameters, context dimension, and time horizon.
New bounds for high-dimensional sparse linear bandits, balancing information and regret.
problem Stochastic linear bandits with high-dimensional sparse features.
method Derivation of minimax regret lower and upper bounds for explore-then-commit algorithm.
result Optimal rate of Θ ( n 2 / 3 ) Θ(n^{2/3}) Θ ( n 2/3 ) for data-poor regime, complemented by O ( n ) O(\sqrt{n}) O ( n ) under signal magnitude assumption. SPARKLE handles high-dimensional covariates for online decision-making.
problem Complex reward-covariate relationships in high-dimensional settings.
method SPARKLE uses a sparse additive reward model with doubly penalized estimator and adaptive screening.
result SPARKLE achieves sublinear regret bound logarithmic in covariate dimensionality.
Unified approach tackles high-dimensional tensor bandits with convex optimization and weakly decomposable regularizers.
problem Challenges in high-dimensional generalized tensor bandits where existing algorithms fail.
method Proposes a generalized linear tensor bandits algorithm with a unified analytical framework using convex optimization and weakly decomposable regularizers.
result Unified analytical framework provides better bounds and broader applicability compared to existing methods.
Develops a method to tackle high-dimensional linear bandits with knapsacks using online sparse estimation and dual variables.
problem High-dimensional linear bandits with knapsacks.
method Online hard thresholding algorithm for sparse estimation, integrated with primal-dual scheme.
result Achieves sub-linear regret that scales logarithmically with feature dimension, improving on prior work.
Paper addresses regret minimization and inference in high-dimensional online decision-making.
problem Regret minimization and statistical inference in high-dimensional online decision-making.
method Integrates ε-greedy bandit algorithm with hard thresholding for sparse bandit parameters and debiasing method for inference.
result Achieves either O ( T 1 / 2 ) O(T^{1/2}) O ( T 1/2 ) regret or O ( T 1 / 2 ) O(T^{1/2}) O ( T 1/2 ) -consistent inference, with trade-off between exploration and exploitation. Contextual bandits with linear payoffs, which are also known as linear bandits, provide a powerful alternative for solving practical problems of sequential decisions, e.g., online advertisements. In the era of big data, contextual data usually tend to be high-dimensional, which leads to new challenges for traditional l…
New algorithm tackles high-dimensional contextual bandits without sparsity.
problem High-dimensional linear contextual bandit problem with large feature space.
method Proposes explore-then-commit (EtC) and adaptive explore-then-commit (AEtC) algorithms.
result Derives optimal rate for ETC algorithm and shows adaptive AEtC achieves it.
Develops TOFU for tensor bandits with low-rank structure.
problem Linear bandit models fail to capture high-dimensional, low-rank tensor structures.
method Develops TOFU, a tensor bandit algorithm that estimates low-dimensional subspaces and uses norm constraints.
result Improves regret bound by a multiplicative factor that grows exponentially in system order.
Jointly tackles assortment and pricing in retail, using bandit models.
problem Maximizing revenue or profit in retail through optimal assortment and pricing.
method Contextual bandits with a flexible, interpretable model for high-dimensional contexts and actions.
result Proves lower regret compared to state-of-the-art methods in various bandit and pricing models.
New algorithm reduces regret in bandit optimization for high-dimensional data.
problem Optimizing decisions in uncertain environments with high-dimensional data.
method Inspired by online Newton step, proposes a simple and efficient BCO algorithm.
result Achieves optimal regret bounds for κ κ κ -convex functions. New method tackles high-dimensional contextual bandits with flexible kernel models.
problem Maximizing rewards in decision-making scenarios with many features.
method Introduces stochastic assumptions and no-regret learning for Gaussian kernels.
result Achieves no-regret learning even with feature dimensions growing with samples.
Optimal algorithm for high-dimensional stochastic linear bandits with sparse parameters.
problem High-dimensional stochastic linear bandits with sparse parameters.
method Three-stage arm selection algorithm using thresholded Lasso for estimation.
result Achieves exact minimax optimality in cumulative regret.
A new algorithm detects changes in high-dimensional data efficiently under sampling constraints.
problem Real-time monitoring of high-dimensional streaming data with limited sampling.
method Incorporates multi-armed bandit approaches into sequential change-point detection.
result Proposes TSSRP algorithm for efficient detection of changes.
Study symmetric linear bandits with hidden symmetry, achieving improved regret bounds.
problem High-dimensional linear bandits with hidden symmetry.
method Model selection within low-dimensional subspaces to learn hidden symmetry.
result Achieved improved regret bounds of O ( d 0 2 / 3 T 2 / 3 log ( d ) ) O(d_0^{2/3} T^{2/3} \log(d)) O ( d 0 2/3 T 2/3 log ( d )) and O ( d 0 T log ( d ) ) O(d_0\sqrt{T\log(d)} ) O ( d 0 T log ( d ) ) . A new method for sparse linear bandits reduces exploration-exploitation tradeoff.
problem Sparse linear bandits in high-dimensional settings with finite actions.
method Best subset selection for parameter estimation and doubly growing epochs for regret minimization.
result Achieves nearly dimension-independent regret of i l d e O ( s T ) ilde{\mathcal{O}}(s\sqrt{T}) i l d e O ( s T ) with high probability. We propose a minimax concave penalized multi-armed bandit algorithm under generalized linear model (G-MCP-Bandit) for a decision-maker facing high-dimensional data in an online learning and decision-making process. We demonstrate that the G-MCP-Bandit algorithm asymptotically achieves the optimal cumulative regret in t…
ALEXP improves model selection in linear bandits with exponential regret improvement.
problem Model selection in linear bandits is challenging due to balancing exploration and exploitation.
method ALEXP uses online learning with favorable bias-variance trade-off to emulate full-information feedback.
result ALEXP achieves an exponentially improved ( log M \log M log M ) regret dependence on the number of models M M M . DFPV improves PCL for confounded bandit policy evaluation.
problem Estimating causal effects in confounded settings with high-dimensional data.
method Deep feature proxy variable method (DFPV) for high-dimensional, nonlinear relationships.
result DFPV outperforms state-of-the-art methods on synthetic benchmarks and confounded bandit problems.
New algorithms balance collaboration and adversarial behavior in linear bandits.
problem Minimizing regret in a collaborative linear bandit problem with adversarial agents.
method Robust collaborative phased elimination algorithm with tight analyses.
result Achieves near-optimal regret bounds of $O\left(α+ 1/\sqrt{M}
ight) \sqrt{dT}$ for good agents.
New method reduces linear regret in high-dimensional bandit problems.
problem Heavy spectral tails in streaming matrices lead to linear regret in sketch-based linear bandits.
method Dyadic Block Sketching, a multi-scale matrix sketching approach.
result Achieves sublinear regret bounds without prior knowledge of streaming matrix properties.
We study the neural-linear bandit model for solving sequential decision-making problems with high dimensional side information. Neural-linear bandits leverage the representation power of deep neural networks and combine it with efficient exploration mechanisms, designed for linear contextual bandits, on top of the last…
Algorithm achieves comparable performance to fully dynamic data with only a few batches.
problem High-dimensional multi-armed contextual bandits with batched feedback.
method Provable sample-efficient algorithm using batch allocation method.
result Achieves regret bounds comparable to fully sequential setting with only L = O(log T) batches.
MOCA-HESP optimizes high-dimensional combinatorial and mixed spaces using hyper-ellipsoid partitioning.
problem Challenges in optimizing high-dimensional, combinatorial and mixed spaces.
method MOCA-HESP uses hyper-ellipsoid space partitioning with different categorical encoders and multi-armed bandit for adaptive selection.
result MOCA-HESP outperforms existing methods on various synthetic and real-world benchmarks.
High-dimensional representations often have a lower dimensional underlying structure. This is particularly the case in many decision making settings. For example, when the representation of actions is generated from a deep neural network, it is reasonable to expect a low-rank structure whereas conventional structures l…
SCaLE tackles dynamic regret in noisy bandit feedback with switching costs.
problem Unbounded metric movement costs in bandit online convex optimization.
method SCaLE algorithm for high-dimensional dynamic quadratic hitting costs and ℓ 2 \ell_2 ℓ 2 -norm switching costs, with spectral regret analysis. result First algorithm achieving sub-linear dynamic regret without hitting cost knowledge.
IDS improves sparse linear bandits by balancing information and regret.
problem Sparse linear bandits in high-dimensional decision-making.
method Information-directed sampling (IDS) with Bayesian regret bounds and empirical Bayesian sparse posterior sampling.
result IDS nearly matches existing lower bounds and significantly reduces regret.
LMC-TS uses MCMC for efficient posterior sampling in contextual bandits.
problem Efficiency of Thompson sampling for high-dimensional contextual bandits.
method Langevin Monte Carlo for direct posterior sampling.
result LMC-TS achieves sublinear regret bound for linear contextual bandits.
Contextual bandit learning is an increasingly popular approach to optimizing recommender systems via user feedback, but can be slow to converge in practice due to the need for exploring a large feature space. In this paper, we propose a coarse-to-fine hierarchical approach for encoding prior knowledge that drastically …
The paper tackles restless bandits with limited observation, proposing a method to analyze and approximate their optimal strategies.
problem Restless bandits with limited observation.
method General probabilistic model, PCL analysis, and approximation process.
result The proposed method can transform the problem into a finite-state problem, enabling the use of existing algorithms.
A new algorithm reduces communication costs for collaborative decision-making across clients.
problem Collaborative decision-making with sparse rewards and heterogeneous contexts.
method Federated Lasso algorithm for sparse linear contextual bandits.
result Achieves near-optimal regret with logarithmic communication costs.
SGD shows distinct phases in learning single-index models, achieving optimal sample complexity and regret.
problem Learning single-index models with SGD in adaptive data settings.
method Stochastic gradient descent (SGD) with an optimal learning rate schedule.
result SGD achieves near-optimal sample complexity and regret guarantees across both burn-in and learning phases.
New algorithm reduces regret for single-index bandits to nearly optimal.
problem Optimizing rewards from unknown projections of high-dimensional contexts.
method Two-phase algorithm: estimate projection direction, reduce to 1D bandit, use UCB.
result Achieved optimal regret of i l d e O ( T 2 / 3 ) ilde{\mathcal{O}}(T^{2/3}) i l d e O ( T 2/3 ) . Improved regret for single-index bandits with optimal algorithm.
problem Optimizing rewards from unknown one-dimensional projections of high-dimensional contexts.
method Two-phase algorithm: first estimate projection direction, then discretize and use UCB.
result Achieved optimal regret of i l d e O ( T 2 / 3 ) ilde{\mathcal{O}}(T^{2/3}) i l d e O ( T 2/3 ) . This paper addresses missing covariates in stochastic linear bandits, providing a high-probability regret bound.
problem Effect of missing covariates on regret in stochastic linear bandit algorithms.
method Proposes an algorithm that provides a high-probability upper bound on regret in terms of covariate sampling probabilities.
result Regret degrades due to missingness by at most ζ m i n 2 ζ_{min}^2 ζ min 2 , where ζ m i n ζ_{min} ζ min is the minimum probability of observing covariates. Paper addresses private online convex optimization with optimal algorithms in various geometries and high-dimensional bandits.
problem Private online convex optimization with streaming and continual release data.
method Proposes a private variant of online Frank-Wolfe algorithm with recursive gradients for variance reduction.
result Achieves optimal excess risk in linear time for 1 < p ≤ 2 1<p\leq 2 1 < p ≤ 2 and state-of-the-art excess risk for 2 < p ≤ ∞ 2<p\leq\infty 2 < p ≤ ∞ . New algorithms for generalized linear bandits with unknown reward functions.
problem Misspecification of reward functions in existing bandit algorithms.
method Introducing single index bandits, proposing STOR, ESTOR, and GSTOR algorithms.
result Achieved nearly optimal regret bound of i l d e O T ( T ) ilde{O}_T(\sqrt{T}) i l d e O T ( T ) . Paper introduces a bandit-learning method for multifidelity approximations.
problem Efficiently using data of varying fidelities in scientific computation.
method Formulates multifidelity approximation as a modified stochastic bandit problem and proposes AETC algorithm.
result Established optimality of AETC algorithm for multifidelity approximation.
Bandit algorithms have been predominantly analyzed in the convex setting with function-value based stationary regret as the performance measure. In this paper, motivated by online reinforcement learning problems, we propose and analyze bandit algorithms for both general and structured nonconvex problems with nonstation…
A planning approach learns skills from interactions, balancing exploration and exploitation.
problem Learning robust high-level skills in noisy environments with unknown pre-conditions.
method Formulates skills as high-level policies, learns plans via bandit problems, balances exploration and exploitation.
result A planner capable of learning robust high-level skills in high-dimensional state spaces.
We consider dynamic pricing with many products under an evolving but low-dimensional demand model. Assuming the temporal variation in cross-elasticities exhibits low-rank structure based on fixed (latent) features of the products, we show that the revenue maximization problem reduces to an online bandit convex optimiza…
Contextual multi-armed bandit algorithms are widely used in sequential decision tasks such as news article recommendation systems, web page ad placement algorithms, and mobile health. Most of the existing algorithms have regret proportional to a polynomial function of the context dimension, d d d . In many applications ho…
Lecture notes on reinforcement learning using statistical methods.
problem Addressing the exploration-exploitation dilemma in decision making.
method Frequentist and Bayesian approaches, function approximation, neural networks.
result Unified framework for decision making and estimation.
This paper improves recommender systems by handling dynamic user preferences and item popularity.
problem Dynamic user preferences and changing item popularity in recommender systems.
method Developed a Thompson sampling-based policy for a high-dimensional linear bandit problem, reducing feature vector dimensionality and using exponentially increasing weights.
result Proved a regret bound that scales with the reduced dimension, demonstrating effectiveness in trade-off between computational complexity and regret performance.