New algorithms improve on bandit feedback in matrix games with unknown payoff matrices.
problem Improving performance in matrix games with unknown payoff matrices and bandit feedback.
method Regret analyses of variants of UCB and K-learning.
result New algorithms achieve lower regret compared to adversarial bandit algorithms.
New algorithm improves game learning with randomised optimism.
problem Learning in matrix games with unknown payoffs and bandit feedback.
method Integrates evolutionary algorithms into bandit framework for randomised optimism.
result Achieves sublinear regret, outperforming classical methods.
New method reduces linear regret in high-dimensional bandit problems.
problem Heavy spectral tails in streaming matrices lead to linear regret in sketch-based linear bandits.
method Dyadic Block Sketching, a multi-scale matrix sketching approach.
result Achieves sublinear regret bounds without prior knowledge of streaming matrix properties.
Novel method for efficient low-rank matrix estimation and bandit algorithms.
problem Low-rank matrix estimation and bandit problems.
method LowPopArt method for low-rank matrix estimation and novel experimental design criterion.
result Improved recovery guarantees and regret bounds for low-rank bandit algorithms.
Unified framework for high-dimensional bandit problems with low-dimensional structures.
problem Stochastic high-dimensional bandit problems with low-dimensional structures.
method Proposed a simple unified algorithm and a general analysis framework for the regret upper bound.
result Unified algorithm achieves comparable regret bounds in various high-dimensional bandit problems.
Contextual bandits have the same exploration-exploitation trade-off as standard multi-armed bandits. On adding positive externalities that decay with time, this problem becomes much more difficult as wrong decisions at the start are hard to recover from. We explore existing policies in this setting and highlight their …
New framework improves efficiency in low-rank matrix bandit problems.
problem Stochastic contextual low-rank matrix bandit problem with unknown rank matrices.
method G-ESTT and G-ESTS frameworks using Stein's method and regularization.
result Achieved improved regret bounds for low-rank matrix bandit problems.
Algorithm for low-rank matrix bandits with heavy-tailed rewards, achieving nearly optimal regret bound.
problem Stochastic low-rank matrix bandit with heavy-tailed rewards.
method LOTUS algorithm using truncation and dynamic exploration.
result Regret bound of order $ ilde O(d^ rac{3}{2}r^ rac{1}{2}T^ rac{1}{1+δ}/ ilde{D}_{rr})$ without knowing T T T . RONM method reduces regret in stochastic convex bandits with decreasing noise.
problem Stochastic convex bandit problem with decreasing noise.
method Regularized Online Newton Method (RONM) based on Online Newton Method (ONM).
result RONM achieves polylogarithmic regret in time horizon n.
Optimizes arm selection with side information in Gaussian bandits.
problem Optimizing arm selection with side information in Gaussian bandits.
method Constructs an LP-based asymptotic instance-dependent lower bound on the regret and develops the first known asymptotically optimal algorithm.
result First known asymptotically optimal algorithm for Gaussian bandits with side information.
New algorithm reduces semi-bandit regret using covariance estimates.
problem Complexity of semi-bandits due to joint distribution of outcomes.
method Develops a new sub-exponential distribution family and an algorithm using covariance estimates.
result Proves a new lower bound on expected regret and constructs an algorithm with asymptotic analysis.
Optimal algorithm for latent bandits with cluster structure reduces regret to nearly optimal.
problem Maximizing cumulative rewards in a multi-armed bandit problem with latent clusters.
method LATTICE algorithm exploiting cluster structure and arm information.
result Minimax optimal regret of O ( ( M + N ) T ) O(\sqrt{(\mathsf{M}+\mathsf{N})\mathsf{T}}) O ( ( M + N ) T ) with O ( log T ) O(\log{\mathsf{T}}) O ( log T ) calls to matrix completion oracle. New algorithm reduces cold-start costs in multi-armed bandits for many products.
problem High burn-in costs in multi-armed bandits for new products.
method Two-phase bandit algorithm using subsampling and low-rank matrix estimation.
result Reduces burn-in costs and expedites experiment in large product sets.
New algorithm improves learning efficiency in multi-task contextual bandits.
problem Improving learning efficiency in multi-task contextual bandits.
method Alternating projected gradient descent (GD) and minimization estimator for low-rank feature matrix recovery.
result Proved regret bound for multi-task learning algorithm.
Optimizes matching in weighted graphs with semi-bandit sampling.
problem Finding optimal pairings in weighted graphs with sequential sampling.
method Leverages rank-1 assumption on adjacency matrix to reduce sample complexity and regret.
result Achieves linear dependency in the number of vertices for sample complexity and regret.
TRAiL is a linear bandit algorithm that ensures optimal regret and guarantees inference quality.
problem Optimal regret and inference quality in linear bandits with convex action sets.
method TRAiL estimates the parameter through regularized least squares and perturbs the action set along the tangent plane.
result TRAiL achieves an Ω ( T ) Ω(\sqrt{T}) Ω ( T ) upper bound on cumulative regret with high probability. Optimal algorithms identify non-dominated arms in multi-output linear bandit models.
problem Identifying the Pareto Set in multi-output linear bandit models.
method Design-based algorithms for Pareto Set Identification (PSI) in a structured multi-output linear bandit model.
result Nearly optimal guarantees in both fixed-budget and fixed-confidence settings.
We consider the problem of online collaborative filtering in the online setting, where items are recommended to the users over time. At each time step, the user (selected by the environment) consumes an item (selected by the agent) and provides a rating of the selected item. In this paper, we propose a novel algorithm …
New spectral methods improve matrix estimation in RL with low-rank structure.
problem Estimating matrices with low-rank structure in reinforcement learning.
method Spectral-based matrix estimation approaches.
result Spectral methods efficiently recover singular subspaces and minimize entry-wise error.
In this paper, we propose a Double Thompson Sampling (D-TS) algorithm for dueling bandit problems. As indicated by its name, D-TS selects both the first and the second candidates according to Thompson Sampling. Specifically, D-TS maintains a posterior distribution for the preference matrix, and chooses the pair of arms…
New method identifies Condorcet winner in dueling bandits with improved sample complexity.
problem Identifying Condorcet winner in noisy pairwise comparisons.
method Exploits full gap matrix Δ to improve sample complexity.
result Improves sample complexity guarantees by leveraging informative comparisons.
We propose a novel algorithm for sequential matrix completion in a recommender system setting, where the ( i , j ) (i,j) ( i , j ) th entry of the matrix corresponds to a user i i i 's rating of product j j j . The objective of the algorithm is to provide a sequential policy for user-product pair recommendation which will yield the highest pos…
We revisit the proof by Qin et al. (2014) of bounded regret of the C 2 ^2 2 UCB contextual combinatorial bandit. We demonstrate an error in the proof of volumetric expansion of the moment matrix, used in upper bounding a function of context vector norms. We prove a relaxed inequality that yields the originally-stated regre…
Jointly tackles assortment and pricing in retail, using bandit models.
problem Maximizing revenue or profit in retail through optimal assortment and pricing.
method Contextual bandits with a flexible, interpretable model for high-dimensional contexts and actions.
result Proves lower regret compared to state-of-the-art methods in various bandit and pricing models.
New algorithms handle online prediction with bandit and delayed feedback, improving regret bounds.
problem Achieving finite bounds on surrogate regret with limited feedback.
method Proposed algorithms for bandit and delayed feedback, including inverse-weighted gradient and pseudo-inverse matrix estimators.
result Achieved improved surrogate regret bounds of O ( K T ) O(\sqrt{KT}) O ( K T ) and O ( T 2 / 3 ) O(T^{2/3}) O ( T 2/3 ) . Study multi-task learning with low-rank representation in stochastic linear bandits.
problem Transfer learning across multiple linear bandit tasks with shared low-dimensional representation.
method Proposes a greedy policy with trace norm regularization to implicitly learn a low-rank representation without knowing the rank.
result Upper bound on multi-task regret of N d T ( T + d ) r \sqrt{NdT(T+d)r} N d T ( T + d ) r , showing benefit over independent task solving. We address the cold start problem in recommendation systems assuming no contextual information is available neither about users, nor items. We consider the case in which we only have access to a set of ratings of items by users. Most of the existing works consider a batch setting, and use cross-validation to tune param…
An algorithm tackles low-rank linear bandit problems with improved regret bounds.
problem Low-rank linear bandit problems where rewards are inner products with an unknown low-rank matrix.
method Combines online-to-confidence-set conversion and exponentially weighted average forecaster with a covering of low-rank matrices.
result Achieves O ~ ( ( d 1 + d 2 ) 3 / 2 r T ) \widetilde{O}((d_1+d_2)^{3/2}\sqrt{rT}) O (( d 1 + d 2 ) 3/2 r T ) regret, improving over standard bounds when r ≪ min { d 1 , d 2 } r \ll \min\{d_1,d_2\} r ≪ min { d 1 , d 2 } . GL-LowPopArt improves minimax-optimal estimation for trace regression.
problem Minimizing estimation error in generalized low-rank trace regression.
method Two-stage approach: nuclear norm regularization followed by matrix Catoni estimation.
result Achieves instance-wise optimal error bounds up to condition number.
Efficient algorithms for low-rank bandits using subspace recovery.
problem Contextual bandits with low-rank reward matrices.
method Spectral methods for subspace recovery, reformulating as linear bandits.
result Nearly optimal policy evaluation and best policy identification, minimax guarantees for regret minimization.
New method learns decisions from collective preferences without individual covariates.
problem Making decisions online without individual covariates.
method Collaborative filtering, matrix completion bandit, ε-greedy policy, online gradient descent, inverse propensity weighting.
result Method outperforms benchmarks and reveals new discoveries.
Paper addresses federated contextual bandits with encryption.
problem Building contextual bandits with vertically distributed data.
method Design of O3M encryption scheme for LinUCB and LinTS.
result Proposed protocols achieve good performance and privacy.
Lower bounds on eigenspectrum show rich action spaces force polynomial regret in linear bandits.
problem Understanding the minimum eigenvalue growth in linear bandits with rich action sets.
method Non-asymptotic lower bound on eigenspectrum of design matrix.
result Minimum eigenvalue of expected design matrix grows as Ω ( n ) Ω(\sqrt{n}) Ω ( n ) for sub-linear regret. In this paper, we propose the first computationally efficient projection-free algorithm for bandit convex optimization (BCO). We show that our algorithm achieves a sublinear regret of O ( n T 4 / 5 ) O(nT^{4/5}) O ( n T 4/5 ) (where T T T is the horizon and n n n is the dimension) for any bounded convex functions with uniformly bounded gradients. We …
A new MDP with Bandits approach for sequential decision making in linear-flow scenarios.
problem Sequential decision making with limited feedback in a linear-flow context.
method Formulated as an MDP with Bandits, using Thompson sampling for action selection and exact dynamic programming for allocation.
result The proposed MDP with Bandits algorithm outperforms other methods in sequential decision making.
New algorithm reduces bandit regret to log^3(T).
problem Noise model for linear stochastic bandits with vanishing noise.
method Weighted least-squares estimation, leveraging eigenvalue relation.
result Minimax regret scaling as log^3(T) for time horizon T.
Motivated by online recommendation and advertising systems, we consider a causal model for stochastic contextual bandits with a latent low-dimensional confounder. In our model, there are L L L observed contexts and K K K arms of the bandit. The observed context influences the reward obtained through a latent confounder var…
Unified framework for ensemble sampling in nonlinear contextual bandits with provable regret bounds.
problem Efficient exploration in nonlinear contextual bandits with unknown feature dimensions.
method Developed GLM-ES and Neural-ES for generalized linear and neural contextual bandits, respectively, using maximum likelihood estimation on randomly perturbed data.
result Unified high-probability frequentist regret bounds for GLM-ES and Neural-ES, matching state-of-the-art results.
LinUCB algorithm handles adaptive sampling biases for linear bandits.
problem Adaptive sampling introduces biases in statistical inference.
method LinUCB algorithm with stability property for linear bandits.
result LinUCB achieves asymptotic normality with Wald-type confidence sets.
Optimistic covariance-adaptive algorithms improve combinatorial semi-bandits regret.
problem Optimal regret in stochastic combinatorial semi-bandits with adaptive covariance estimation.
method Design of OLS-UCB-C and COS-V algorithms leveraging online covariance estimation.
result Improved gap-free regret with T^1/2 complexity for COS-V.
Improved Bayesian regret bound for linear Thompson sampling with general distributions.
problem Proving an improved Bayesian regret bound for linear Thompson sampling with general distributions.
method Generalized elliptical potential lemma for non-Gaussian noise and prior distributions.
result Minimax optimal regret bound for changing action sets with general prior and noise distributions.
Algorithm minimizes regret in non-stationary dueling bandits with unknown parameters.
problem Minimizing regret in dueling bandits with time-varying preferences.
method Proposes Beat the Winner Reset algorithm and meta-algorithms DETECT and Monitored Dueling Bandits.
result Proves bounds on expected weak and strong regret for non-stationary dueling bandits.
Optimized strategies for graph-structured bandits reduce regret efficiently.
problem Structured multi-armed bandit problem with graph constraints.
method Adapted IMED algorithm to graph-structured bandits, introducing IMED-GS ⋆ ^\star ⋆ . result IMED-GS ⋆ ^\star ⋆ is asymptotically optimal and requires minimal exploration. Algorithm maximizes user rewards under per-item budget constraints.
problem Maximizing cumulative rewards in collaborative bandits with budget constraints.
method Collaborative algorithm B-LATTICE that clusters users and collaborates across groups.
result Achieves sub-linear regret bounds matching minimax bounds.
New algorithm for minimizing regret in adversarial dueling bandits.
problem Minimizing regret in dueling bandits with adversarial feedback.
method Introduced an algorithm with T T T -round regret of i l d e O ( K 1 / 3 T 2 / 3 ) ilde{O}(K^{1/3}T^{2/3}) i l d e O ( K 1/3 T 2/3 ) . result Algorithm achieves Ω ( K 1 / 3 T 2 / 3 ) Ω(K^{1/3}T^{2/3}) Ω ( K 1/3 T 2/3 ) regret lower bound. We consider dynamic pricing with many products under an evolving but low-dimensional demand model. Assuming the temporal variation in cross-elasticities exhibits low-rank structure based on fixed (latent) features of the products, we show that the revenue maximization problem reduces to an online bandit convex optimiza…
New algorithms converge faster to Nash equilibrium in zero-sum games with bandit feedback.
problem Learning in zero-sum games with bandit feedback without communication.
method Developed two uncoupled algorithms achieving optimal rate of Ω ( T − 1 / 4 ) Ω(T^{-1/4}) Ω ( T − 1/4 ) . result Achieved optimal rate of Ω ( T − 1 / 4 ) Ω(T^{-1/4}) Ω ( T − 1/4 ) for convergence of policy profiles to Nash equilibrium. Study best arm identification in restless bandits with unknown TPMs.
problem Identify the best arm with fixed confidence in restless bandits with unknown TPMs.
method Proposed a policy for best arm identification and proved its expected stopping time matches the lower bound.
result The state-action visitation proportions match the optimal proportions under any asymptotically optimal policy.