arXiv research

SCaLE tackles dynamic regret in noisy bandit feedback with switching costs.

New algorithms reduce dueling bandits' regret with neural networks and efficient exploration.

Unified analysis of kernel-based and locally adaptive bandit optimization methods.

New method reduces linear regret in high-dimensional bandit problems.

Efficient algorithm predicts unknown linear systems with long-term memory.

This paper analyzes the multi-armed bandit problem using frequency-domain methods.

Improved Bandit PCA with optimal regret bound.

This paper improves parameter estimation for autonomous systems with unmodeled dynamics.

SpectralTS improves efficiency of Thompson Sampling for graph-based bandits.

New method controls linear systems with adversarial disturbances.

Improved guarantees for misspecified kernelized bandit optimization.

Reinforcement learning (RL) in Markov decision processes (MDPs) with large state spaces is a challenging problem. The performance of standard RL algorithms degrades drastically with the dimensionality of state space. However, in practice, these large MDPs typically incorporate a latent or hidden low-dimensional structu…

New method controls linear systems with partial info and disturbances.

Lower bounds on eigenspectrum show rich action spaces force polynomial regret in linear bandits.

In this paper, we provide a novel construction of the linear-sized spectral sparsifiers of Batson, Spielman and Srivastava [BSS14]. While previous constructions required $Ω(n^4)$ running time [BSS14, Zou12], our sparsification routine can be implemented in almost-quadratic running time $O(n^{2+\varepsilon})$. The funda…

A new algorithm reduces the time and space complexity for multinomial logistic bandits.

Study risk-sensitive reinforcement learning with Lipschitz dynamic risk measures, establishing regret bounds.

New framework for Adam-type algorithms with constant β1, improving regret analysis.

In this paper we focus on the problem of Online Principal Component Analysis in the regret minimization framework. For this problem, all existing regret minimization algorithms for the fully-adversarial setting are based on a positive semidefinite convex relaxation, and hence require quadratic memory and SVD computatio…

A decentralized algorithm minimizes cumulative regret in stochastic linear bandits with safety constraints.

Improved analysis of UCRL2 with empirical Bernstein inequality reduces exploration-exploitation regret.

Improved dynamic regret analysis for strongly convex and smooth functions.

Optimistic Hedge achieves optimal regret bounds in two-player zero-sum games.

Significant improvements in regret analysis for adaptive online learning problems.

We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…

New algorithms for private generalized linear contextual bandits.

We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…

This paper analyzes data-driven Newsvendor problems and finds a wide range of possible regrets.

New analysis shows how temporal variability affects online learning performance.

Study on regret minimization in deterministic MDPs.

We study a decentralized cooperative stochastic multi-armed bandit problem with $K$ arms on a network of $N$ agents. In our model, the reward distribution of each arm is the same for each agent and rewards are drawn independently across agents and time steps. In each round, each agent chooses an arm to play and subsequ…

Study shows certainty equivalent policy minimizes regret in continuous-time systems.

We prove a new minimax theorem connecting the worst-case Bayesian regret and minimax regret under partial monitoring with no assumptions on the space of signals or decisions of the adversary. We then generalise the information-theoretic tools of Russo and Van Roy (2016) for proving Bayesian regret bounds and combine th…

Unified analysis of Gaussian Process Thompson Sampling without discretization.

The paper tackles robust policy learning from multiple data sources.

We propose algorithms for online principal component analysis (PCA) and variance minimization for adaptive settings. Previous literature has focused on upper bounding the static adversarial regret, whose comparator is the optimal fixed action in hindsight. However, static regret is not an appropriate metric when the un…

This paper considers the stability of online learning algorithms and its implications for learnability (bounded regret). We introduce a novel quantity called {\em forward regret} that intuitively measures how good an online learning algorithm is if it is allowed a one-step look-ahead into the future. We show that given…

Mirror descent with an entropic regularizer is known to achieve shifting regret bounds that are logarithmic in the dimension. This is done using either a carefully designed projection or by a weight sharing technique. Via a novel unified analysis, we show that these two approaches deliver essentially equivalent bounds …

Ensemble sampling approximates Thompson sampling for complex models.

Improved Thompson Sampling reduces regret in contextual bandits and reinforcement learning.

Paper tackles efficient BAI in graph-smooth bandits.

We consider an online learning process to forecast a sequence of outcomes for nonconvex models. A typical measure to evaluate online learning algorithms is regret but such standard definition of regret is intractable for nonconvex models even in offline settings. Hence, gradient based definition of regrets are common f…

New insights into balancing reward and fairness in stochastic MAB.

Improved FPL algorithms for adversarial MDPs with better regret bounds.

New model-free algorithm achieves similar LQR regret guarantees.

Paper improves regret bounds for Gaussian process upper confidence bound in Bayesian optimization.

Randomized exploration in linear bandits achieves optimal regret bounds.

Improved regret bound for linear ensemble sampling.