arXiv research

Optimal switching regret for all segmentations in online convex optimisation.

As a metric to measure the performance of an online method, dynamic regret with switching cost has drawn much attention for online decision making problems. Although the sublinear regret has been provided in many previous researches, we still have little knowledge about the relation between the dynamic regret and the s…

SCaLE tackles dynamic regret in noisy bandit feedback with switching costs.

New RL algorithm reduces policy switching cost to loglog(T) with similar regret.

Paper presents an efficient algorithm for linear MDP with low switching cost.

We study the problem of switching-constrained online convex optimization (OCO), where the player has a limited number of opportunities to change her action. While the discrete analog of this online learning task has been studied extensively, previous work in the continuous setting has neither established the minimax ra…

This paper tackles near-optimal adversarial RL with switching costs, providing algorithms and matching lower bounds.

New algorithm reduces switching costs in multinomial logit bandit problems.

PCGS-TF uses a Transformer to adaptively control expert switching in non-stationary environments.

We consider the classical stochastic multi-armed bandit problem with a constraint that limits the total cost incurred by switching between actions to be no larger than a given switching budget. For this problem, we prove matching upper and lower bounds on the optimal (i.e., minimax) regret, and provide efficient rate-o…

Algorithm for bandits with switching costs achieves optimal regret bounds.

New algorithm reduces switching costs in RL beyond linear MDPs.

The paper improves competitive and dynamic regret bounds for smoothed online learning.

Near-logarithmic regret per switch achieved for mixable/exp-concave losses.

New RL algorithms reduce costs for single-agent and federated learning.

We take initial steps in studying PAC-MDP algorithms with limited adaptivity, that is, algorithms that change its exploration policy as infrequently as possible during regret minimization. This is motivated by the difficulty of running fully adaptive algorithms in real-world applications (such as medical domains), and …

We study online learning when partial feedback information is provided following every action of the learning process, and the learner incurs switching costs for changing his actions. In this setting, the feedback information system can be represented by a graph, and previous works studied the expected regret of the le…

Efficient algorithms for online convex optimization with limited switching decisions.

We study the power of different types of adaptive (nonoblivious) adversaries in the setting of prediction with expert advice, under both full-information and bandit feedback. We measure the player's performance using a new notion of regret, also known as policy regret, which better captures the adversary's adaptiveness…

This paper studies the impact of limited switches on resource-constrained dynamic pricing with demand learning. We focus on the classical price-based blind network revenue management problem and extend our results to the bandits with knapsacks problem. In both settings, a decision maker faces stochastic and distributio…

New algorithm reduces regret for many bandit algorithms with logarithmic dependence on number of algorithms.

Efficient RL algorithms for linear function approximation with limited adaptivity constraints.

In this paper, we investigate the non-stationary combinatorial semi-bandit problem, both in the switching case and in the dynamic case. In the general case where (a) the reward function is non-linear, (b) arms may be probabilistically triggered, and (c) only approximate offline oracle exists \cite{wang2017improving}, o…

New algorithm minimizes cumulative loss in dynamic linear bandits without prior knowledge of comparator switches.

New algorithm reduces decision switching in dynamic environments.

New algorithms tackle adversarial combinatorial bandits with switching costs.

LaMBO optimizes modular systems with switching costs, achieving better results than existing methods.

OMGD algorithm optimizes online convex optimization with switching costs and delayed gradients.

We study the adversarial multi-armed bandit problem where partial observations are available and where, in addition to the loss incurred for each action, a \emph{switching cost} is incurred for shifting to a new action. All previously known results incur a factor proportional to the independence number of the feedback …

New algorithm achieves both static and dynamic regret optimally against an oblivious adversary for deterministic losses.

This paper improves Q-learning bounds using reference-advantage decomposition.

Reinforcement learning agents have been mostly developed and evaluated under the assumption that they will operate in a fully autonomous manner -- they will take all actions. In this work, our goal is to develop algorithms that, by learning to switch control between agents, allow existing reinforcement learning agents …

New algorithm learns optimal policies with minimal memory and time.

We consider online algorithms under both the competitive ratio criteria and the regret minimization one. Our main goal is to build a unified methodology that would be able to guarantee both criteria simultaneously. For a general class of online algorithms, namely any Metrical Task System (MTS), we show that one can sim…

Two algorithms achieve optimal regret with limited adaptivity in multinomial logistic bandits.

FTRL algorithm with negative entropy regularizer achieves best-of-three-world results for linear bandits.

Efficiently optimize GPs by reusing candidate solutions multiple times.

New algorithm reduces RL complexity with low switching costs.

New method tracks significant arm switches to improve bandit algorithms.

We propose the first reduction-based approach to obtaining long-term memory guarantees for online learning in the sense of Bousquet and Warmuth, 2002, by reducing the problem to achieving typical switching regret. Specifically, for the classical expert problem with $K$ actions and $T$ rounds, using our framework we dev…

Online learning algorithms are designed to learn even when their input is generated by an adversary. The widely-accepted formal definition of an online algorithm's ability to learn is the game-theoretic notion of regret. We argue that the standard definition of regret becomes inadequate if the adversary is allowed to a…

Study on adaptivity constraints in linear contextual bandits with optimal design.

New algorithms for multitask learning with long-term memory.

UCB-Advantage learns MDPs with $O(\sqrt{H^2SAT})$ regret.

New algorithm minimizes expert selection regret in partial bandit feedback.

We consider the exploration-exploitation tradeoff in linear quadratic (LQ) control problems, where the state dynamics is linear and the cost function is quadratic in states and controls. We analyze the regret of Thompson sampling (TS) (a.k.a. posterior-sampling for reinforcement learning) in the frequentist setting, i.…

New private algorithms for online learning improve regret in high privacy regimes.

To deal with changing environments, a new performance measure -- adaptive regret, defined as the maximum static regret over any interval, was proposed in online learning. Under the setting of online convex optimization, several algorithms have been successfully developed to minimize the adaptive regret. However, existi…