arXiv research

Study preference-based reinforcement learning in episodic kernel MDPs.

Algorithm tackles constrained reinforcement learning with concave-convex and knapsack constraints.

BerlinUCB learns from episodic rewards in nonstationary contexts.

Algorithm improves online learning in adversarial bandits.

Paper proposes DAC-ML, a cognitive architecture that learns quickly from few episodes.

Many interesting real world domains involve reinforcement learning (RL) in partially observable environments. Efficient learning in such domains is important, but existing sample complexity bounds for partially observable RL are at least exponential in the episode length. We give, to our knowledge, the first partially …

We introduce a new class of reinforcement learning methods referred to as {\em episodic multi-armed bandits} (eMAB). In eMAB the learner proceeds in {\em episodes}, each composed of several {\em steps}, in which it chooses an action and observes a feedback signal. Moreover, in each step, it can take a special action, c…

We consider online learning in episodic loop-free Markov decision processes (MDPs), where the loss function can change arbitrarily between episodes, and the transition function is not known to the learner. We show $\tilde{O}(L|X|\sqrt{|A|T})$ regret bound, where $T$ is the number of episodes, $X$ is the state space, $A…

Recent advances in deep reinforcement learning algorithms have shown great potential and success for solving many challenging real-world problems, including Go game and robotic applications. Usually, these algorithms need a carefully designed reward function to guide training in each time step. However, in real world, …

We introduce a lifelong language learning setup where a model needs to learn from a stream of text examples without any dataset identifier. We propose an episodic memory model that performs sparse experience replay and local adaptation to mitigate catastrophic forgetting in this setup. Experiments on text classificatio…

This paper explores how environmental properties can simplify reinforcement learning in non-episodic settings.

New algorithm for average reward learning with bounded hitting time assumption.

MEMEC improves sample efficiency in reinforcement learning.

We propose Episodic Backward Update (EBU) - a novel deep reinforcement learning algorithm with a direct value propagation. In contrast to the conventional use of the experience replay with uniform random sampling, our agent samples a whole episode and successively propagates the value of a state to its previous states.…

Extends Thompson sampling for RL with fewer episodes.

Paper proposes RRD to learn proxy rewards for sparse delayed rewards in episodic reinforcement learning.

New method learns near-optimal policies with polynomial samples in A and H.

Improved reinforcement learning for episodes with varying action sets.

Long horizon reinforcement learning is as hard as short horizon learning.

Single episode policy transfer in RL without access to rewards.

In continual learning (CL), an agent learns from a stream of tasks leveraging prior experience to transfer knowledge to future tasks. It is an ideal framework to decrease the amount of supervision in the existing learning algorithms. But for a successful knowledge transfer, the learner needs to remember how to perform …

Meta-learning agents excel at rapidly learning new tasks from open-ended task distributions; yet, they forget what they learn about each task as soon as the next begins. When tasks reoccur - as they do in natural environments - metalearning agents must explore again instead of immediately exploiting previously discover…

Reinforcement learning (RL) algorithms have made huge progress in recent years by leveraging the power of deep neural networks (DNN). Despite the success, deep RL algorithms are known to be sample inefficient, often requiring many rounds of interaction with the environments to obtain satisfactory performance. Recently,…

State of the art deep reinforcement learning algorithms take many millions of interactions to attain human-level performance. Humans, on the other hand, can very quickly exploit highly rewarding nuances of an environment upon first discovery. In the brain, such rapid learning is thought to depend on the hippocampus and…

PS4POMDPs algorithm simplifies online learning for episodic POMDPs with unknown models.

New method improves continual learning by anchoring past knowledge.

We consider a reinforcement learning (RL) setting in which the agent interacts with a sequence of episodic MDPs. At the start of each episode the agent has access to some side-information or context that determines the dynamics of the MDP for that episode. Our setting is motivated by applications in healthcare where ba…

Current deep neural networks can achieve remarkable performance on a single task. However, when the deep neural network is continually trained on a sequence of tasks, it seems to gradually forget the previous learned knowledge. This phenomenon is referred to as \textit{catastrophic forgetting} and motivates the field c…

We investigate the robustness properties of image recognition models equipped with two features inspired by human vision, an explicit episodic memory and a shape bias, at the ImageNet scale. As reported in previous work, we show that an explicit episodic memory improves the robustness of image recognition models agains…

StepMix algorithm ensures safe exploration in reinforcement learning with near-optimal performance.

Paper proposes new $γ$-regret measure for non-episodic RL.

Most provably-efficient learning algorithms introduce optimism about poorly-understood states and actions to encourage exploration. We study an alternative approach for efficient exploration, posterior sampling for reinforcement learning (PSRL). This algorithm proceeds in repeated episodes of known duration. At the sta…

Quantum UCB algorithm reduces reinforcement learning regret exponentially.

Paper optimizes learning MDPs with adversarial rewards using linear models.

We introduce an off-policy evaluation procedure for highlighting episodes where applying a reinforcement learned (RL) policy is likely to have produced a substantially different outcome than the observed policy. In particular, we introduce a class of structural causal models (SCMs) for generating counterfactual traject…

The success of popular algorithms for deep reinforcement learning, such as policy-gradients and Q-learning, relies heavily on the availability of an informative reward signal at each timestep of the sequential decision-making process. When rewards are only sparsely available during an episode, or a rewarding feedback i…

UCB-Advantage learns MDPs with $O(\sqrt{H^2SAT})$ regret.

We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…

New algorithm reduces reinforcement learning complexity, approaching contextual bandits.

Algorithm learns both stochastic and adversarial MDPs with best-of-both-worlds guarantees.

Algorithm improves multi-armed bandit performance by transferring reward samples.

New algorithm reduces regret and constraint violation in adversarial CMDP learning.

Efficiently stores and retrieves past states for faster learning in reinforcement learning.

UC-SSP algorithm tackles exploration in stochastic shortest path problems without loop-free assumption.

This paper studies poisoning attacks in episodic RL and discovers their effectiveness depends on reward bounds.

UDRL fails to converge in stochastic environments with episodic resets.

Proposes a CL technique to improve accuracy and reduce forgetting.

Study shows reinforcement learning is possible with once-per-episode feedback.