arXiv research

BerlinUCB learns from episodic rewards in nonstationary contexts.

Algorithm improves multi-armed bandit performance by transferring reward samples.

Paper proposes RRD to learn proxy rewards for sparse delayed rewards in episodic reinforcement learning.

Study preference-based reinforcement learning in episodic kernel MDPs.

This paper explores how environmental properties can simplify reinforcement learning in non-episodic settings.

Recent advances in deep reinforcement learning algorithms have shown great potential and success for solving many challenging real-world problems, including Go game and robotic applications. Usually, these algorithms need a carefully designed reward function to guide training in each time step. However, in real world, …

New algorithm for average reward learning with bounded hitting time assumption.

The success of popular algorithms for deep reinforcement learning, such as policy-gradients and Q-learning, relies heavily on the availability of an informative reward signal at each timestep of the sequential decision-making process. When rewards are only sparsely available during an episode, or a rewarding feedback i…

This paper studies poisoning attacks in episodic RL and discovers their effectiveness depends on reward bounds.

State of the art deep reinforcement learning algorithms take many millions of interactions to attain human-level performance. Humans, on the other hand, can very quickly exploit highly rewarding nuances of an environment upon first discovery. In the brain, such rapid learning is thought to depend on the hippocampus and…

Transforming sparse outcomes into dense process rewards for efficient reinforcement learning.

Paper optimizes learning MDPs with adversarial rewards using linear models.

We introduce a new class of reinforcement learning methods referred to as {\em episodic multi-armed bandits} (eMAB). In eMAB the learner proceeds in {\em episodes}, each composed of several {\em steps}, in which it chooses an action and observes a feedback signal. Moreover, in each step, it can take a special action, c…

Algorithm learns near-optimal policies for reward-mixing MDPs with few latent contexts.

Single episode policy transfer in RL without access to rewards.

Paper proposes DAC-ML, a cognitive architecture that learns quickly from few episodes.

We introduce an off-policy evaluation procedure for highlighting episodes where applying a reinforcement learned (RL) policy is likely to have produced a substantially different outcome than the observed policy. In particular, we introduce a class of structural causal models (SCMs) for generating counterfactual traject…

New algorithm learns policies without explicit rewards for MDPs.

Algorithm tackles constrained reinforcement learning with concave-convex and knapsack constraints.

New rule reduces exploration regret to logarithmic, improving bad episode handling.

Paper proposes new $γ$-regret measure for non-episodic RL.

New RL algorithm tackles adversarial RMAB with unknown transitions and bandit feedback.

Extends Thompson sampling for RL with fewer episodes.

New algorithm finds optimal policies without knowing reward functions.

We propose Episodic Backward Update (EBU) - a novel deep reinforcement learning algorithm with a direct value propagation. In contrast to the conventional use of the experience replay with uniform random sampling, our agent samples a whole episode and successively propagates the value of a state to its previous states.…

Thompson Sampling reduces regret in non-episodic restless bandits.

VO$Q$L optimizes RL with sparse rewards using weighted bounds.

Rewards are sparse in the real world and most of today's reinforcement learning algorithms struggle with such sparsity. One solution to this problem is to allow the agent to create rewards for itself - thus making rewards dense and more suitable for learning. In particular, inspired by curious behaviour in animals, obs…

MEMEC improves sample efficiency in reinforcement learning.

We propose a new method for learning from a single demonstration to solve hard exploration tasks like the Atari game Montezuma's Revenge. Instead of imitating human demonstrations, as proposed in other recent works, our approach is to maximize rewards directly. Our agent is trained using off-the-shelf reinforcement lea…

We initiate the study of multi-stage episodic reinforcement learning under adversarial corruptions in both the rewards and the transition probabilities of the underlying system extending recent results for the special case of stochastic bandits. We provide a framework which modifies the aggressive exploration enjoyed b…

We present a new deep meta reinforcement learner, which we call Deep Episodic Value Iteration (DEVI). DEVI uses a deep neural network to learn a similarity metric for a non-parametric model-based reinforcement learning algorithm. Our model is trained end-to-end via back-propagation. Despite being trained using the mode…

New algorithm tackles adversarial RL without horizon constraints.

New method trains GFlowNets from partial episodes to improve convergence and stability.

A new algorithm for offline RL with trajectory-wise reward reduces bias and variance errors.

Algorithm for online learning in MDPs with linear function approximation and bandit feedback.

Automatically generates curricula for reinforcement learning agents.

Two algorithms tackle heavy-tailed rewards in reinforcement learning with linear function approximation.

Proposes PIC and POIC for measuring task difficulty in RL.

We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…

Align-RUDDER improves reinforcement learning with few demonstrations by redistributing rewards.

Improved reinforcement learning for episodes with varying action sets.

Algorithm improves RL model selection for repeated games with utility maximization.

New algorithm reduces dynamic regret for MDPs with unknown transition and adversarial rewards.

Efficient algorithm for learning from indirect feedback in complex decision-making scenarios.

New approach for reward-free exploration reduces estimation error.

Deep Deterministic Policy Gradient (DDPG) has been proved to be a successful reinforcement learning (RL) algorithm for continuous control tasks. However, DDPG still suffers from data insufficiency and training inefficiency, especially in computationally complex environments. In this paper, we propose Asynchronous Episo…

SMRL uses score matching for efficient RL with exponential family models.