arXiv research

Single episode policy transfer in RL without access to rewards.

Algorithm tackles constrained reinforcement learning with concave-convex and knapsack constraints.

We consider online learning in episodic loop-free Markov decision processes (MDPs), where the loss function can change arbitrarily between episodes, and the transition function is not known to the learner. We show $\tilde{O}(L|X|\sqrt{|A|T})$ regret bound, where $T$ is the number of episodes, $X$ is the state space, $A…

Study preference-based reinforcement learning in episodic kernel MDPs.

SPAQL improves RL by adaptively partitioning state-action space and learning a time-invariant policy.

In continual learning (CL), an agent learns from a stream of tasks leveraging prior experience to transfer knowledge to future tasks. It is an ideal framework to decrease the amount of supervision in the existing learning algorithms. But for a successful knowledge transfer, the learner needs to remember how to perform …

Freezing of gait (FoG) is a common gait disability in Parkinson's disease, that usually appears in its advanced stage. Freeze episodes are associated with falls, injuries, and psychological consequences, negatively affecting the patients' quality of life. For detecting FoG episodes automatically, a highly accurate dete…

Current deep neural networks can achieve remarkable performance on a single task. However, when the deep neural network is continually trained on a sequence of tasks, it seems to gradually forget the previous learned knowledge. This phenomenon is referred to as \textit{catastrophic forgetting} and motivates the field c…

We propose a new method for learning from a single demonstration to solve hard exploration tasks like the Atari game Montezuma's Revenge. Instead of imitating human demonstrations, as proposed in other recent works, our approach is to maximize rewards directly. Our agent is trained using off-the-shelf reinforcement lea…

Quantile gradient boosted trees outperform other models in predicting NO2 concentration distributions.

Efficient algorithm for learning from indirect feedback in complex decision-making scenarios.

Bootstrap method for Markov chains in reinforcement learning.

This paper analyzes arbitrage opportunities in Polymarket's NBA markets.

Algorithm improves multi-armed bandit performance by transferring reward samples.

BerlinUCB learns from episodic rewards in nonstationary contexts.

Algorithm reduces episode count for CMDPs with constraints.

Agents struggle with solving tasks in new environments, but new models improve performance.

Many interesting real world domains involve reinforcement learning (RL) in partially observable environments. Efficient learning in such domains is important, but existing sample complexity bounds for partially observable RL are at least exponential in the episode length. We give, to our knowledge, the first partially …

Algorithm improves online learning in adversarial bandits.

The majority of ML research concerns slow, statistical learning of i.i.d. samples from large, labelled datasets. Animals do not learn this way. An enviable characteristic of animal learning is `episodic' learning - the ability to memorise a specific experience as a composition of existing concepts, after just one exper…

This paper sets communication complexity bounds for distributed RL.

Reinforcement learning (RL) algorithms have made huge progress in recent years by leveraging the power of deep neural networks (DNN). Despite the success, deep RL algorithms are known to be sample inefficient, often requiring many rounds of interaction with the environments to obtain satisfactory performance. Recently,…

We introduce a new class of reinforcement learning methods referred to as {\em episodic multi-armed bandits} (eMAB). In eMAB the learner proceeds in {\em episodes}, each composed of several {\em steps}, in which it chooses an action and observes a feedback signal. Moreover, in each step, it can take a special action, c…

Paper proposes DAC-ML, a cognitive architecture that learns quickly from few episodes.

Paper proposes a method to optimize policies for diverse individuals using heterogeneous data.

In this paper we address the problem of discovering a small set of frequent serial episodes from sequential data so as to adequately characterize or summarize the data. We discuss an algorithm based on the Minimum Description Length (MDL) principle and the algorithm is a slight modification of an earlier method, called…

We propose Episodic Backward Update (EBU) - a novel deep reinforcement learning algorithm with a direct value propagation. In contrast to the conventional use of the experience replay with uniform random sampling, our agent samples a whole episode and successively propagates the value of a state to its previous states.…

New rule reduces exploration regret to logarithmic, improving bad episode handling.

Recurrent major mood episodes and subsyndromal mood instability cause substantial disability in patients with bipolar disorder. Early identification of mood episodes enabling timely mood stabilisation is an important clinical goal. Recent technological advances allow the prospective reporting of mood in real time enabl…

This study optimizes DRL for American option hedging with new training methods.

Federated UCBVI reduces communication costs while minimizing regret in multi-agent settings.

Scaling laws found for reinforcement learning performance with model size and compute.

We introduce a lifelong language learning setup where a model needs to learn from a stream of text examples without any dataset identifier. We propose an episodic memory model that performs sparse experience replay and local adaptation to mitigate catastrophic forgetting in this setup. Experiments on text classificatio…

We introduce an off-policy evaluation procedure for highlighting episodes where applying a reinforcement learned (RL) policy is likely to have produced a substantially different outcome than the observed policy. In particular, we introduce a class of structural causal models (SCMs) for generating counterfactual traject…

Model-free reinforcement learning (RL) algorithms, such as Q-learning, directly parameterize and update value functions or policies without explicitly modeling the environment. They are typically simpler, more flexible to use, and thus more prevalent in modern deep RL than model-based approaches. However, empirical wor…

New algorithms reduce dynamic regret in online MDPs with changing losses.

Extends Thompson sampling for RL with fewer episodes.

Improved reinforcement learning for episodes with varying action sets.

We investigate the robustness properties of image recognition models equipped with two features inspired by human vision, an explicit episodic memory and a shape bias, at the ImageNet scale. As reported in previous work, we show that an explicit episodic memory improves the robustness of image recognition models agains…

Paper proposes new $γ$-regret measure for non-episodic RL.

Recent advances in deep reinforcement learning algorithms have shown great potential and success for solving many challenging real-world problems, including Go game and robotic applications. Usually, these algorithms need a carefully designed reward function to guide training in each time step. However, in real world, …

New algorithm for average reward learning with bounded hitting time assumption.

New RL algorithms reduce costs for single-agent and federated learning.

This paper explores how environmental properties can simplify reinforcement learning in non-episodic settings.

MEMEC improves sample efficiency in reinforcement learning.

New method learns near-optimal policies with polynomial samples in A and H.

Study finds stock prices rarely appreciate during capital inflows but often appreciate during normal flows.

Paper proposes RRD to learn proxy rewards for sparse delayed rewards in episodic reinforcement learning.