arXiv research

UDRL fails to converge in stochastic environments with episodic resets.

This paper explores how environmental properties can simplify reinforcement learning in non-episodic settings.

New framework finds periodic policies in reset-free MDPs with sublinear regret.

New method extends supervised learning for non-stationary control problems.

This is a brief technical note to clarify some of the issues with applying the application of the algorithm posterior sampling for reinforcement learning (PSRL) in environments without fixed episodes. In particular, this paper aims to: - Review some of results which have been proven for finite horizon MDPs (Osband et a…

Optimizes search times by resetting agents when a threshold is reached.

We propose a new method for learning from a single demonstration to solve hard exploration tasks like the Atari game Montezuma's Revenge. Instead of imitating human demonstrations, as proposed in other recent works, our approach is to maximize rewards directly. Our agent is trained using off-the-shelf reinforcement lea…

We study how resetting affects geometric Brownian motion, showing it becomes stationary but remains non-ergodic.

The paper solves a pricing problem for a multiple reset put option using integral equations.

Ridge regression linked to Poisson resetting in statistical physics.

Optimal threshold resetting reduces search time for multiple diffusive searchers.

CSER improves SGD efficiency by resetting errors and partial synchronization.

Resetting from checkpoints improves DNN training with noisy labels.

The paper analyzes XVA reduction strategies in financial crises using Mandatory Breaks, Restructuring, and Resets.

Exact results on power-law distributions in systems with resets.

This paper tackles belief-state selection in simulators with latent states.

Study shows MSE with sigmoid can match SCE in classification tasks, especially with noisy data.

Study agnostic RL in large state spaces with weak function approximation.

Deep learning models price convertible bonds with complex reset and call features.

ET-GP-UCB optimizes time-varying functions without knowing change rates.

A variable annuity is an equity-linked financial product typically offered by insurance companies. The policyholder makes an upfront payment to the insurance company and, in return, the insurer is required to make a series of payments starting at an agreed upon date. For a higher premium, many insurance companies offer…

New RL approach uses resets to learn complex MDPs efficiently.

We provide an analytically treatable model that describes in a unified manner income distribution for all income categories. The approach is based on a master equation with growth and reset terms. The model assumptions on the growth and reset rates are tested on an exhaustive database with incomes on individual level s…

Study preference-based reinforcement learning in episodic kernel MDPs.

Algorithm tackles constrained reinforcement learning with concave-convex and knapsack constraints.

Algorithm improves multi-armed bandit performance by transferring reward samples.

BerlinUCB learns from episodic rewards in nonstationary contexts.

We consider online learning in episodic loop-free Markov decision processes (MDPs), where the loss function can change arbitrarily between episodes, and the transition function is not known to the learner. We show $\tilde{O}(L|X|\sqrt{|A|T})$ regret bound, where $T$ is the number of episodes, $X$ is the state space, $A…

Markov switching models (MSMs) are probabilistic models that employ multiple sets of parameters to describe different dynamic regimes that a time series may exhibit at different periods of time. The switching mechanism between regimes is controlled by unobserved random variables that form a first-order Markov chain. Ex…

Deep RL algorithms can overfit to early experiences, leading to poor performance.

GRUwE improves irregular time series prediction with simpler, efficient RNN-based approach.

Tax dynamics affects wealth distribution in a linearly growing socio-economic model.

We consider the sequential Bayesian optimization problem with bandit feedback, adopting a formulation that allows for the reward function to vary with time. We model the reward function using a Gaussian process whose evolution obeys a simple Markov model. We introduce two natural extensions of the classical Gaussian pr…

Algorithm reduces episode count for CMDPs with constraints.

News sentiment in U.S. economic newspapers has become more persistent over 45 years.

Many interesting real world domains involve reinforcement learning (RL) in partially observable environments. Efficient learning in such domains is important, but existing sample complexity bounds for partially observable RL are at least exponential in the episode length. We give, to our knowledge, the first partially …

Algorithm improves online learning in adversarial bandits.

Reinforcement learning (RL) algorithms have made huge progress in recent years by leveraging the power of deep neural networks (DNN). Despite the success, deep RL algorithms are known to be sample inefficient, often requiring many rounds of interaction with the environments to obtain satisfactory performance. Recently,…

We introduce a new class of reinforcement learning methods referred to as {\em episodic multi-armed bandits} (eMAB). In eMAB the learner proceeds in {\em episodes}, each composed of several {\em steps}, in which it chooses an action and observes a feedback signal. Moreover, in each step, it can take a special action, c…

Paper proposes DAC-ML, a cognitive architecture that learns quickly from few episodes.

In this paper we address the problem of discovering a small set of frequent serial episodes from sequential data so as to adequately characterize or summarize the data. We discuss an algorithm based on the Minimum Description Length (MDL) principle and the algorithm is a slight modification of an earlier method, called…

We propose Episodic Backward Update (EBU) - a novel deep reinforcement learning algorithm with a direct value propagation. In contrast to the conventional use of the experience replay with uniform random sampling, our agent samples a whole episode and successively propagates the value of a state to its previous states.…

The paper evaluates three variants of the Gated Recurrent Unit (GRU) in recurrent neural networks (RNN) by reducing parameters in the update and reset gates. We evaluate the three variant GRU models on MNIST and IMDB datasets and show that these GRU-RNN variant models perform as well as the original GRU RNN model while…

New rule reduces exploration regret to logarithmic, improving bad episode handling.

Recurrent major mood episodes and subsyndromal mood instability cause substantial disability in patients with bipolar disorder. Early identification of mood episodes enabling timely mood stabilisation is an important clinical goal. Recent technological advances allow the prospective reporting of mood in real time enabl…

We introduce a lifelong language learning setup where a model needs to learn from a stream of text examples without any dataset identifier. We propose an episodic memory model that performs sparse experience replay and local adaptation to mitigate catastrophic forgetting in this setup. Experiments on text classificatio…

We introduce an off-policy evaluation procedure for highlighting episodes where applying a reinforcement learned (RL) policy is likely to have produced a substantially different outcome than the observed policy. In particular, we introduce a class of structural causal models (SCMs) for generating counterfactual traject…

New algorithms reduce dynamic regret in online MDPs with changing losses.