A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
BerlinUCB learns from episodic rewards in nonstationary contexts.
Algorithm improves multi-armed bandit performance by transferring reward samples.
Paper proposes RRD to learn proxy rewards for sparse delayed rewards in episodic reinforcement learning.
Study preference-based reinforcement learning in episodic kernel MDPs.
This paper explores how environmental properties can simplify reinforcement learning in non-episodic settings.
Recent advances in deep reinforcement learning algorithms have shown great potential and success for solving many challenging real-world problems, including Go game and robotic applications. Usually, these algorithms need a carefully designed reward function to guide training in each time step. However, in real world, …
New algorithm for average reward learning with bounded hitting time assumption.
The success of popular algorithms for deep reinforcement learning, such as policy-gradients and Q-learning, relies heavily on the availability of an informative reward signal at each timestep of the sequential decision-making process. When rewards are only sparsely available during an episode, or a rewarding feedback i…
This paper studies poisoning attacks in episodic RL and discovers their effectiveness depends on reward bounds.
State of the art deep reinforcement learning algorithms take many millions of interactions to attain human-level performance. Humans, on the other hand, can very quickly exploit highly rewarding nuances of an environment upon first discovery. In the brain, such rapid learning is thought to depend on the hippocampus and…
Transforming sparse outcomes into dense process rewards for efficient reinforcement learning.
Paper optimizes learning MDPs with adversarial rewards using linear models.
We introduce a new class of reinforcement learning methods referred to as {\em episodic multi-armed bandits} (eMAB). In eMAB the learner proceeds in {\em episodes}, each composed of several {\em steps}, in which it chooses an action and observes a feedback signal. Moreover, in each step, it can take a special action, c…
Algorithm learns near-optimal policies for reward-mixing MDPs with few latent contexts.
Paper proposes DAC-ML, a cognitive architecture that learns quickly from few episodes.
We introduce an off-policy evaluation procedure for highlighting episodes where applying a reinforcement learned (RL) policy is likely to have produced a substantially different outcome than the observed policy. In particular, we introduce a class of structural causal models (SCMs) for generating counterfactual traject…
New algorithm learns policies without explicit rewards for MDPs.
Algorithm tackles constrained reinforcement learning with concave-convex and knapsack constraints.
New rule reduces exploration regret to logarithmic, improving bad episode handling.
New RL algorithm tackles adversarial RMAB with unknown transitions and bandit feedback.
Extends Thompson sampling for RL with fewer episodes.
New algorithm finds optimal policies without knowing reward functions.
We propose Episodic Backward Update (EBU) - a novel deep reinforcement learning algorithm with a direct value propagation. In contrast to the conventional use of the experience replay with uniform random sampling, our agent samples a whole episode and successively propagates the value of a state to its previous states.…
Transfer and adaptation to new unknown environmental dynamics is a key challenge for reinforcement learning (RL). An even greater challenge is performing near-optimally in a single attempt at test time, possibly without access to dense rewards, which is not addressed by current methods that require multiple experience …
Reinforcement learning (RL) has traditionally been understood from an episodic perspective; the concept of non-episodic RL, where there is no restart and therefore no reliable recovery, remains elusive. A fundamental question in non-episodic RL is how to measure the performance of a learner and derive algorithms to max…
VOL optimizes RL with sparse rewards using weighted bounds.
Rewards are sparse in the real world and most of today's reinforcement learning algorithms struggle with such sparsity. One solution to this problem is to allow the agent to create rewards for itself - thus making rewards dense and more suitable for learning. In particular, inspired by curious behaviour in animals, obs…
Reinforcement learning algorithms use correlations between policies and rewards to improve agent performance. But in dynamic or sparsely rewarding environments these correlations are often too small, or rewarding events are too infrequent to make learning feasible. Human education instead relies on curricula--the break…
We propose a new method for learning from a single demonstration to solve hard exploration tasks like the Atari game Montezuma's Revenge. Instead of imitating human demonstrations, as proposed in other recent works, our approach is to maximize rewards directly. Our agent is trained using off-the-shelf reinforcement lea…
We initiate the study of multi-stage episodic reinforcement learning under adversarial corruptions in both the rewards and the transition probabilities of the underlying system extending recent results for the special case of stochastic bandits. We provide a framework which modifies the aggressive exploration enjoyed b…
We present a new deep meta reinforcement learner, which we call Deep Episodic Value Iteration (DEVI). DEVI uses a deep neural network to learn a similarity metric for a non-parametric model-based reinforcement learning algorithm. Our model is trained end-to-end via back-propagation. Despite being trained using the mode…
New algorithm tackles adversarial RL without horizon constraints.
A new algorithm for offline RL with trajectory-wise reward reduces bias and variance errors.
New method trains GFlowNets from partial episodes to improve convergence and stability.
Algorithm for online learning in MDPs with linear function approximation and bandit feedback.
Deep networks have enabled reinforcement learning to scale to more complex and challenging domains, but these methods typically require large quantities of training data. An alternative is to use sample-efficient episodic control methods: neuro-inspired algorithms which use non-/semi-parametric models that predict valu…
Two algorithms tackle heavy-tailed rewards in reinforcement learning with linear function approximation.
Restless bandit problems assume time-varying reward distributions of the arms, which adds flexibility to the model but makes the analysis more challenging. We study learning algorithms over the unknown reward distributions and prove a sub-linear, , regret bound for a variant of Thompson sampling. Our…
Proposes PIC and POIC for measuring task difficulty in RL.
We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…
Align-RUDDER improves reinforcement learning with few demonstrations by redistributing rewards.
Improved reinforcement learning for episodes with varying action sets.
Algorithm improves RL model selection for repeated games with utility maximization.
New algorithm reduces dynamic regret for MDPs with unknown transition and adversarial rewards.
Efficient algorithm for learning from indirect feedback in complex decision-making scenarios.
New approach for reward-free exploration reduces estimation error.
Deep Deterministic Policy Gradient (DDPG) has been proved to be a successful reinforcement learning (RL) algorithm for continuous control tasks. However, DDPG still suffers from data insufficiency and training inefficiency, especially in computationally complex environments. In this paper, we propose Asynchronous Episo…
SMRL uses score matching for efficient RL with exponential family models.