arXiv research

Study assesses health plan risk measures for Solvency Capital Requirement.

Agents struggle with solving tasks in new environments, but new models improve performance.

Long horizon reinforcement learning is as hard as short horizon learning.

New algorithm reduces reinforcement learning complexity, approaching contextual bandits.

Novel approach to learning models based on subjective timescales for better exploration and decision-making.

Algorithm reduces episode count for CMDPs with constraints.

New algorithms for planning with adversarial changes in costs.

Paper addresses offline policy evaluation in RL, achieving near-optimal bounds for various policy classes.

Planning has been very successful for control tasks with known environment dynamics. To leverage planning in unknown environments, the agent needs to learn the dynamics from interactions with the world. However, learning dynamics models that are accurate enough for planning has been a long-standing challenge, especiall…

We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…

New model for personalized online advertising with multi-user interaction.

Retro* uses neural networks to efficiently find high-quality synthetic routes in organic chemistry.

RandQL is a new model-free algorithm for MDPs with a novel learning rate randomization approach.

New algorithm reduces regret in stochastic shortest path problems.

New RL algorithm achieves nearly optimal performance for linear MDPs.

NIPA aims to translate brain learning mechanisms into scalable Bayesian inference.

Algorithm learns multiple tasks with minimal planning, achieving near-optimal performance.

New algorithm learns policies without explicit rewards for MDPs.

In this paper, we propose a decision making algorithm intended for automated vehicles that negotiate with other possibly non-automated vehicles in intersections. The decision algorithm is separated into two parts: a high-level decision module based on reinforcement learning, and a low-level planning module based on mod…

New algorithm tackles adversarial RL without horizon constraints.

Model predicts wound and episode-level readmission risk and time to re-admit.

New RL algorithm reduces regret in finite-horizon episodic tasks.

New algorithm for RL with horizon-free reward-free exploration for linear MDPs.

ZoomRL learns efficient strategies for large state-action spaces using a metric.

Optimistic algorithm reduces regret in non-stationary linear MDPs.

Managing risk in dynamic decision problems is of cardinal importance in many fields such as finance and process control. The most common approach to defining risk is through various variance related criteria such as the Sharpe Ratio or the standard deviation adjusted reward. It is known that optimizing many of the vari…

New RL method learns K-step lookahead Q-functions for fixed-horizon MDPs.

Study preference-based reinforcement learning in episodic kernel MDPs.

The paper analyzes CMDPs, balancing exploration and exploitation to avoid constraint violations.

TensorPlan algorithm finds δ-optimal policies with poly$(H,d)$ queries under linearly realizable state-value function.

Algorithm tackles constrained reinforcement learning with concave-convex and knapsack constraints.

Algorithm improves multi-armed bandit performance by transferring reward samples.

BerlinUCB learns from episodic rewards in nonstationary contexts.

We consider online learning in episodic loop-free Markov decision processes (MDPs), where the loss function can change arbitrarily between episodes, and the transition function is not known to the learner. We show $\tilde{O}(L|X|\sqrt{|A|T})$ regret bound, where $T$ is the number of episodes, $X$ is the state space, $A…

We introduce two tactics to attack agents trained by deep reinforcement learning algorithms using adversarial examples, namely the strategically-timed attack and the enchanting attack. In the strategically-timed attack, the adversary aims at minimizing the agent's reward by only attacking the agent at a small subset of…

Many interesting real world domains involve reinforcement learning (RL) in partially observable environments. Efficient learning in such domains is important, but existing sample complexity bounds for partially observable RL are at least exponential in the episode length. We give, to our knowledge, the first partially …

Algorithm improves online learning in adversarial bandits.

Reinforcement learning (RL) algorithms have made huge progress in recent years by leveraging the power of deep neural networks (DNN). Despite the success, deep RL algorithms are known to be sample inefficient, often requiring many rounds of interaction with the environments to obtain satisfactory performance. Recently,…

Logarithmic regret achieved in Q-learning with positive gap.

We introduce a new class of reinforcement learning methods referred to as {\em episodic multi-armed bandits} (eMAB). In eMAB the learner proceeds in {\em episodes}, each composed of several {\em steps}, in which it chooses an action and observes a feedback signal. Moreover, in each step, it can take a special action, c…

Paper proposes DAC-ML, a cognitive architecture that learns quickly from few episodes.

In this paper we address the problem of discovering a small set of frequent serial episodes from sequential data so as to adequately characterize or summarize the data. We discuss an algorithm based on the Minimum Description Length (MDL) principle and the algorithm is a slight modification of an earlier method, called…

We propose Episodic Backward Update (EBU) - a novel deep reinforcement learning algorithm with a direct value propagation. In contrast to the conventional use of the experience replay with uniform random sampling, our agent samples a whole episode and successively propagates the value of a state to its previous states.…

New rule reduces exploration regret to logarithmic, improving bad episode handling.

Recurrent major mood episodes and subsyndromal mood instability cause substantial disability in patients with bipolar disorder. Early identification of mood episodes enabling timely mood stabilisation is an important clinical goal. Recent technological advances allow the prospective reporting of mood in real time enabl…

FTPL method shows near-optimal regret bounds for AMDPs with bandit feedback.

We introduce a lifelong language learning setup where a model needs to learn from a stream of text examples without any dataset identifier. We propose an episodic memory model that performs sparse experience replay and local adaptation to mitigate catastrophic forgetting in this setup. Experiments on text classificatio…

We introduce an off-policy evaluation procedure for highlighting episodes where applying a reinforcement learned (RL) policy is likely to have produced a substantially different outcome than the observed policy. In particular, we introduce a class of structural causal models (SCMs) for generating counterfactual traject…