arXiv research

This work explores representation complexity in RL paradigms, revealing model-based RL as the easiest task.

New method defends RL agents from poisoning attacks without MDP knowledge.

New GFlowNet training framework using policy gradients for combinatorial object generation.

Develops ODRPO to improve RL algorithms with better performance and stability.

The goal of reinforcement learning (RL) is to let an agent learn an optimal control policy in an unknown environment so that future expected rewards are maximized. The model-free RL approach directly learns the policy based on data samples. Although using many samples tends to improve the accuracy of policy learning, c…

Survey of RL in finance, tackling complex decision-making.

New algorithm reduces RL policy optimization gap.

New RL algorithm tackles non-stationary environments with flexible policy updates.

We establish a new connection between value and policy based reinforcement learning (RL) based on a relationship between softmax temporal value consistency and policy optimality under entropy regularization. Specifically, we show that softmax consistent action values correspond to optimal entropy regularized policy pro…

Privileged Information Dropout improves RL performance without distillation.

Policy optimization is a core component of reinforcement learning (RL), and most existing RL methods directly optimize parameters of a policy based on maximizing the expected total reward, or its surrogate. Though often achieving encouraging empirical success, its underlying mathematical principle on {\em policy-distri…

MOReL learns offline RL policies using pessimistic MDPs.

Direct and indirect RL methods classified and compared.

Modern deep learning methods provide effective means to learn good representations. However, is a good representation itself sufficient for sample efficient reinforcement learning? This question has largely been studied only with respect to (worst-case) approximation error, in the more classical approximate dynamic pro…

RL agents outperform baselines in asset allocation.

The paper reclassifies RL algorithms using inference concepts.

Paper uses RL to optimize ICU load during COVID-19.

There are great interests as well as many challenges in applying reinforcement learning (RL) to recommendation systems. In this setting, an online user is the environment; neither the reward function nor the environment dynamics are clearly defined, making the application of RL challenging. In this paper, we propose a …

Neuroscientific theory suggests that dopaminergic neurons broadcast global reward prediction errors to large areas of the brain influencing the synaptic plasticity of the neurons in those regions. We build on this theory to propose a multi-agent learning framework with spiking neurons in the generalized linear model (G…

Real-world tasks are often highly structured. Hierarchical reinforcement learning (HRL) has attracted research interest as an approach for leveraging the hierarchical structure of a given task in reinforcement learning (RL). However, identifying the hierarchical policy structure that enhances the performance of RL is n…

Study risk-sensitive RL in offline settings, improving efficiency and accuracy.

Value-based reinforcement learning (RL) methods like Q-learning have shown success in a variety of domains. One challenge in applying Q-learning to continuous-action RL problems, however, is the continuous action maximization (max-Q) required for optimal Bellman backup. In this work, we develop CAQL, a (class of) algor…

Financial portfolio management is one of the problems that are most frequently encountered in the investment industry. Nevertheless, it is not widely recognized that both Kelly Criterion and Risk Parity collapse into Mean Variance under some conditions, which implies that a universal solution to the portfolio optimizat…

Pessimistic RL algorithm improves offline RL performance.

Papers learn from data to make decisions without interacting, improving on previous methods.

We study the problem of off-policy value evaluation in reinforcement learning (RL), where one aims to estimate the value of a new policy based on data collected by a different policy. This problem is often a critical step when applying RL in real-world problems. Despite its importance, existing general methods either h…

In this paper, we study a multi-step interactive recommendation problem, where the item recommended at current step may affect the quality of future recommendations. To address the problem, we develop a novel and effective approach, named CFRL, which seamlessly integrates the ideas of both collaborative filtering (CF) …

New Bellman error estimator improves offline model selection performance.

OPPO optimizes policy learning with provable efficiency.

Study uses RL to optimize risky vs. risk-free asset allocation.

We solve POMDPs by approximating them as finite-state MDPs.

Recent studies have shown that reinforcement learning (RL) models are vulnerable in various noisy scenarios. For instance, the observed reward channel is often subject to noise in practice (e.g., when rewards are collected through sensors), and is therefore not credible. In addition, for applications such as robotics, …

ENIAC method optimizes and explores complex RL problems with non-linear policies.

Mixreg improves RL generalization by mixing diverse training environments.

Overparameterized models generalize well in offline contextual bandits, but policy-based algorithms struggle.

Unified framework for policy learning using weak supervision.

Step-DAD improves BED by periodically updating a design policy during experiments.

Model-free approaches for reinforcement learning (RL) and continuous control find policies based only on past states and rewards, without fitting a model of the system dynamics. They are appealing as they are general purpose and easy to implement; however, they also come with fewer theoretical guarantees than model-bas…

This work improves policy-based training by proposing an evaluation balance objective for GFlowNets.

In artificial intelligence, we often specify tasks through a reward function. While this works well in some settings, many tasks are hard to specify this way. In deep reinforcement learning, for example, directly specifying a reward as a function of a high-dimensional observation is challenging. Instead, we present an …

Recent years have witnessed significant advances in reinforcement learning (RL), which has registered great success in solving various sequential decision-making problems in machine learning. Most of the successful RL applications, e.g., the games of Go and Poker, robotics, and autonomous driving, involve the participa…

New actor-critic algorithm achieves optimal sample efficiency in RL.

Optimistic PPO variant solves linear MDPs with improved regret bound.

New algorithm for RL with horizon-free reward-free exploration for linear MDPs.

New algorithm reduces dynamic regret for MDPs with unknown transition and adversarial rewards.

Random layer-wise pruning profiles are as effective as metric-based ones for various datasets.

Off-policy deep reinforcement learning (RL) algorithms are incapable of learning solely from batch offline data without online interactions with the environment, due to the phenomenon known as \textit{extrapolation error}. This is often due to past data available in the replay buffer that may be quite different from th…

Many complex domains, such as robotics control and real-time strategy (RTS) games, require an agent to learn a continuous control. In the former, an agent learns a policy over $\mathbb{R}^d$ and in the latter, over a discrete set of actions each of which is parametrized by a continuous parameter. Such problems are natu…