arXiv research

This work explores representation complexity in RL paradigms, revealing model-based RL as the easiest task.

Unified framework for CO problems using RL, providing optimal solutions and convergence guarantees.

Value-based methods constitute a fundamental methodology in planning and deep reinforcement learning (RL). In this paper, we propose to exploit the underlying structures of the state-action value function, i.e., Q function, for both planning and deep RL. In particular, if the underlying system dynamics lead to some glo…

New RL algorithms adapt to time limits, improving task performance.

Stochastic Q-learning tackles large action spaces with reduced computation.

New RL algorithm handles delayed feedback with posterior sampling.

Reincarnating RL reuses prior work to accelerate RL progress.

This paper shows using classification instead of regression improves deep RL scalability.

RL teaches agents to make decisions through trial and error.

The paper explores how different loss functions impact reinforcement learning algorithms.

Reinforcement learning (RL) typically defines a discount factor as part of the Markov Decision Process. The discount factor values future rewards by an exponential scheme that leads to theoretical convergence guarantees of the Bellman equation. However, evidence from psychology, economics and neuroscience suggests that…

We investigate statistical uncertainty quantification for reinforcement learning (RL) and its implications in exploration policy. Despite ever-growing literature on RL applications, fundamental questions about inference and error quantification, such as large-sample behaviors, appear to remain quite open. In this paper…

RL tackles decision making in unknown environments, focusing on efficiency and efficacy.

Modern deep learning methods provide effective means to learn good representations. However, is a good representation itself sufficient for sample efficient reinforcement learning? This question has largely been studied only with respect to (worst-case) approximation error, in the more classical approximate dynamic pro…

Privileged Information Dropout improves RL performance without distillation.

Unified framework for finite-sample RL algorithms using Lyapunov theory.

Direct and indirect RL methods classified and compared.

RL agents outperform baselines in asset allocation.

Under-parameterization hinders deep RL's efficiency.

SUNRISE improves off-policy RL algorithms by integrating ensemble methods.

New GFlowNet training framework using policy gradients for combinatorial object generation.

New RL method learns from state transitions without actions.

In this work, we consider the problem of model selection for deep reinforcement learning (RL) in real-world environments. Typically, the performance of deep RL algorithms is evaluated via on-policy interactions with the target environment. However, comparing models in a real-world environment for the purposes of early …

Survey of RL in finance, tackling complex decision-making.

New method improves policies by combining Markov and non-Markov strategies.

New algorithms estimate Q-functions under partial coverage and realizability, improving offline RL guarantees.

Deep RL controls anesthesia more accurately than traditional methods.

Most practical recommender systems focus on estimating immediate user engagement without considering the long-term effects of recommendations on user behavior. Reinforcement learning (RL) methods offer the potential to optimize recommendations for long-term user engagement. However, since users are often presented with…

Paper introduces efficient online sub-sampling for RL with function approximation, reducing policy updates.

Value-based reinforcement learning (RL) methods like Q-learning have shown success in a variety of domains. One challenge in applying Q-learning to continuous-action RL problems, however, is the continuous action maximization (max-Q) required for optimal Bellman backup. In this work, we develop CAQL, a (class of) algor…

New RL algorithm tackles non-stationary environments with flexible policy updates.

Financial portfolio management is one of the problems that are most frequently encountered in the investment industry. Nevertheless, it is not widely recognized that both Kelly Criterion and Risk Parity collapse into Mean Variance under some conditions, which implies that a universal solution to the portfolio optimizat…

BCPO optimizes offline RL policies by converting uncertainty into conservative bounds.

We study reinforcement learning (RL) in high dimensional episodic Markov decision processes (MDP). We consider value-based RL when the optimal Q-value is a linear function of d-dimensional state-action feature representation. For instance, in deep-Q networks (DQN), the Q-value is a linear function of the feature repres…

DRL agents perform poorly at high decision frequencies, but a new algorithm improves performance.

DR-RPO optimizes robust policies in RL with limited interaction, achieving sublinear regret.

This paper proposes a new approach to RL by focusing on the value-improvement path.

OPPO optimizes policy learning with provable efficiency.

Distributional approaches to value-based reinforcement learning model the entire distribution of returns, rather than just their expected values, and have recently been shown to yield state-of-the-art empirical performance. This was demonstrated by the recently proposed C51 algorithm, based on categorical distributiona…

Mixreg improves RL generalization by mixing diverse training environments.

Paper tackles sample-efficient RL for linearly realizable MDPs with limited revisiting.

This work explores efficient reinforcement learning with density features in low-rank MDPs.

This paper proposes a new optimization objective for value-based deep reinforcement learning. We extend conventional Deep Q-Networks (DQNs) by adding a model-learning component yielding a transcoder network. The prediction errors for the model are included in the basic DQN loss as additional regularizers. This augmente…

Reward shaping is one of the most effective methods to tackle the crucial yet challenging problem of credit assignment in Reinforcement Learning (RL). However, designing shaping functions usually requires much expert knowledge and hand-engineering, and the difficulties are further exacerbated given multiple similar tas…

EarnHFT tackles HFT challenges with hierarchical RL, significantly outperforming existing methods.

New RL method improves financial index tracking accuracy.

New issue found in value-based reinforcement learning for stochastic environments.

New method improves stability of soft FQI for offline RL.