arXiv research

Investigates Q value evolution in Stable Baselines for DQL in simple vs complex environments.

The paper improves energy decay estimates for Dir-stationary Q-valued functions and applies them to Liouville-type theorems and continuity.

A new DQN algorithm improves portfolio management and risk assessment in digital assets.

A new method stabilizes deep reinforcement learning by using QGraphs to retain replay memory information.

Q-Distribution Guided Q-Learning corrects overestimation of uncertain OOD actions in offline RL.

DNAF accelerates DQL for efficient resource allocation in network slicing.

This work uses variational quantum circuits for deep reinforcement learning.

We study reinforcement learning (RL) in high dimensional episodic Markov decision processes (MDP). We consider value-based RL when the optimal Q-value is a linear function of d-dimensional state-action feature representation. For instance, in deep-Q networks (DQN), the Q-value is a linear function of the feature repres…

We methodologically address the problem of Q-value overestimation in deep reinforcement learning to handle high-dimensional state spaces efficiently. By adapting concepts from information theory, we introduce an intrinsic penalty signal encouraging reduced Q-value estimates. The resultant algorithm encompasses a wide r…

Proposes Optimistic Pessimistically Initialised Q-Learning (OPIQ) for better exploration in RL.

Improved Q-learning for multi-agent reinforcement learning by weighting joint action values.

The paper proves the convergence of Q-value for Gaussian rewards.

The paper studies uncertainty quantification and exploration in RL, providing methods and results.

Equivariant CNNs improve RL performance in symmetric environments.

We analyze a notion of multiple valued sections of a vector bundle over an abstract smooth Riemannian manifold, which was suggested by W. Allard in the unpublished note "Some useful techniques for dealing with multiple valued functions" and generalizes Almgren's $Q$-valued functions. We study some relevant properties o…

Instability and variability of Deep Reinforcement Learning (DRL) algorithms tend to adversely affect their performance. Averaged-DQN is a simple extension to the DQN algorithm, based on averaging previously learned Q-values estimates, which leads to a more stable training procedure and improved performance by reducing …

Proves continuity and singular set dimension for 2D maps with Q values.

New approach optimizes policies in adversarial MDPs using adversarial learning.

A novel Q-learning variant reduces underestimation bias in deep actor-critic methods for reinforcement learning.

New RL theory predicts deep RL success based on greedy actions under random policies.

Proposes a value-based method for continuous control without an actor.

Improves neural networks' ability to learn new tasks without forgetting earlier ones.

A new Q-learning variant reduces underestimation bias in deep reinforcement learning.

Deep neural networks can estimate Q-values efficiently on low-dimensional state-action spaces.

Study Q-learning with averaging for reinforcement learning, proving efficient inference and error bounds.

GRAC improves reinforcement learning by self-guiding and self-regularizing.

Paper improves RL algorithms with known optimal Q-value predictions.

Optimal trade execution is an important problem faced by essentially all traders. Much research into optimal execution uses stringent model assumptions and applies continuous time stochastic control to solve them. Here, we instead take a model free approach and develop a variation of Deep Q-Learning to estimate the opt…

POWSS simplifies Q-value estimation in POMDPs with continuous observations.

New method estimates optimal Q-values with better accuracy for specific problems.

In a discounted reward Markov Decision Process (MDP), the objective is to find the optimal value function, i.e., the value function corresponding to an optimal policy. This problem reduces to solving a functional equation known as the Bellman equation and a fixed point iteration scheme known as the value iteration is u…

Deep Reinforcement Learning (RL) recently emerged as one of the most competitive approaches for learning in sequential decision making problems with fully observable environments, e.g., computer Go. However, very little work has been done in deep RL to handle partially observable environments. We propose a new architec…

The paper proposes a method to infer Q-values online with Q-Learning.

This research introduces an autonomous robot navigation method using reinforcement learning.

Proposes a deep RL framework for optimizing task allocation in crowdsourcing platforms.

This work explores Target Networks and Functional Regularization in deep Reinforcement Learning.

A framework for estimating both epistemic and aleatoric uncertainties in reinforcement learning.

Bayesian approach improves $\varepsilon$-greedy exploration in RL.

Two new RL methods enable deep learning of MFG equilibria.

We construct Lipschitz $Q$-valued functions which approximate carefully integral currents when their cylindrical excess is small and they are almost minimizing in a suitable sense. This result is used in two subsequent works to prove the discreteness of the singular set for the following three classes of $2$-dimensiona…

We propose RUDDER, a novel reinforcement learning approach for delayed rewards in finite Markov decision processes (MDPs). In MDPs the Q-values are equal to the expected immediate reward plus the expected future rewards. The latter are related to bias problems in temporal difference (TD) learning and to high variance p…

Policy gradient is an efficient technique for improving a policy in a reinforcement learning setting. However, vanilla online variants are on-policy only and not able to take advantage of off-policy data. In this paper we describe a new technique that combines policy gradient with off-policy Q-learning, drawing experie…

Combining deep model-free reinforcement learning with on-line planning is a promising approach to building on the successes of deep RL. On-line planning with look-ahead trees has proven successful in environments where transition models are known a priori. However, in complex environments where transition models need t…

New method finds unseen states for RL, improving performance.

Proposes a new policy gradient algorithm to improve reinforcement learning efficiency and stability.

Bayes-UCBVI tackles reinforcement learning with a new upper confidence bound method.

CQL learns conservative Q-functions to improve offline RL performance.

In reinforcement learning the Q-values summarize the expected future rewards that the agent will attain. However, they cannot capture the epistemic uncertainty about those rewards. In this work we derive a new Bellman operator with associated fixed point we call the `knowledge values'. These K-values compress both the …