arXiv research

The paper improves energy decay estimates for Dir-stationary Q-valued functions and applies them to Liouville-type theorems and continuity.

Q-Distribution Guided Q-Learning corrects overestimation of uncertain OOD actions in offline RL.

POWSS simplifies Q-value estimation in POMDPs with continuous observations.

The paper studies uncertainty quantification and exploration in RL, providing methods and results.

Investigates Q value evolution in Stable Baselines for DQL in simple vs complex environments.

New method finds unseen states for RL, improving performance.

Proposes Optimistic Pessimistically Initialised Q-Learning (OPIQ) for better exploration in RL.

Proves continuity and singular set dimension for 2D maps with Q values.

New approach optimizes policies in adversarial MDPs using adversarial learning.

New method estimates optimal Q-values with better accuracy for specific problems.

We methodologically address the problem of Q-value overestimation in deep reinforcement learning to handle high-dimensional state spaces efficiently. By adapting concepts from information theory, we introduce an intrinsic penalty signal encouraging reduced Q-value estimates. The resultant algorithm encompasses a wide r…

A new DQN algorithm improves portfolio management and risk assessment in digital assets.

Proposes a value-based method for continuous control without an actor.

Study Q-learning with averaging for reinforcement learning, proving efficient inference and error bounds.

Reinforcement learning agents are faced with two types of uncertainty. Epistemic uncertainty stems from limited data and is useful for exploration, whereas aleatoric uncertainty arises from stochastic environments and must be accounted for in risk-sensitive applications. We highlight the challenges involved in simultan…

KL regularization helps RL algorithms by implicitly averaging q-values.

A new method stabilizes deep reinforcement learning by using QGraphs to retain replay memory information.

The paper proves the convergence of Q-value for Gaussian rewards.

Policy gradient is an efficient technique for improving a policy in a reinforcement learning setting. However, vanilla online variants are on-policy only and not able to take advantage of off-policy data. In this paper we describe a new technique that combines policy gradient with off-policy Q-learning, drawing experie…

We analyze a notion of multiple valued sections of a vector bundle over an abstract smooth Riemannian manifold, which was suggested by W. Allard in the unpublished note "Some useful techniques for dealing with multiple valued functions" and generalizes Almgren's $Q$-valued functions. We study some relevant properties o…

Instability and variability of Deep Reinforcement Learning (DRL) algorithms tend to adversely affect their performance. Averaged-DQN is a simple extension to the DQN algorithm, based on averaging previously learned Q-values estimates, which leads to a more stable training procedure and improved performance by reducing …

Paper provides estimates for varifolds with critical mean curvature.

We propose RUDDER, a novel reinforcement learning approach for delayed rewards in finite Markov decision processes (MDPs). In MDPs the Q-values are equal to the expected immediate reward plus the expected future rewards. The latter are related to bias problems in temporal difference (TD) learning and to high variance p…

PQ-learning improves Q-learning by periodically updating target estimates.

Paper improves RL algorithms with known optimal Q-value predictions.

Improved Q-learning for multi-agent reinforcement learning by weighting joint action values.

We give a formula of the Upsilon invariant of any L-space cable knot $K_{p,q}$ using $p,Υ_K$ and $Υ_{T_{p,q}}$. The integral value of the Upsilon invariant gives a ${\mathbb Q}$-valued knot concordance invariant. We compute the integral values for L-space iterated cable knots.

Optimal trade execution is an important problem faced by essentially all traders. Much research into optimal execution uses stringent model assumptions and applies continuous time stochastic control to solve them. Here, we instead take a model free approach and develop a variation of Deep Q-Learning to estimate the opt…

The paper proposes a method to infer Q-values online with Q-Learning.

Being able to reach any desired location in the environment can be a valuable asset for an agent. Learning a policy to navigate between all pairs of states individually is often not feasible. An all-goals updating algorithm uses each transition to learn Q-values towards all goals simultaneously and off-policy. However …

Network slicing promises to provision diversified services with distinct requirements in one infrastructure. Deep reinforcement learning (e.g., deep $\mathcal{Q}$-learning, DQL) is assumed to be an appropriate algorithm to solve the demand-aware inter-slice resource management issue in network slicing by regarding the …

Bayesian approach improves $\varepsilon$-greedy exploration in RL.

A novel Q-learning variant reduces underestimation bias in deep actor-critic methods for reinforcement learning.

Proposes a new policy gradient algorithm to improve reinforcement learning efficiency and stability.

We construct Lipschitz $Q$-valued functions which approximate carefully integral currents when their cylindrical excess is small and they are almost minimizing in a suitable sense. This result is used in two subsequent works to prove the discreteness of the singular set for the following three classes of $2$-dimensiona…

Improves neural networks' ability to learn new tasks without forgetting earlier ones.

In a discounted reward Markov Decision Process (MDP), the objective is to find the optimal value function, i.e., the value function corresponding to an optimal policy. This problem reduces to solving a functional equation known as the Bellman equation and a fixed point iteration scheme known as the value iteration is u…

Deep neural networks can estimate Q-values efficiently on low-dimensional state-action spaces.

We study reinforcement learning (RL) in high dimensional episodic Markov decision processes (MDP). We consider value-based RL when the optimal Q-value is a linear function of d-dimensional state-action feature representation. For instance, in deep-Q networks (DQN), the Q-value is a linear function of the feature repres…

POMBU improves model-based RL's asymptotic performance by estimating and using uncertainty.

Deep Q-learning is investigated as an end-to-end solution to estimate the optimal strategies for acting on time series input. Experiments are conducted on two idealized trading games. 1) Univariate: the only input is a wave-like price time series, and 2) Bivariate: the input includes a random stepwise price time series…

Bayes-UCBVI tackles reinforcement learning with a new upper confidence bound method.

QFIL improves offline RL by filtering data to reduce bias and variance.

In this paper we study the singular set of Dirichlet-minimizing $Q$-valued maps from $\mathbb{R}^m$ into a smooth compact manifold $\mathcal{N}$ without boundary. Similarly to what happens in the case of single valued minimizing harmonic maps, we show that this set is always $(m-3)$-rectifiable with uniform Minkowski b…

A new Q-learning variant reduces underestimation bias in deep reinforcement learning.

Calibration without labels in multiple testing

Off-policy reinforcement learning (RL) using a fixed offline dataset of logged interactions is an important consideration in real world applications. This paper studies offline RL using the DQN replay dataset comprising the entire replay experience of a DQN agent on 60 Atari 2600 games. We demonstrate that recent off-p…

A RL approach optimizes metal AM process parameters for consistent melt pool depth.