arXiv research

Double Q-learning has the same mean-squared error as Q-learning under certain conditions.

Paper addresses underestimation bias in double Q-learning, proposing a method to improve learning performance.

Paper analyzes finite-time convergence of double Q-learning.

Currently, many applications in Machine Learning are based on define new models to extract more information about data, In this case Deep Reinforcement Learning with the most common application in video games like Atari, Mario, and others causes an impact in how to computers can learning by himself with only informatio…

EBQL reduces bias in Q-learning for improved performance.

A novel Q-learning variant reduces underestimation bias in deep actor-critic methods for reinforcement learning.

Boosts Q-learning by using value function bounds.

Finding the optimal signal timing strategy is a difficult task for the problem of large-scale traffic signal control (TSC). Multi-Agent Reinforcement Learning (MARL) is a promising method to solve this problem. However, there is still room for improvement in extending to large-scale problems and modeling the behaviors …

The use of target networks has been a popular and key component of recent deep Q-learning algorithms for reinforcement learning, yet little is known from the theory side. In this work, we introduce a new family of target-based temporal difference (TD) learning algorithms and provide theoretical analysis on their conver…

This paper describes an improvement in Deep Q-learning called Reverse Experience Replay (also RER) that solves the problem of sparse rewards and helps to deal with reward maximizing tasks by sampling transitions successively in reverse order. On tasks with enough experience for training and enough Experience Replay mem…

Temporal-difference (TD) learning is an important field in reinforcement learning. Sarsa and Q-Learning are among the most used TD algorithms. The Q($σ$) algorithm (Sutton and Barto (2017)) unifies both. This paper extends the Q($σ$) algorithm to an online multi-step algorithm Q($σ, λ$) using eligibility traces and int…

Optimal trade execution is an important problem faced by essentially all traders. Much research into optimal execution uses stringent model assumptions and applies continuous time stochastic control to solve them. Here, we instead take a model free approach and develop a variation of Deep Q-Learning to estimate the opt…

RL agent learns to place limit orders for trading signals in financial markets.

Proposes a new policy gradient algorithm to improve reinforcement learning efficiency and stability.

Drawing an inspiration from behavioral studies of human decision making, we propose here a more general and flexible parametric framework for reinforcement learning that extends standard Q-learning to a two-stream model for processing positive and negative rewards, and allows to incorporate a wide range of reward-proce…

In value-based reinforcement learning methods such as deep Q-learning, function approximation errors are known to lead to overestimated value estimates and suboptimal policies. We show that this problem persists in an actor-critic setting and propose novel mechanisms to minimize its effects on both the actor and the cr…

We introduce a novel Deep Reinforcement Learning (DRL) algorithm called Deep Quality-Value (DQV) Learning. DQV uses temporal-difference learning to train a Value neural network and uses this network for training a second Quality-value network that learns to estimate state-action values. We first test DQV's update rules…

With the increasing complexity of modern power systems, conventional dynamic load modeling with ZIP and induction motors (ZIP + IM) is no longer adequate to address the current load characteristic transitions. In recent years, the WECC composite load model (WECC CLM) has shown to effectively capture the dynamic load re…

Optimizes mobile notifications for multiple objectives using reinforcement learning.

Paper uses RL to optimize trading in time-varying liquidity markets.

In classical Q-learning, the objective is to maximize the sum of discounted rewards through iteratively using the Bellman equation as an update, in an attempt to estimate the action value function of the optimal policy. Conventionally, the loss function is defined as the temporal difference between the action value and…

The impact of softmax on the value function itself in reinforcement learning (RL) is often viewed as problematic because it leads to sub-optimal value (or Q) functions and interferes with the contraction properties of the Bellman operator. Surprisingly, despite these concerns, and independent of its effect on explorati…

Paper analyzes Q-learning with neural networks, proving a fast convergence rate.

DE-QT detects optimal Q-learning stopping points.

This paper analyzes momentum Q-learning with finite-sample guarantees.

PQ-learning improves Q-learning by periodically updating target estimates.

New self-imitation learning method improves performance in continuous control tasks.

Linear Q-learning converges to a bounded set without divergence.

We present a framework, which we call Molecule Deep $Q$-Networks (MolDQN), for molecule optimization by combining domain knowledge of chemistry and state-of-the-art reinforcement learning techniques (double $Q$-learning and randomized value functions). We directly define modifications on molecules, thereby ensuring 100…

Q-learning requires more samples than minimax bounds suggest for optimal Q-function approximation.

Q-learning for average cost MDPs gets a concentration bound.

New algorithm uses Whittle index to improve Q-learning for restless bandits.

This paper makes one step forward towards characterizing a new family of \textit{model-free} Deep Reinforcement Learning (DRL) algorithms. The aim of these algorithms is to jointly learn an approximation of the state-value function ($V$), alongside an approximation of the state-action value function ($Q$). Our analysis…

LBQL improves Q-learning by using lookahead bounds for better performance.

This paper analyzes how periodic and soft target updates stabilize linear Q-learning.

Kernelized Q-learning achieves good performance with minimal data.

New algorithm reduces sample and communication complexities in federated Q-learning.

This paper improves Q-learning bounds using reference-advantage decomposition.

Study Whittle index learning algorithms for restless bandits with constant stepsizes.

Extended analysis of Q-learning's efficiency, matching optimal regret.

Proposes a new Q-learning method for survival outcomes in clinical trials.

AQL uses amortized inference to handle high-dimensional action spaces in Q-learning.

Robust $Q$-learning for mean-field control under Wasserstein uncertainty

Q-learning is one of the most popular methods in Reinforcement Learning (RL). Transfer Learning aims to utilize the learned knowledge from source tasks to help new tasks to improve the sample complexity of the new tasks. Considering that data collection in RL is both more time and cost consuming and Q-learning converge…

This paper introduces sample-averaged Q-learning for better RL performance.

Improved sample complexity for target Q-learning in finite MDPs with generative oracle.

Proposes a robust Q-learning method to improve treatment strategy estimation.

A new Q-learning controller improves line follower robot control.