arXiv research

Online random forests improve Q-learning performance in specific gym environments.

The paper proposes a method to infer Q-values online with Q-Learning.

The use of target networks is a common practice in deep reinforcement learning for stabilizing the training; however, theoretical understanding of this technique is still limited. In this paper, we study the so-called periodic Q-learning algorithm (PQ-learning for short), which resembles the technique used in deep Q-le…

Improved online Q-learning for MDPs with concentration bounds.

Paper proposes methods to handle missing data in online RL, improving efficiency and uncertainty capture.

New Q-learning methods improve sample complexity for MDPs with continuous states.

Pretraining reinforcement learning methods with demonstrations has been an important concept in the study of reinforcement learning since a large amount of computing power is spent on online simulations with existing reinforcement learning algorithms. Pretraining reinforcement learning remains a significant challenge i…

Study Q-learning with averaging for reinforcement learning, proving efficient inference and error bounds.

Policy gradient is an efficient technique for improving a policy in a reinforcement learning setting. However, vanilla online variants are on-policy only and not able to take advantage of off-policy data. In this paper we describe a new technique that combines policy gradient with off-policy Q-learning, drawing experie…

Optimistic Q-learning reduces sample complexity for systems with known dynamics.

A new algorithm SRG-DQN reduces variance in deep Q-learning.

SPQR improves Q-ensemble diversity in reinforcement learning.

QPLEX learns efficient multi-agent Q-values by enforcing IGM principle.

Transfer learning significantly accelerates the reinforcement learning process by exploiting relevant knowledge from previous experiences. The problem of optimally selecting source policies during the learning process is of great importance yet challenging. There has been little theoretical analysis of this problem. In…

Paper improves RL algorithms with known optimal Q-value predictions.

The use of target networks has been a popular and key component of recent deep Q-learning algorithms for reinforcement learning, yet little is known from the theory side. In this work, we introduce a new family of target-based temporal difference (TD) learning algorithms and provide theoretical analysis on their conver…

This paper presents a distributionally robust Q-Learning algorithm (DrQ) which leverages Wasserstein ambiguity sets to provide idealistic probabilistic out-of-sample safety guarantees during online learning. First, we follow past work by separating the constraint functions from the principal objective to create a hiera…

Study shows offline RL under $Q^\star$-approximation and partial coverage is harder than previously thought.

Deep reinforcement learning (DRL) has gained a lot of attention in recent years, and has been proven to be able to play Atari games and Go at or above human levels. However, those games are assumed to have a small fixed number of actions and could be trained with a simple CNN network. In this paper, we study a special …

In an episodic Markov Decision Process (MDP) problem, an online algorithm chooses from a set of actions in a sequence of $H$ trials, where $H$ is the episode length, in order to maximize the total payoff of the chosen actions. Q-learning, as the most popular model-free reinforcement learning (RL) algorithm, directly pa…

Optimizes mobile notifications for multiple objectives using reinforcement learning.

QAM uses adjoint matching to optimize continuous-action RL policies efficiently.

Double Q-learning has the same mean-squared error as Q-learning under certain conditions.

DE-QT detects optimal Q-learning stopping points.

New method learns credit prices offline without interaction.

This paper analyzes momentum Q-learning with finite-sample guarantees.

EBQL reduces bias in Q-learning for improved performance.

Model-free Reinforcement Learning (RL) works well when experience can be collected cheaply and model-based RL is effective when system dynamics can be modeled accurately. However, both assumptions can be violated in real world problems such as robotics, where querying the system can be expensive and real-world dynamics…

New self-imitation learning method improves performance in continuous control tasks.

Linear Q-learning converges to a bounded set without divergence.

Q-learning with neural network function approximation (neural Q-learning for short) is among the most prevalent deep reinforcement learning algorithms. Despite its empirical success, the non-asymptotic convergence rate of neural Q-learning remains virtually unknown. In this paper, we present a finite-time analysis of a…

Proposes a RL method using simulators for stabilizing uncertain systems.

Temporal-difference (TD) learning is an important field in reinforcement learning. Sarsa and Q-Learning are among the most used TD algorithms. The Q($σ$) algorithm (Sutton and Barto (2017)) unifies both. This paper extends the Q($σ$) algorithm to an online multi-step algorithm Q($σ, λ$) using eligibility traces and int…

Q-learning requires more samples than minimax bounds suggest for optimal Q-function approximation.

Q-learning for average cost MDPs gets a concentration bound.

New algorithm uses Whittle index to improve Q-learning for restless bandits.

One-step Bellman alignment improves online RL by reducing task mismatch.

LBQL improves Q-learning by using lookahead bounds for better performance.

This paper analyzes how periodic and soft target updates stabilize linear Q-learning.

FedLCB-Q learns optimal policies from federated offline data with linear speedup.

Kernelized Q-learning achieves good performance with minimal data.

Paper addresses underestimation bias in double Q-learning, proposing a method to improve learning performance.

Paper presents a deep reinforcement learning algorithm for online trading without offline training.

New algorithm reduces sample and communication complexities in federated Q-learning.

This paper improves Q-learning bounds using reference-advantage decomposition.

Study Whittle index learning algorithms for restless bandits with constant stepsizes.

Extended analysis of Q-learning's efficiency, matching optimal regret.

Proposes a new Q-learning method for survival outcomes in clinical trials.