arXiv research

New algorithms estimate Q-functions under partial coverage and realizability, improving offline RL guarantees.

New approach transfers rewards learned in one environment to reinforcement learning in a new environment.

CriticSMC improves planning efficiency in constrained environments.

Boosts Q-learning by using value function bounds.

Algorithm learns Nash equilibria in stochastic games using entropy-regularized policies.

Enhances RL by controlling policy stochasticity through trajectory entropy constraints.

A new algorithm reduces sample complexity for learning Q-functions in reinforcement learning.

SAVO actor improves reinforcement learning by avoiding local optima in complex Q-functions.

The paper introduces a new method for estimating optimal policies in dynamic treatment regimes using information geometry.

Study on $Q$-function estimation for continuous state-action MDPs, deriving rates and conditions.

This paper develops q-learning methods for mean-field control problems.

Q-learning is one of the most popular methods in Reinforcement Learning (RL). Transfer Learning aims to utilize the learned knowledge from source tasks to help new tasks to improve the sample complexity of the new tasks. Considering that data collection in RL is both more time and cost consuming and Q-learning converge…

Improves DRL for long-term causal inference with semiparametric methods.

Deep Reinforcement Learning (DRL) has achieved impressive success in many applications. A key component of many DRL models is a neural network representing a Q function, to estimate the expected cumulative reward following a state-action pair. The Q function neural network contains a lot of implicit knowledge about the…

Continuous-time Q-learning theory developed for reinforcement learning.

Value-based methods constitute a fundamental methodology in planning and deep reinforcement learning (RL). In this paper, we propose to exploit the underlying structures of the state-action value function, i.e., Q function, for both planning and deep RL. In particular, if the underlying system dynamics lead to some glo…

This paper explores how IV methods can improve Q-function estimates in offline policy evaluation.

The objective is to study an on-line Hidden Markov model (HMM) estimation-based Q-learning algorithm for partially observable Markov decision process (POMDP) on finite state and action sets. When the full state observation is available, Q-learning finds the optimal action-value function given the current action (Q func…

Proposes a new reinforcement learning algorithm using Q-function.

We present a new Q-function operator for temporal difference (TD) learning methods that explicitly encodes robustness against significant rare events (SRE) in critical domains. The operator, which we call the $κ$-operator, allows to learn a robust policy in a model-based fashion without actually observing the SRE. We i…

The paper proves the convergence of Q-value for Gaussian rewards.

New RL method learns K-step lookahead Q-functions for fixed-horizon MDPs.

We compare the model-free reinforcement learning with the model-based approaches through the lens of the expressive power of neural networks for policies, $Q$-functions, and dynamics. We show, theoretically and empirically, that even for one-dimensional continuous state space, there are many MDPs whose optimal $Q$-func…

Proposes a RL method using simulators for stabilizing uncertain systems.

New method estimates and optimizes policy differences using orthogonal learning.

In this paper, two Q-learning (QL) methods are proposed and their convergence theories are established for addressing the model-free optimal control problem of general nonlinear continuous-time systems. By introducing the Q-function for continuous-time systems, policy iteration based QL (PIQL) and value iteration based…

Employers actively look for talents having not only specific hard skills but also various soft skills. To analyze the soft skill demands on the job market, it is important to be able to detect soft skill phrases from job advertisements automatically. However, a naive matching of soft skill phrases can lead to false pos…

DQNs can approximate optimal Q-functions with high accuracy on compact sets.

The impact of softmax on the value function itself in reinforcement learning (RL) is often viewed as problematic because it leads to sub-optimal value (or Q) functions and interferes with the contraction properties of the Bellman operator. Surprisingly, despite these concerns, and independent of its effect on explorati…

MACC learns communication protocols by adapting counterfactual reasoning.

Soft labeling impacts OOD detection in neural networks.

Developed a new thresholding method that connects soft and hard thresholding.

Deep Q-Learning models optimal exercise strategies for option-type products.

Soft cells fill space without gaps, derived from minimal surfaces and deformed using edge bending.

ASBART accelerates Soft BART for faster Bayesian regression.

Maximum entropy deep reinforcement learning (RL) methods have been demonstrated on a range of challenging continuous tasks. However, existing methods either suffer from severe instability when training on large off-policy data or cannot scale to tasks with very high state and action dimensionality such as 3D humanoid l…

New method improves stability of soft FQI for offline RL.

Policy evaluation or value function or Q-function approximation is a key procedure in reinforcement learning (RL). It is a necessary component of policy iteration and can be used for variance reduction in policy gradient methods. Therefore its quality has a significant impact on most RL algorithms. Motivated by manifol…

ZDPG learns model-free policies without critics, improving on PG.

A new method recovers rewards from behavior policies using classification and regression.

Generalizes soft noncommutative schemes to flag varieties.

Paper proposes a new loss function for conditional models using soft targets.

Introduces Soft-SVM for binary classification bridging logistic and SVM.

Deep model learns from labeled and unlabeled data for industrial soft sensing.

In the past few years, off-policy reinforcement learning methods have shown promising results in their application for robot control. Deep Q-learning, however, still suffers from poor data-efficiency and is susceptible to stochasticity in the environment or reward functions which is limiting with regard to real-world a…

This paper introduces TNTK to study infinite soft tree ensembles.

SF-DQN improves RL transfer by learning successor features.

Improves deep neural networks using soft labels through alternating minimization.