arXiv research

New method extends fitted Q-evaluation for distributional off-policy reinforcement learning.

Iran's FiT policy boosts REs but risks financial crisis and plant failure.

Paper proposes FPG algorithm for unbiased off-policy PG estimation.

KBB algorithm reduces sample complexity for policy evaluation in general state spaces.

New method stabilizes FQE by reweighting Bellman targets.

SARSA is an on-policy algorithm to learn a Markov decision process policy in reinforcement learning. We investigate the SARSA algorithm with linear function approximation under the non-i.i.d.\ data, where a single sample trajectory is available. With a Lipschitz continuous policy improvement operator that is smooth eno…

We propose a method for efficient training of Q-functions for continuous-state Markov Decision Processes (MDPs) such that the traces of the resulting policies satisfy a given Linear Temporal Logic (LTL) property. LTL, a modal logic, can express a wide range of time-dependent logical properties (including "safety") that…

Paper improves bootstrapping for off-policy reinforcement learning inference.

VA-OPE improves OPE by incorporating variance information, achieving tighter error bounds.

Sparse feature selection improves batch RL efficiency.

A new Thompson Sampling framework handles uncertainty by imputing missing data.

New method improves stability of soft FQI for offline RL.

The study analyzes how neural reward models learn features for policy optimization in a Gaussian single-index model.

Proposes a method to learn policies from offline data with reduced bias.

New method for robust policy evaluation in offline reinforcement learning with sequentially exogenous unobserved confounders.

Deep neural networks can estimate Q-values efficiently on low-dimensional state-action spaces.

FQE with deep neural networks achieves asymptotic normality and finite-sample bounds.

FORE evaluates occupancy ratios without requiring Bellman completeness.

Policy gradient based reinforcement learning algorithms coupled with neural networks have shown success in learning complex policies in the model free continuous action space control setting. However, explicitly parameterized policies are limited by the scope of the chosen parametric probability distribution. We show t…

This work characterizes conditions for offline policy evaluation in reinforcement learning.

AHS framework selects hyperparameters for FQE with error guarantees.

Batch Reinforcement Learning (RL) algorithms attempt to choose a policy from a designer-provided class of policies given a fixed set of training data. Choosing the policy which maximizes an estimate of return often leads to over-fitting when only limited data is available, due to the size of the policy class in relatio…

Study analyzes FIT schemes under market and regulatory uncertainty.

DDO-RM improves reward-based policies by converting reward scores into a target distribution.

Develops CLTs for Markov chain transition probabilities and policies.

Paper uses DRL to improve volatility fitting in equity derivatives.

Policy gradient is an efficient technique for improving a policy in a reinforcement learning setting. However, vanilla online variants are on-policy only and not able to take advantage of off-policy data. In this paper we describe a new technique that combines policy gradient with off-policy Q-learning, drawing experie…

Typically, operational risk losses are reported above a threshold. Fitting data reported above a constant threshold is a well known and studied problem. However, in practice, the losses are scaled for business and other factors before the fitting and thus the threshold is varying across the scaled data sample. A report…

Framework predicts household incomes using common indicators across regions.

A new method removes policy optimization in adversarial imitation learning.

In the context of learning deterministic policies in continuous domains, we revisit an approach, which was first proposed in Continuous Actor Critic Learning Automaton (CACLA) and later extended in Neural Fitted Actor Critic (NFAC). This approach is based on a policy update different from that of deterministic policy g…

Develops a new model to predict training dynamics of large language models.

Novel algorithm reduces computational burden in IRL with finite-time guarantees.

Adaptive policies for multi-agent RL improve adaptiveness in changing environments.

New method evaluates policies with latent confounders using optimal balance.

Counterfactual policy evaluation improves autonomous driving policies' generalization.

Develops a new trading strategy for statistical arbitrage with path-dependent signals.

This paper optimizes off-policy evaluation in reinforcement learning with function approximation.

We consider the problem of learning a policy for a Markov decision process consistent with data captured on the state-actions pairs followed by the policy. We assume that the policy belongs to a class of parameterized policies which are defined using features associated with the state-action pairs. The features are kno…

Paper proposes a new DR estimator for adaptive experiments with improved performance.

We present a new method of blackbox optimization via gradient approximation with the use of structured random orthogonal matrices, providing more accurate estimators than baselines and with provable theoretical guarantees. We show that this algorithm can be successfully applied to learn better quality compact policies …

We study how the behavior of deep policy gradient algorithms reflects the conceptual framework motivating their development. To this end, we propose a fine-grained analysis of state-of-the-art methods based on key elements of this framework: gradient estimation, value prediction, and optimization landscapes. Our result…

This work uses statistical bootstrapping to provide accurate confidence intervals for policy value in reinforcement learning.

DOLCE improves off-policy evaluation and learning by decomposing effects.

We present an off-policy actor-critic algorithm for Reinforcement Learning (RL) that combines ideas from gradient-free optimization via stochastic search with learned action-value function. The result is a simple procedure consisting of three steps: i) policy evaluation by estimating a parametric action-value function;…

The paper develops models for asset returns based on market conditions and uses them to construct a trading policy.

A new reinforcement learning method for medical decisions with limited data.

Risk-controlled post-processing optimizes decision policies under risk constraints.