arXiv research

The paper improves energy decay estimates for Dir-stationary Q-valued functions and applies them to Liouville-type theorems and continuity.

We present a fully nonparametric method to estimate the value function, via simulation, in the context of expected infinite-horizon discounted rewards for Markov chains. Estimating such value functions plays an important role in approximate dynamic programming and applied probability in general. We incorporate "soft in…

New method improves value function estimation in noisy environments.

Estimating the value function for a fixed policy is a fundamental problem in reinforcement learning. Policy evaluation algorithms---to estimate value functions---continue to be developed, to improve convergence rates, improve stability and handle variability, particularly for off-policy learning. To understand the prop…

A new estimator for evaluating policies in unknown environments.

Boosts Q-learning by using value function bounds.

Study uses randomized value functions to enhance exploration in reinforcement learning.

DMPC combines MPC and value function estimation for efficient control tasks.

A new method for estimating joint value functions in multi-scene reinforcement learning.

Paper presents an efficient exploration method for reinforcement learning.

New method for off-policy evaluation in POMDPs using future-dependent value functions.

Kernel smoothing improves LLM reasoning efficiency.

Randomized value functions offer a promising approach towards the challenge of efficient exploration in complex environments with high dimensional state and action spaces. Unlike traditional point estimate methods, randomized value functions maintain a posterior distribution over action-space values. This prevents the …

Estimates personalized treatment response curves using covariates.

Improved TD learning reduces batch sampling error.

Develops hierarchical reinforcement learning value function approximators.

We propose a flexible method for estimating value functions in reinforcement learning without parametric assumptions.

The paper proposes a method to decompose value functions in RL for better understanding and prediction.

The paper develops methods for welfare analysis in dynamic models.

Tackling large approximate dynamic programming or reinforcement learning problems requires methods that can exploit regularities, or intrinsic structure, of the problem in hand. Most current methods are geared towards exploiting the regularities of either the value function or the policy. We introduce a general classif…

Study on future-dependent value functions for off-policy evaluation in complex environments.

Value function estimation is an important task in reinforcement learning, i.e., prediction. The Boltzmann softmax operator is a natural value estimator and can provide several benefits. However, it does not satisfy the non-expansion property, and its direct use may fail to converge even in value iteration. In this pape…

Develops a new method for statistical optimal allocation problems.

Study non-parametric value function estimation from a single path.

A novel approach uses an ensemble of Gaussian processes for robust and adaptive reinforcement learning.

Paper reduces variance in infinite horizon off-policy evaluation with bias reduction.

We propose a plan online and learn offline (POLO) framework for the setting where an agent, with an internal model, needs to continually act and learn in the world. Our work builds on the synergistic relationship between local model-based control, global value function learning, and exploration. We study how local traj…

Paper tackles robust control of SDEs with ambiguity, proving value function existence and applying to investment problems.

We consider the problem of learning the optimal action-value function in the discounted-reward Markov decision processes (MDPs). We prove a new PAC bound on the sample-complexity of model-based value iteration algorithm in the presence of the generative model, which indicates that for an MDP with N state-action pairs a…

UVU simplifies value uncertainty quantification in RL.

Improves reinforcement learning agent's scene-specific value function.

UA-LQE improves value function learning by selectively erasing uncertain entries in Q-matrix.

TD learning reduces prediction error in Markov chain problems.

Deep Galerkin Method estimates value function for mean-field control problem.

In traditional reinforcement learning, an agent maximizes the reward collected during its interaction with the environment by approximating the optimal policy through the estimation of value functions. Typically, given a state s and action a, the corresponding value is the expected discounted sum of rewards. The optima…

The goal of reinforcement learning algorithms is to estimate and/or optimise the value function. However, unlike supervised learning, no teacher or oracle is available to provide the true value function. Instead, the majority of reinforcement learning algorithms estimate and/or optimise a proxy for the value function. …

We examine the impact of learning Lipschitz continuous models in the context of model-based reinforcement learning. We provide a novel bound on multi-step prediction error of Lipschitz models where we quantify the error using the Wasserstein metric. We go on to prove an error bound for the value-function estimate arisi…

New RL algorithm achieves nearly optimal performance for linear MDPs.

ZOSPI improves RL policies with global value function exploitation.

Bayesian approach improves Shapley value estimation efficiency.

New method estimates optimal personalized treatment rules from mixed data sources.

The paper develops methods to estimate value functions in reinforcement learning for infinite horizon problems.

A new RL algorithm POWR learns world models to estimate action-values.

Paper introduces a new value function for state transitions and optimal policy learning.

We present a framework to derive risk bounds for vector-valued learning with a broad class of feature maps and loss functions. Multi-task learning and one-vs-all multi-category learning are treated as examples. We discuss in detail vector-valued functions with one hidden layer, and demonstrate that the conditions under…

The paper introduces a novel method for stable off-policy learning using value function chaining.

This work provides guarantees for off-policy function estimation under realizability assumptions.

A method to reduce bias in model-based policy evaluation by shifting operators.