arXiv research

The paper improves energy decay estimates for Dir-stationary Q-valued functions and applies them to Liouville-type theorems and continuity.

Estimating the value function for a fixed policy is a fundamental problem in reinforcement learning. Policy evaluation algorithms---to estimate value functions---continue to be developed, to improve convergence rates, improve stability and handle variability, particularly for off-policy learning. To understand the prop…

New method estimates minimizer and minimum value of a regression function.

We present a fully nonparametric method to estimate the value function, via simulation, in the context of expected infinite-horizon discounted rewards for Markov chains. Estimating such value functions plays an important role in approximate dynamic programming and applied probability in general. We incorporate "soft in…

UA-LQE improves value function learning by selectively erasing uncertain entries in Q-matrix.

A new estimator for evaluating policies in unknown environments.

Efficient estimators for smooth Hilbert-valued parameters with theoretical guarantees.

Many biological characteristics of evolutionary interest are not scalar variables but continuous functions. Here we use phylogenetic Gaussian process regression to model the evolution of simulated function-valued traits. Given function-valued data only from the tips of an evolutionary tree and utilising independent pri…

Boosts Q-learning by using value function bounds.

Kernel smoothing improves LLM reasoning efficiency.

Estimates personalized treatment response curves using covariates.

A new method for estimating joint value functions in multi-scene reinforcement learning.

Despite recent successes in Reinforcement Learning, value-based methods often suffer from high variance hindering performance. In this paper, we illustrate this in a continuous control setting where state of the art methods perform poorly whenever sensor noise is introduced. To overcome this issue, we introduce Recurre…

The paper establishes inequalities and gradient estimates for harmonic functions on Finsler measure spaces.

Many problems in financial engineering involve the estimation of unknown conditional expectations across a time interval. Often Least Squares Monte Carlo techniques are used for the estimation. One method that can be combined with Least Squares Monte Carlo is the "Regress-Later" method. Unlike conventional methods wher…

Policy evaluation is a key process in reinforcement learning. It assesses a given policy using estimation of the corresponding value function. When using a parameterized function to approximate the value, it is common to optimize the set of parameters by minimizing the sum of squared Bellman Temporal Differences errors…

Temporal difference methods enable efficient estimation of value functions in reinforcement learning in an incremental fashion, and are of broader interest because they correspond learning as observed in biological systems. Standard value functions correspond to the expected value of a sum of discounted returns. While …

New method uses extreme value theory to estimate neural network errors.

This paper provides a block coordinate descent algorithm to solve unconstrained optimization problems. In our algorithm, computation of function values or gradients is not required. Instead, pairwise comparison of function values is used. Our algorithm consists of two steps; one is the direction estimate step and the o…

Value function estimation is an important task in reinforcement learning, i.e., prediction. The Boltzmann softmax operator is a natural value estimator and can provide several benefits. However, it does not satisfy the non-expansion property, and its direct use may fail to converge even in value iteration. In this pape…

In traditional reinforcement learning, an agent maximizes the reward collected during its interaction with the environment by approximating the optimal policy through the estimation of value functions. Typically, given a state s and action a, the corresponding value is the expected discounted sum of rewards. The optima…

Improves reinforcement learning agent's scene-specific value function.

The paper develops methods to estimate value functions in reinforcement learning for infinite horizon problems.

This paper studies a recent proposal to use randomized value functions to drive exploration in reinforcement learning. These randomized value functions are generated by injecting random noise into the training data, making the approach compatible with many popular methods for estimating parameterized value functions. B…

Paper presents an efficient exploration method for reinforcement learning.

TD learning reduces prediction error in Markov chain problems.

Study functional confounders in causal inference, enabling estimable effects.

New method for off-policy evaluation in POMDPs using future-dependent value functions.

New method estimates optimal Q-values with better accuracy for specific problems.

Randomized value functions offer a promising approach towards the challenge of efficient exploration in complex environments with high dimensional state and action spaces. Unlike traditional point estimate methods, randomized value functions maintain a posterior distribution over action-space values. This prevents the …

Bayesian approach improves Shapley value estimation efficiency.

A new method estimates SDEs using occupation kernels.

BASIS improves LLM reasoning by sharing batchwise rollout info, reducing MSE by 69%.

Dyna is an architecture for model-based reinforcement learning (RL), where simulated experience from a model is used to update policies or value functions. A key component of Dyna is search-control, the mechanism to generate the state and action from which the agent queries the model, which remains largely unexplored. …

New algorithms estimate function levels with near-optimal efficiency.

Improved TD learning reduces batch sampling error.

In this paper, we introduce an actor-critic algorithm called Deep Value Model Predictive Control (DMPC), which combines model-based trajectory optimization with value function estimation. The DMPC actor is a Model Predictive Control (MPC) optimizer with an objective function defined in terms of a value function estimat…

Develops a new method for statistical optimal allocation problems.

Study non-parametric value function estimation from a single path.

This work provides guarantees for off-policy function estimation under realizability assumptions.

Proposes EVE for efficient exploration in reinforcement learning.

New RL algorithm achieves nearly optimal performance for linear MDPs.

This paper addresses the problem of estimating, in the presence of random censoring as well as competing risks, the extreme value index of the (sub)-distribution function associated to one particular cause, in the heavy-tail case. Asymptotic normality of the proposed estimator (which has the form of an Aalen-Johansen i…

We propose a flexible method for estimating value functions in reinforcement learning without parametric assumptions.

This paper introduces metrics for welfare analysis in dynamic models. We develop estimation and inference for these parameters even in the presence of a high-dimensional state space. Examples of welfare metrics include average welfare, average marginal welfare effects, and welfare decompositions into direct and indirec…

New method estimates optimal personalized treatment rules from mixed data sources.

We consider the problem of learning the optimal action-value function in the discounted-reward Markov decision processes (MDPs). We prove a new PAC bound on the sample-complexity of model-based value iteration algorithm in the presence of the generative model, which indicates that for an MDP with N state-action pairs a…

Optimizes information acquisition to reduce estimation risk and maximize utility.