arXiv research

We approximate derivatives of functions on manifolds by embedding them and applying vector-valued operators.

Paper develops efficient RL algorithm for general value function approximation.

Develops hierarchical reinforcement learning value function approximators.

Researchers develop neural networks for approximating functions in Banach spaces.

In this work, we study value function approximation in reinforcement learning (RL) problems with high dimensional state or action spaces via a generalized version of representation policy iteration (RPI). We consider the limitations of proto-value functions (PVFs) at accurately approximating the value function in low d…

In this paper, we propose a generic framework for devising an adaptive approximation scheme for value function approximation in reinforcement learning, which introduces multiscale approximation. The two basic ingredients are multiresolution analysis as well as tree approximation. Starting from simple refinable function…

CVNNs improve performance in tasks with complex-valued inputs.

Efficiently plans large MDPs with weak function approximations.

New algorithms accelerate value function convergence in MDPs.

Recently, Petrik et al. demonstrated that L1Regularized Approximate Linear Programming (RALP) could produce value functions and policies which compared favorably to established linear value function approximation techniques like LSPI. RALP's success primarily stems from the ability to solve the feature selection and va…

Complex-valued neural networks can approximate any continuous function.

A core operation in reinforcement learning (RL) is finding an action that is optimal with respect to a learned value function. This operation is often challenging when the learned value function takes continuous actions as input. We introduce deep radial-basis value functions (RBVFs): value functions learned using a de…

Method uses DNNs to approximate functions with specific asymptotic behavior.

We consider the problem of portfolio optimization in a simple incomplete market and under a general utility function. By working with the associated Hamilton-Jacobi-Bellman partial differential equation (HJB PDE), we obtain a closed-form formula for a trading strategy which approximates the optimal trading strategy whe…

In reinforcement learning, temporal difference (TD) is the most direct algorithm to learn the value function of a policy. For large or infinite state spaces, exact representations of the value function are usually not available, and it must be approximated by a function in some parametric family. However, with \emph{no…

New algorithm learns value and advantage functions for continuous-time Markov processes without structural assumptions.

A large proportion of market making models derive from the seminal model of Avellaneda and Stoikov. The numerical approximation of the value function and the optimal quotes in these models remains a challenge when the number of assets is large. In this article, we propose closed-form approximations for the value functi…

We consider a singular control problem with regime switching that arises in problems of optimal investment decisions of cash-constrained firms. The value function is proved to be the unique viscosity solution of the associated Hamilton-Jacobi-Bellman equation. Moreover, we give regularity properties of the value functi…

We propose a plan online and learn offline (POLO) framework for the setting where an agent, with an internal model, needs to continually act and learn in the world. Our work builds on the synergistic relationship between local model-based control, global value function learning, and exploration. We study how local traj…

Papers learn from data to make decisions without interacting, improving on previous methods.

Complex-valued neural networks can approximate any continuous function with bounded widths and depths.

In this paper, we consider the stochastic iterative counterpart of the value iteration scheme wherein only noisy and possibly biased approximations of the Bellman operator are available. We call this counterpart as the approximate value iteration (AVI) scheme. Neural networks are often used as function approximators, i…

We study the finite horizon Merton portfolio optimization problem in a general local-stochastic volatility setting. Using model coefficient expansion techniques, we derive approximations for the both the value function and the optimal investment strategy. We also analyze the `implied Sharpe ratio' and derive a series a…

Deep, wide ConvResNets can approximate functions and their smoothness.

In this paper we study a utility maximization problem with both optimal control and optimal stopping in a finite time horizon. The value function can be characterized by a variational equation that involves a free boundary problem of a fully nonlinear partial differential equation. Using the dual control method, we der…

Paper analyzes distributional reinforcement learning with value function approximation, introducing Bellman unbiasedness and a new algorithm.

Pessimistic Minimax Value Iteration finds efficient NE policies from offline data.

Temporal difference learning and Residual Gradient methods are the most widely used temporal difference based learning algorithms; however, it has been shown that none of their objective functions is optimal w.r.t approximating the true value function $V$. Two novel algorithms are proposed to approximate the true value…

Extends Tanimoto kernel to real-valued functions.

Deep networks with path norm regularization can approximate analytic functions.

Offline RL struggles with sample efficiency due to fundamental barriers.

Improved multilevel scheme for value-at-risk computation.

Study approximates unknown function levels with queries.

Policy gradient methods with aggregated states can achieve better performance than approximate policy iteration.

Universal approximation for ODENet and ResNet with a single activation function.

Bayesian approach improves Shapley value estimation efficiency.

This paper studies optimal approximation factors in misspecified off-policy RL, identifying key factors under various settings.

Randomized value functions offer a promising approach towards the challenge of efficient exploration in complex environments with high dimensional state and action spaces. Unlike traditional point estimate methods, randomized value functions maintain a posterior distribution over action-space values. This prevents the …

Matrix rank minimization problem is in general NP-hard. The nuclear norm is used to substitute the rank function in many recent studies. Nevertheless, the nuclear norm approximation adds all singular values together and the approximation error may depend heavily on the magnitudes of singular values. This might restrict…

This paper proposes a new approach to RL by focusing on the value-improvement path.

Polynomial-time RL algorithm for constant actions under linear Bellman completeness.

A new method reduces the computational cost of KernelSHAP for explaining predictions.

Sobolev training helps neural nets fit function values and derivatives.

We propose a time value related decision function to treat a classical option pricing problem raised by Hutchinson-Lo-Poggio. In numerical experiments, the new decision function significantly improves the original model of Hutchinson-Lo-Poggio with faster convergence and better generalization performance. By proving a …

New RL algorithm tackles large state spaces using optimistic function approximation.

VA-OPE improves OPE by incorporating variance information, achieving tighter error bounds.

This paper compares linear regression and neural networks for pricing swing options.

Policy evaluation is a key process in reinforcement learning. It assesses a given policy using estimation of the corresponding value function. When using a parameterized function to approximate the value, it is common to optimize the set of parameters by minimizing the sum of squared Bellman Temporal Differences errors…