arXiv research

Boosts Q-learning by using value function bounds.

The paper establishes a Poisson integral formula for bounded pluriharmonic functions on Teichmüller space.

This paper studies a recent proposal to use randomized value functions to drive exploration in reinforcement learning. These randomized value functions are generated by injecting random noise into the training data, making the approach compatible with many popular methods for estimating parameterized value functions. B…

Paper develops efficient RL algorithm for general value function approximation.

The paper develops bounds for predictive values in binary classification.

We present a framework to derive risk bounds for vector-valued learning with a broad class of feature maps and loss functions. Multi-task learning and one-vs-all multi-category learning are treated as examples. We discuss in detail vector-valued functions with one hidden layer, and demonstrate that the conditions under…

New hyperbolic 4-manifolds found with special functions.

We consider the problem of learning the optimal action-value function in the discounted-reward Markov decision processes (MDPs). We prove a new PAC bound on the sample-complexity of model-based value iteration algorithm in the presence of the generative model, which indicates that for an MDP with N state-action pairs a…

Algorithm optimizes cascaded functions with known structure.

Study introduces indecomposability for varifolds, leading to geometric consequences.

Complex-valued neural networks can approximate any continuous function with bounded widths and depths.

We study the use of randomized value functions to guide deep exploration in reinforcement learning. This offers an elegant means for synthesizing statistically and computationally efficient exploration with common practical approaches to value function learning. We present several reinforcement learning algorithms that…

Paper improves worst-case regret bounds for RLSVI in reinforcement learning.

We approximate derivatives of functions on manifolds by embedding them and applying vector-valued operators.

Estimating the value function for a fixed policy is a fundamental problem in reinforcement learning. Policy evaluation algorithms---to estimate value functions---continue to be developed, to improve convergence rates, improve stability and handle variability, particularly for off-policy learning. To understand the prop…

We give a unified statement and proof of a class of wellknown mean value inequalities for nonnegative functions with a nonlinear bound on the Laplacian. We generalize these to domains with boundary, requiring a (possibly nonlinear) bound on the normal derivative at the boundary. These inequalities give rise to an energ…

TensorPlan algorithm finds δ-optimal policies with poly$(H,d)$ queries under linearly realizable state-value function.

This paper improves reinforcement learning policies in a scalable way.

New algorithm explores reinforcement learning with noisy data.

Paper tackles transfer RL under unobserved context, developing methods to reduce bias.

Let (M,g) be a non-compact and complete Riemannian manifold with minimal horospheres and infinite injectivity radius. We prove that bounded functions on (M,g) satisfying the mean-value property are constant. We extend thus a result of A. Ranjan and H. Shah who proved a similar result for bounded harmonic functions on h…

Optimal rates for vector-valued regression on various norms.

NP-PROV separates mean and variance spaces to improve function uncertainty.

Develops robust MDPs for unknown disturbances with performance guarantees.

We derive bounds on the distribution function, therefore also on the Value-at-Risk, of $\varphi(\mathbf X)$ where $\varphi$ is an aggregation function and $\mathbf X = (X_1,\dots,X_d)$ is a random vector with known marginal distributions and partially known dependence structure. More specifically, we analyze three type…

We use noncommutative localization to construct a chain complex which counts the critical points of a circle-valued Morse function on a manifold, generalizing the Novikov complex. As a consequence we obtain new topological lower bounds on the minimum number of critical points of a circle-valued Morse function within a …

The aim of this paper is to study the fast computation of the lower and upper bounds on the value function for utility maximization under the Heston stochastic volatility model with general utility functions. It is well known there is a closed form solution of the HJB equation for power utility due to its homothetic pr…

Estimates fat-shattering dimension of aggregated function classes.

The paper improves energy decay estimates for Dir-stationary Q-valued functions and applies them to Liouville-type theorems and continuity.

We study the two-times differentiability of the value functions of the primal and dual optimization problems that appear in the setting of expected utility maximization in incomplete markets. We also study the differentiability of the solutions to these problems with respect to their initial values. We show that the ke…

New RL method handles large state-action spaces with complex models.

New algorithms estimate function levels with near-optimal efficiency.

Proposes a method to handle missing inputs in Bayesian optimization.

Study optimality conditions for interval-valued optimization problems on Riemannian manifolds.

New method estimates minimizer and minimum value of a regression function.

Study tight offline learning bounds for linear MDPs using variance information.

Study confirms learning rates for vector-valued spectral algorithms, proving consistency.

Let $R$ be a compact surface and let $Γ$ be a Jordan curve which separates $R$ into two connected components $Σ_1$ and $Σ_2$. A harmonic function $h_1$ on $Σ_1$ of bounded Dirichlet norm has boundary values $H$ in a certain conformally invariant non-tangential sense on $Γ$. We show that if $Γ$ is a quasicircle, then th…

Pessimistic Minimax Value Iteration finds efficient NE policies from offline data.

The paper examines smoothness of value function in consumption-investment models with borrowing constraints.

New research shows exponential lower bounds for planning in MDPs with linearly-realizable optimal action-value functions.

Strong worst-case performance bounds for episodic reinforcement learning exist but fortunately in practice RL algorithms perform much better than such bounds would predict. Algorithms and theory that provide strong problem-dependent bounds could help illuminate the key features of what makes a RL problem hard and reduc…

New method estimates optimal Q-values with better accuracy for specific problems.

We propose randomized least-squares value iteration (RLSVI) -- a new reinforcement learning algorithm designed to explore and generalize efficiently via linearly parameterized value functions. We explain why versions of least-squares value iteration that use Boltzmann or epsilon-greedy exploration can be highly ineffic…

We examine the impact of learning Lipschitz continuous models in the context of model-based reinforcement learning. We provide a novel bound on multi-step prediction error of Lipschitz models where we quantify the error using the Wasserstein metric. We go on to prove an error bound for the value-function estimate arisi…

New method certifies neural network function space norms from point evaluations.

The paper introduces a method to learn and apply value envelopes for faster online reinforcement learning.

Optimizes target value in stochastic black box functions.