arXiv research

Sharp bounds for approximating Sobolev functions by ridge functions and networks.

Paper develops efficient RL algorithm for general value function approximation.

We propose fast approximations for the generalized sliced-Wasserstein distance.

There has been a growing interest in expressivity of deep neural networks. However, most of the existing work about this topic focuses only on the specific activation function such as ReLU or sigmoid. In this paper, we investigate the approximation ability of deep neural networks with a broad class of activation functi…

Generalizes neural network approximation to infinite-dimensional manifolds and derivatives.

Improved neural network approximates analytic and L^p functions efficiently.

In this paper, we propose a generic framework for devising an adaptive approximation scheme for value function approximation in reinforcement learning, which introduces multiscale approximation. The two basic ingredients are multiresolution analysis as well as tree approximation. Starting from simple refinable function…

Analytic networks with bounded coefficients can't outperform polynomial approximations.

Efficiently plans large MDPs with weak function approximations.

Develops hierarchical reinforcement learning value function approximators.

Deep neural networks with piecewise-polynomial activations can approximate smooth functions and their derivatives.

We consider the problem of portfolio optimization in a simple incomplete market and under a general utility function. By working with the associated Hamilton-Jacobi-Bellman partial differential equation (HJB PDE), we obtain a closed-form formula for a trading strategy which approximates the optimal trading strategy whe…

Deep, wide ConvResNets can approximate functions and their smoothness.

Develops a new theory for approximating functions on massive data.

This work improves sample efficiency in neural function approximation for reinforcement learning.

This paper optimizes sampling for least-squares approximation.

We show that finite-width deep ReLU neural networks yield rate-distortion optimal approximation (Bölcskei et al., 2018) of polynomials, windowed sinusoidal functions, one-dimensional oscillatory textures, and the Weierstrass function, a fractal function which is continuous but nowhere differentiable. Together with thei…

Universal approximation theorem for differentiable maps on infinite-dimensional manifolds

Estimates generalization gap for overparameterized models using Langevin approximation.

Improved GSPGS estimators reduce bias in noisy function measurements.

Minimum width for ReLU networks to approximate L^p functions is max(d_x+1, d_y).

Three-hidden-layer neural networks can approximate Hölder continuous functions uniformly with exponential rate.

Let $U\subseteq\mathbb{R}^{n}$ be open and convex. We show that every (not necessarily Lipschitz or strongly) convex function $f:U\to\mathbb{R}$ can be approximated by real analytic convex functions, uniformly on all of $U$. In doing so we provide a technique which transfers results on uniform approximation on bounded …

TabPFN learns to approximate functions on tabular data.

Researchers develop neural networks for approximating functions in Banach spaces.

A new method uses deep learning to efficiently sample rare transitions for estimating committor functions.

Zap Q-learning is a recent class of reinforcement learning algorithms, motivated primarily as a means to accelerate convergence. Stability theory has been absent outside of two restrictive classes: the tabular setting, and optimal stopping. This paper introduces a new framework for analysis of a more general class of r…

We study the finite horizon Merton portfolio optimization problem in a general local-stochastic volatility setting. Using model coefficient expansion techniques, we derive approximations for the both the value function and the optimal investment strategy. We also analyze the `implied Sharpe ratio' and derive a series a…

Develops wavelet-based neural network approximation theory.

We propose a new method for learning deep neural network models that is based on a greedy learning approach: we add one basis function at a time, and a new basis function is generated as a non-linear activation function applied to a linear combination of the previous basis functions. Such a method (growing deep neural …

Minimum width for ReLU networks on compact domain is exactly max{d_x, d_y, 2}

We consider in this paper the optimal approximations of convex univariate functions with feed-forward Relu neural networks. We are interested in the following question: what is the minimal approximation error given the number of approximating linear pieces? We establish the necessary and sufficient conditions and uniqu…

Generative adversarial networks (GAN) approximate a target data distribution by jointly optimizing an objective function through a "two-player game" between a generator and a discriminator. Despite their empirical success, however, two very basic questions on how well they can approximate the target distribution remain…

Neural networks can approximate functionals on RKHS with error bounds.

New bound on neural nets complexity for approximating functions.

It has long been agreed by academics that the inversion method is the method of choice for generating random variates, given the availability of the quantile function. However for several probability distributions arising in practice a satisfactory method of approximating these functions is not available. The main focu…

$Q$-learning with function approximation is one of the most popular methods in reinforcement learning. Though the idea of using function approximation was proposed at least 60 years ago, even in the simplest setup, i.e, approximating $Q$-functions with linear functions, it is still an open problem on how to design a pr…

FQE with deep neural networks achieves asymptotic normality and finite-sample bounds.

We show that $C^0$-fine approximation of convex functions by smooth (or real analytic) convex functions on $\R^d$ is possible in general if and only if $d=1$. Nevertheless, for $d\geq 2$ we give a characterization of the class of convex functions on $\R^d$ which can be approximated by real analytic (or just smoother) c…

Universal approximation for stochastic processes using Brownian motion.

The optimal dividend problem by De Finetti (1957) has been recently generalized to the spectrally negative Lévy model where the implementation of optimal strategies draws upon the computation of scale functions and their derivatives. This paper proposes a phase-type fitting approximation of the optimal strategy. We con…

Complex-valued neural networks can approximate any continuous function.

The approximation power of general feedforward neural networks with piecewise linear activation functions is investigated. First, lower bounds on the size of a network are established in terms of the approximation error and network depth and width. These bounds improve upon state-of-the-art bounds for certain classes o…

Deep networks can approximate score functions in high-dimensional graphical models efficiently.

Let $U\subseteq\mathbb{R}^d$ be open and convex. We prove that every (not necessarily Lipschitz or strongly) convex function $f:U\to\mathbb{R}$ can be approximated by real analytic convex functions, uniformly on all of $U$. We also show that $C^0$-fine approximation of convex functions by smooth (or real analytic) conv…

New RL method explores environments without rewards, achieving efficient policy generation.

Simple neural networks approximate any continuous function with fixed neurons.

Deep networks can approximate functions with fewer learnable parameters than previously thought.