arXiv research

Study approximates nonlinear functionals using deep ReLU networks.

While there are convergence guarantees for temporal difference (TD) learning when using linear function approximators, the situation for nonlinear models is far less understood, and divergent examples are known. Here we take a first step towards extending theoretical convergence guarantees to TD learning with nonlinear…

Leveled Homomorphic Encryption (LHE) offers a potential solution that could allow sectors with sensitive data to utilize the cloud and securely deploy their models for remote inference with Deep Neural Networks (DNN). However, this application faces several obstacles due to the limitations of LHE. One of the main probl…

Method solves complex optimization problems with high probability bounds.

Study on statistical inference for nonlinear stochastic approximation with Markovian data.

Richberg technique adapted for nonlinear subequations.

A new model approximates complex functions in parameter space.

Kernel-based Bayesian filter for nonlinear systems using infinite-dimensional operators.

We introduce a data-based approach to estimating key quantities which arise in the study of nonlinear control systems and random nonlinear dynamical systems. Our approach hinges on the observation that much of the existing linear theory may be readily extended to nonlinear systems - with a reasonable expectation of suc…

Zap Q-learning is a recent class of reinforcement learning algorithms, motivated primarily as a means to accelerate convergence. Stability theory has been absent outside of two restrictive classes: the tabular setting, and optimal stopping. This paper introduces a new framework for analysis of a more general class of r…

A new deep neural network tackles nonlinear functional regression with improved dimensionality reduction.

Improved bounds for function approximation in nonlinear sets.

In this paper, we have extended the well-established universal approximator theory to neural networks that use the unbounded ReLU activation function and a nonlinear softmax output layer. We have proved that a sufficiently large neural network using the ReLU activation function can approximate any function in $L^1$ up …

We are interested in approximation of a multivariate function $f(x_1,\dots,x_d)$ by linear combinations of products $u^1(x_1)\cdots u^d(x_d)$ of univariate functions $u^i(x_i)$, $i=1,\dots,d$. In the case $d=2$ it is a classical problem of bilinear approximation. In the case of approximation in the $L_2$ space the bili…

New matrix approximation method using RBF components for better memory efficiency.

Study efficient convergence of RL algorithm with function approximation.

The purpose of this paper is to analyze and compute the early exercise boundary for a class of nonlinear Black--Scholes equations with a nonlinear volatility which can be a function of the second derivative of the option price itself. A motivation for studying the nonlinear Black--Scholes equation with a nonlinear vola…

New method for separating mixed signals with nonlinear functions.

Kernel discriminant analysis uses nonlinear embeddings to improve classification.

Function approximation from input and output data pairs constitutes a fundamental problem in supervised learning. Deep neural networks are currently the most popular method for learning to mimic the input-output relationship of a general nonlinear system, as they have proven to be very effective in approximating comple…

We study the expressivity of deep neural networks. Measuring a network's complexity by its number of connections or by its number of neurons, we consider the class of functions for which the error of best approximation with networks of a given complexity decays at a certain rate when increasing the complexity budget. U…

In this paper we propose a new robust algorithm to find the optimal static replicating portfolios for general nonlinear payoff functions and give the estimate of the rate of convergence that is absent in the literature. We choose the static replication by minimizing the error bound between the nonlinear payoff function…

Method solves Bayesian inverse problems in function space without assuming log-concavity.

This paper introduces a new method for semi-supervised learning on high dimensional nonlinear manifolds, which includes a phase of unsupervised basis learning and a phase of supervised function learning. The learned bases provide a set of anchor points to form a local coordinate system, such that each data point $x$ on…

EnSF improves accuracy in tracking high-dimensional nonlinear systems.

Given a function dictionary $\cal D$ and an approximation budget $N\in\mathbb{N}^+$, nonlinear approximation seeks the linear combination of the best $N$ terms $\{T_n\}_{1\le n\le N}\subseteq{\cal D}$ to approximate a given function $f$ with the minimum approximation error\[\varepsilon_{L,f}:=\min_{\{g_n\}\subseteq{\ma…

Enhances RSCNs with hybrid regularization for nonlinear dynamics.

New methods for scalable causal discovery from complex data.

While it is widely known that neural networks are universal approximators of continuous functions, a less known and perhaps more powerful result is that a neural network with a single hidden layer can approximate accurately any nonlinear continuous operator. This universal approximation theorem is suggestive of the pot…

Temporal Difference Learning analysis under non-i.i.d. data and nonlinear approximation.

Papers learn from data to make decisions without interacting, improving on previous methods.

Deep neural networks and the ENO procedure are both efficient frameworks for approximating rough functions. We prove that at any order, the ENO interpolation procedure can be cast as a deep ReLU neural network. This surprising fact enables the transfer of several desirable properties of the ENO procedure to deep neural…

We simplify complex regression coefficients using linearization and feature comparison.

A deep learning method solves nonlinear filtering problems efficiently.

Develops a method for causal inference with noisy confounders.

Reduces function approximation dimensions from high to low with sparse data.

Neural networks with random hidden nodes have gained increasing interest from researchers and practical applications. This is due to their unique features such as very fast training and universal approximation property. In these networks the weights and biases of hidden nodes determining the nonlinear feature mapping a…

Rational neural networks approximate functions more efficiently with less depth.

Paper quantifies neural operators' efficiency for solving nonlinear parabolic PDEs.

RVFL NNs perform well without direct links and output bias for regression.

Deep residual networks can approximate any continuous function using control theory.

In reinforcement learning, temporal difference (TD) is the most direct algorithm to learn the value function of a policy. For large or infinite state spaces, exact representations of the value function are usually not available, and it must be approximated by a function in some parametric family. However, with \emph{no…

New machine learning methods solve complex PDEs with improved accuracy.

Improved state estimation in nonlinear models using amortized backward variational inference.

DeepRSCN models nonlinear systems using stochastic configurations.

We discuss the approximation of the value function for infinite-horizon discounted Markov Reward Processes (MRP) with nonlinear functions trained with the Temporal-Difference (TD) learning algorithm. We first consider this problem under a certain scaling of the approximating function, leading to a regime called lazy tr…

We study sparse approximate solutions to convex optimization problems. It is known that in many engineering applications researchers are interested in an approximate solution of an optimization problem as a linear combination of elements from a given system of elements. There is an increasing interest in building such …

The purpose of this survey chapter is to present a transformation technique that can be used in analysis and numerical computation of the early exercise boundary for an American style of vanilla options that can be modelled by class of generalized Black-Scholes equations. We analyze qualitatively and quantitatively the…