The paper analyzes deep ReLU CNNs' approximation properties in 2D space.
problem Establishing L2 approximation properties for deep ReLU CNNs. method Analysis based on decomposition theorem for convolutional kernels, properties of ReLU activation, and connections with one-hidden-layer ReLU NNs.
result Universal approximation theorem for deep ReLU CNNs with classic structure.
Digital trees have approximate fixed point property, and conditions for products are explored.
problem Conditions for the approximate fixed point property in digital tree products.
method Analyzes digital trees and their products, explores conditions for the AFPP.
result Conditions are found for the AFPP in digital tree products.
The PAC-Bayesian approach is a powerful set of techniques to derive non- asymptotic risk bounds for random estimators. The corresponding optimal distribution of estimators, usually called the Gibbs posterior, is unfortunately intractable. One may sample from it using Markov chain Monte Carlo, but this is often too slow…
The paper extends a theorem to number fields without infinite places.
problem Finiteness properties of arithmetic approximate lattices.
method Geometric and homological finiteness properties for countable approximate groups.
result The finiteness length is finite and can be computed explicitly.
The paper studies geometric properties of quasi-trees and tree approximations.
problem Geometric properties and tree approximations of quasi-trees.
method Construction of a tree approximating quasi-trees, proving quasi-isometric properties.
result Every quasi-tree is (1,C)-quasi-isometric to a simplicial tree. Quasispheres can be approximated by smooth spheres.
problem Characterizing quasispheres using geometric conditions.
method Proving every quasisphere is a limit of smooth spheres and providing necessary and sufficient conditions for uniform quasispheres.
result Every quasisphere can be approximated by uniform quasispheres that satisfy specific geometric conditions.
Neural networks can approximate functions uniformly across various measures.
problem Universal approximation of functions across different probability measures.
method Proving neural networks are dense in Orlicz spaces, extending classical theorems.
result Neural networks uniformly approximate functions for weakly compact families of measures.
UDENet and ResNet can approximate any function, with ODENet showing UAP for continuous functions.
problem Approximating any function using ODENet and ResNet.
method Proved UAP for ODENet and ResNet, derived gradient, and applied to various problems.
result UDENet and ResNet can approximate any function, with ODENet showing UAP for continuous functions.
INNs can approximate diverse functions despite layer restrictions.
problem Can INNs approximate sufficiently diverse functions?
method Developed a theoretical framework based on differential geometry to simplify the approximation problem of diffeomorphisms.
result INNs have the universal approximation property.
In the last decade, the approximate vanishing ideal and its basis construction algorithms have been extensively studied in computer algebra and machine learning as a general model to reconstruct the algebraic variety on which noisy data approximately lie. In particular, the basis construction algorithms developed in ma…
New algorithms improve likelihood of finding global optima in Bayesian inference.
problem Finding global optima in Bayesian inference is difficult due to nonconvexity.
method Developed two algorithms: consistent Laplace approximation (CLA) and consistent stochastic variational inference (CSVI).
result Both CSVI and CLA improve likelihood of obtaining global optima compared to standard methods.
Estimating symmetric properties of a distribution, e.g. support size, coverage, entropy, distance to uniformity, are among the most fundamental problems in algorithmic statistics. While each of these properties have been studied extensively and separate optimal estimators are known for each, in striking recent work, Ac…
We consider a class of operator-induced norms, acting as finite-dimensional surrogates to the L2 norm, and study their approximation properties over Hilbert subspaces of L2 . The class includes, as a special case, the usual empirical norm encountered, for example, in the context of nonparametric regression in reproduci…
New efficient algorithm for approximate PML distribution.
problem Computing the profile maximum likelihood (PML) distribution efficiently.
method Exploiting sparsity structure and new matrix rounding algorithm.
result First provable computationally efficient implementation of PseudoPML.
In this article we study the validity of the Whitney C1 extension property for horizontal curves in sub-Riemannian manifolds endowed with 1-jets that satisfy a first-order Taylor expansion compatibility condition. We first consider the equiregular case, where we show that the extension property holds true whenever a…
NODEs can approximate a wide range of diffeomorphisms with strong guarantees.
problem The approximation power of NODEs under certain conditions.
method Leveraging a structure theorem of the diffeomorphism group.
result NODEs can approximate a large class of diffeomorphisms with a stronger guarantee.
Complex-valued neural networks can approximate any continuous function.
problem Generalizing the universal approximation theorem to complex-valued networks.
method Characterizing activation functions for complex networks to approximate any continuous function.
result Different activation functions are required for deep vs shallow complex networks to achieve universal approximation.
Universal approximation for ODENet and ResNet with a single activation function.
problem Approximating complex dynamical systems with limited vector fields.
method Examined ODENet and ResNet with vector fields composed of a single activation function and affine mapping.
result ODENet and ResNet with restricted vector fields can uniformly approximate those with general vector fields.
New algorithm learns value and advantage functions for continuous-time Markov processes without structural assumptions.
problem Learning value and advantage functions for continuous-time Markov processes without structural assumptions.
method Proposes Sobolev-prox fitted q-learning algorithm based on Hilbert-space positive definiteness and boundedness properties of Bellman operators. result Identifies ellipticity as a key structural property enabling reinforcement learning for Markov diffusions.
This paper analyzes and improves GANs' approximation ability.
problem Theoretical and algorithmic analysis of GANs' approximation property.
method Theoretical analysis and SDG approach to enhance GANs' approximation ability.
result The generator of GANs can universally approximate the potential data distribution.
We consider the fundamental learning problem of estimating properties of distributions over large domains. Using a novel piecewise-polynomial approximation technique, we derive the first unified methodology for constructing sample- and time-efficient estimators for all sufficiently smooth, symmetric and non-symmetric, …
New method uses scalar-based models to approximate spherical tensors efficiently.
problem Efficiently approximating spherical tensors with equivariant functions.
method Expressing equivariant functions as the product of a scalar function and a small tensor basis.
result Approximations are fast, simple to implement, and accurate in practical settings.
Generalizing Cusick's theorem on the closedness of the classical Lagrange spectrum for the approximation of real numbers by rational ones, we prove that various approximation spectra are closed, using penetration properties of the geodesic flow in cusp neighbourhoods in negatively curved manifolds and a result of Mauco…
We present the particle stochastic approximation EM (PSAEM) algorithm for learning of dynamical systems. The method builds on the EM algorithm, an iterative procedure for maximum likelihood inference in latent variable models. By combining stochastic approximation EM and particle Gibbs with ancestor sampling (PGAS), PS…
Dropout neural networks can approximate any function with high probability.
problem Approximating functions with dropout neural networks.
method Two universal approximation theorems for dropout neural networks in random and deterministic modes.
result Dropout neural networks can approximate any function in probability and in Lq. New method improves submodular maximization for machine learning applications.
problem Inexact monotonicity in submodular functions limits traditional algorithms' performance.
method Introduces monotonicity ratio as a continuous version of monotonicity, leading to improved approximation guarantees.
result Improved approximation ratios for movie recommendation, quadratic programming, and image summarization.
Paper explores properties of slice-matching operators for measure transfer.
problem Efficiently transferring measures in high dimensions.
method Examines an associated slice-matching operator with source, target measures and slicing directions.
result Establishes invariance, equivariance, Lipschitz continuity, and error bounds.
Efficiently selects important variables in high-dimensional logistic regression.
problem Variable selection in high-dimensional logistic regression with binary responses.
method Developed a variational empirical Bayes approach for efficient model space marginal distribution.
result The variational approximation inherits strong selection consistency from the posterior distribution.
Paper approximates Kelly betting for wealth growth.
problem Optimizing wealth growth in Kelly betting.
method Taylor-based approximation for quadratic programming.
result Closed-form approximate solution with interesting properties.
We study the asymptotic consistency properties of α-Rényi approximate posteriors, a class of variational Bayesian methods that approximate an intractable Bayesian posterior with a member of a tractable family of distributions, the member chosen to minimize the α-Rényi divergence from the true posterior. Unique to o…
Discrete conjugate systems are quadrilateral nets with all planar faces. Discrete orthogonal systems are defined by the additional property of all faces being concircular. Their geometric properties allow one to consider them as proper discretization of conjugate, resp. orthogonal coordinate systems of classical differ…
Neural networks can approximate complex stochastic equations well.
problem Approximating general stochastic differential equations.
method Identified neural network classes approximating continuous functions.
result Neural stochastic differential equations can approximate general stochastic differential equations arbitrarily well.
Positive definite kernels and their associated Reproducing Kernel Hilbert Spaces provide a mathematically compelling and practically competitive framework for learning from data. In this paper we take the approximation theory point of view to explore various aspects of smooth kernels related to their inferential proper…
Investigates the impact of finite VC dimension on neural network approximation and learning.
problem The influence of VC dimension on neural network approximation and learning from samples.
method Analysis of high-dimensional geometry and statistical learning theory, focusing on VC dimension.
result Finite VC dimension is beneficial for uniform convergence of empirical errors but not for approximation of functions from a probability distribution.
A metric space M us said to have the fibered approximation property in dimension n (br., M∈FAP(n)) if for any ε>0, m≥0 and any map g:Im×In→M there exists a map g′:Im×In→M such that g′ is ε-homotopic to g and dimg′({z}×In)≤n for all $z\i…
The paper shows how heat flow approximates area functional on specific geometric spaces.
problem Approximating the area functional on $\RCD(K,\infty)$ spaces.
method Using heat flow and properties of $\RCD(K,\infty)$ spaces.
result The area functional coincides with its relaxation in $\RCD(K,\infty)$ spaces.
Study reveals Transformer's expressive power and mechanisms.
problem Understanding the approximation properties of Transformer for sequence modeling.
method Systematic study of Transformer's components and their combined effects, establishing approximation rates.
result Reveals roles of critical parameters in Transformer, such as number of layers and attention heads.
We are analysing the convexity and continuity properties of the Mabuchi functional along weak geodesics. The key technical point in our paper is the global approximation of weak geodesics obtained via a well-chosen family of Monge-Ampère equations.
Good sparse approximations are essential for practical inference in Gaussian Processes as the computational cost of exact methods is prohibitive for large datasets. The Fully Independent Training Conditional (FITC) and the Variational Free Energy (VFE) approximations are two recent popular methods. Despite superficial …
Study approximates top Lyapunov exponents for surface mapping classes.
problem Approximating topological Lyapunov exponents for surface mapping classes.
method Periodic approximation and joint spectral radius extension.
result Top Lyapunov exponents can be approximated by periodic orbits.
Machine learning approximates Calabi-Yau Hodge numbers from weight systems.
problem Approximating Hodge numbers of Calabi-Yau manifolds from weight systems.
method Neural networks learned Hodge numbers from weight systems, symbolic regression inspired truncation, and machine learning generated new datasets.
result Approximation provides tight lower bounds and dramatically faster computation.
We propose a differentiable nonparametric algorithm, the Delaunay triangulation learner (DTL), to solve the functional approximation problem on the basis of a p-dimensional feature space. By conducting the Delaunay triangulation algorithm on the data points, the DTL partitions the feature space into a series of p-d…
The most fruitful approach to studying low energy soliton dynamics in field theories of Bogomol'nyi type is the geodesic approximation of Manton. In the case of vortices and monopoles, Stuart has obtained rigorous estimates of the errors in this approximation, and hence proved that it is valid in the low speed regime. …
New RBF networks can approximate any continuous function.
problem Approximating any continuous function on a compact subset.
method Replacing smoothing factors with shifts in RBF networks and proving approximation under certain conditions.
result RBF networks can approximate any continuous function on any compact subset.
The universal approximation property of various machine learning models is currently only understood on a case-by-case basis, limiting the rapid development of new theoretically justified neural network architectures and blurring our understanding of our current models' potential. This paper works towards overcoming th…
Paper shows L∞-positivity and stochastic completeness are equivalent.
problem Analyzing L∞-positivity preserving property and stochastic completeness. method Using monotone approximation results for distributional solutions of −Δ+1≥0. result The L∞-positivity preserving property is equivalent to stochastic completeness. Random feature models approximate functions in Banach spaces efficiently.
problem Approximating functions in Banach spaces efficiently.
method Randomly initialized feature maps and linear readout training.
result Universal approximation in Bochner spaces for Banach space-valued models.
Large-scale deep neural networks are both memory intensive and computation-intensive, thereby posing stringent requirements on the computing platforms. Hardware accelerations of deep neural networks have been extensively investigated in both industry and academia. Specific forms of binary neural networks (BNNs) and sto…