The paper defines approximate fibrations in higher topos theory.
problem Defining approximate fibrations in a new mathematical framework.
method Introducing approximate fibrations for geometric morphisms of ∞ \infty ∞ -topoi, providing characterizations and comparing to previous definitions. result Generalization of shape-theoretic characterizations to a topos-theoretical proof.
Deep learning networks are approximated using dynamical systems theory.
problem Understanding the approximation capabilities of deep learning networks.
method Modeling deep residual networks as continuous-time dynamical systems and using approximation theories in L p L^p L p . result Established general sufficient conditions for universal approximation of deep residual networks.
Unified theory of deep learning from approximation to emergence.
problem Understanding the mechanisms behind deep learning.
method Unified, proof-oriented approach tracing from classical foundations to contemporary mechanisms.
result Unified theory explaining deep learning from approximation to emergence.
Theory for deep neural network approximation of score function and its derivatives.
problem Handling data distributions with low-dimensional structure and unbounded support.
method Simultaneous approximation of the score function and its derivatives using deep neural networks.
result Approximation error bounds match literature but relax bounded support requirement.
Survey on learning Boolean functions in computational theory.
problem Learning Boolean function classes in computational theory.
method Overview of known results in PAC and related models.
result Discussion of various learning results for Boolean functions.
Deep, wide ConvResNets can approximate functions and their smoothness.
problem Function approximation and smoothness in deep networks.
method Analyzing ConvResNets, proving their ability to approximate functions and their smoothness.
result Large ConvResNets can approximate functions and exhibit sufficient first-order smoothness.
Develops a new theory for approximating functions on massive data.
problem Challenges in machine learning with massive data.
method eignets theory for local, stratified approximation.
result Solves inverse problems like finding data probability law and function smoothness.
We study sparse approximate solutions to convex optimization problems. It is known that in many engineering applications researchers are interested in an approximate solution of an optimization problem as a linear combination of elements from a given system of elements. There is an increasing interest in building such …
Unified theory for semi-implicit variational inference, bridging approximation and optimization.
problem Developing a statistical theory for semi-implicit variational inference.
method Unified theory combining approximation and optimization analyses.
result Unified theory characterizes SIVI's ability to recover target distributions and governs asymptotic behavior.
New game approximates mean curvature flow evolution.
problem Approximating geometric mean curvature flow evolution.
method Two-player zero-sum game with probabilistic elements.
result Value function approximates mean curvature flow.
Develops wavelet-based neural network approximation theory.
problem Analyzing neural network approximation capabilities over various activation functions.
method Wavelet frame theory on spaces of homogeneous type, sufficient conditions for approximation, error estimates.
result Derives sufficient conditions for neural networks to approximate any functions in a given space, including non-smooth activations.
This research develops approximation theory for OOMs of infinite-dimensional processes.
problem Developing an approximation theory for OOMs of infinite-dimensional processes.
method Establishing an inner product structure and proving continuity of observable operators.
result A fundamental obstacle in making an infinite-dimensional space of future distributions into a Hilbert space is described.
Simplified proof for approximations of set systems.
problem Approximations of set systems in various fields.
method Modular, self-contained proof using Chernoff's bound.
result Accessible proof for a wider audience.
Deep ReLU networks can approximate matrix-vector products with error bounds.
problem Can deep ReLU networks accurately approximate matrix-vector products?
method Derived error bounds in Lebesgue and Sobolev norms for deep ReLU FNNs.
result Developed deep approximation theory with successful applications.
The paper develops AMP theory for sparse and robust regression with polynomial iterations.
problem Challenges in high-dimensional statistical estimation due to asymptotic theory breakdown.
method Non-asymptotic distributional theory of AMP for sparse and robust regression.
result First finite-sample non-asymptotic distributional theory of AMP for polynomial iterations.
New Hermite approximations accelerate convergence with adaptive coordinate transformations.
problem Accelerating convergence of spectral approximations for Hermite expansions.
method Using normalizing flows for adaptive coordinate transformations and deriving error estimates.
result Error estimates for Hermite expansions under adaptive coordinate transformations.
In this paper, from a theoretical perspective, we study how powerful graph neural networks (GNNs) can be for learning approximation algorithms for combinatorial problems. To this end, we first establish a new class of GNNs that can solve a strictly wider variety of problems than existing GNNs. Then, we bridge the gap b…
Paper improves variational inference by tightening bounds using perturbation theory.
problem Improving variational inference's bias and KL divergence approximation.
method Revisits perturbation theory to derive corrections that tighten variational bounds.
result New bounds are tighter and more mass-covering, leading to higher likelihoods.
Improved neural network approximates analytic and L^p functions efficiently.
problem Efficiently approximating analytic and L^p functions using neural networks.
method Three-dimensional ReLU network architecture for sawtooth functions, improving approximation rates.
result Substantially improved exponential approximation rates for analytic functions and general L^p functions.
In this paper, we develop a theory about the relationship between G G G -invariant/equivariant functions and deep neural networks for finite group G G G . Especially, for a given G G G -invariant/equivariant function, we construct its universal approximator by deep neural network whose layers equip G G G -actions and each affine t…
New findings on how convolutional architectures approximate time series data.
problem Understanding the approximation properties of convolutional architectures in time series modeling.
method Mathematical analysis of convolutional architectures applied to time series modeling.
result A new definition of spectrum-based regularity for measuring temporal relationships under convolutional approximation.
Transformers enable in-context learning with guarantees for a wide range of tasks.
problem How to enable in-context learning with transformers for various tasks.
method Developed a universal approximation theory integrating Barron's function approximation with transformer capabilities.
result Transformers can approximate any target function with vanishingly small risk using a few in-context examples.
Study variation spaces for neural networks, linking them to approximation theory.
problem Understanding the variation spaces of shallow neural networks.
method Examined variation spaces defined by convex hulls and integral representations for a dictionary of functions.
result Found that Barron space, spectral Barron space, and Radon BV space are variation spaces for certain neural networks.
New knots found with tough, unsliceable discs.
problem Finding tough knots that can't be sliced smoothly.
method Constructed infinitely many knots with non-approximable slice discs.
result Smoothly sliceable knots have non-approximable slice discs.
Deep residual networks can approximate any continuous function using control theory.
problem Universal approximation capabilities of deep residual neural networks.
method Relating residual networks to control systems and using Lie algebraic techniques.
result Deep residual networks with adequately deep layers can approximate any continuous function on a compact set.
Gradient descent trains shallow neural networks to approximate functions in 1D.
problem Approximating functions in 1D with shallow neural networks trained by gradient descent.
method Gradient descent optimization of non-convex weight space for finite width networks in 1D.
result Gradient descent can approximate functions in 1D with a minimal number of weights, balancing practical performance and theoretical capabilities.
Paper develops a new kernel approximation framework.
problem High time and space complexity of kernel methods for large datasets.
method Perturbation-based kernel approximation framework using classical perturbation theory.
result Framework generalizes and improves upon existing methods.
Asynchronous stochastic approximations (SAs) are an important class of model-free algorithms, tools and techniques that are popular in multi-agent and distributed control scenarios. To counter Bellman's curse of dimensionality, such algorithms are coupled with function approximations. Although the learning/ control pro…
FQE with deep neural networks achieves asymptotic normality and finite-sample bounds.
problem Theoretical understanding of FQE with general differentiable function approximators.
method Z-estimation theory applied to FQE with deep neural networks.
result FQE estimation error is asymptotically normal with explicit variance.
Study approximates top Lyapunov exponents for surface mapping classes.
problem Approximating topological Lyapunov exponents for surface mapping classes.
method Periodic approximation and joint spectral radius extension.
result Top Lyapunov exponents can be approximated by periodic orbits.
New Zap Q-learning accelerates reinforcement learning with neural networks.
problem Accelerate convergence of reinforcement learning algorithms.
method Introduces a new framework for analysis of stochastic approximation algorithms, proving consistency under non-degeneracy assumption.
result Zap Q-learning with neural network function approximation converges quickly and is robust to function approximation architecture choice.
Survey discusses new ideas in geometric group theory and their applications.
problem Understanding geodesic metric spaces and their equivariant wall structures.
method Introduces and highlights the impact of injective metric spaces and cubical approximation theorem.
result Rich equivariant wall structures in various geodesic metric spaces.
Almost perimeter-minimizing boundaries in plentiful groups can be approximated by Lipschitz graphs.
problem Regularity of boundaries in plentiful groups.
method Lipschitz approximation of boundaries.
result Boundary of almost minimizers can be approximated by intrinsic Lipschitz graphs.
We study the Stochastic Gradient Descent (SGD) method in nonconvex optimization problems from the point of view of approximating diffusion processes. We prove rigorously that the diffusion process can approximate the SGD algorithm weakly using the weak form of master equation for probability evolution. In the small ste…
Paper studies Transformer learning theory for Euclidean and Riemannian domains.
problem Understanding and optimizing Transformer networks for regression tasks.
method Constructive approximation framework using softmax partition of unity and attention mechanism.
result Transformer can achieve uniform ε-approximation error with minimal parameters.
Gaussian and bootstrap methods improve ATE estimator accuracy.
problem Improving the accuracy of Average Treatment Effect (ATE) estimators.
method Gaussian approximation and bootstrap procedures.
result Precise bounds on ATE estimator accuracy quantifying key parameters.
We extend the Eliashberg-Thurston theorem on approximations of taut oriented C 2 C^2 C 2 -foliations of 3-manifolds by both positive and negative contact structures to a large class of taut oriented C 1 , 0 C^{1,0} C 1 , 0 -foliations, where by C 1 , 0 C^{1,0} C 1 , 0 foliation, we mean a foliation with continuous tangent plane field. These C 1 , 0 C^{1,0} C 1 , 0 -fol…
New study shows exponential sample growth for ReQU neural networks.
problem Computing neural network approximations from samples is challenging.
method Information-based complexity tools.
result Functions can be approximated by ReQU neural networks at arbitrary rates but require exponentially growing samples.
New theory approximates functions between metric spaces using random probability measures.
problem Building universal functions approximators between arbitrary metric spaces.
method Using elementary functions between Euclidean spaces, randomization to output discrete probability measures over target space.
result Very general qualitative guarantees and quantitative guarantees for Hölder-like maps.
Paper develops neural network for distribution regression.
problem Regression with probability measures.
method Develops a novel fully connected neural network (FNN) for distribution inputs.
result Almost optimal learning rates for distribution regression derived.
We propose a general formalism of iterated random functions with semigroup property, under which exact and approximate Bayesian posterior updates can be viewed as specific instances. A convergence theory for iterated random functions is presented. As an application of the general theory we analyze convergence behaviors…
New algorithms tackle RKHS bandits with reduced complexity and improved performance.
problem Adversarial and stochastic RKHS bandit problems with high computational complexity.
method Combining approximation theory with misspecified linear bandit methods.
result First general algorithm for adversarial RKHS bandit problem.
Survey on statistical theories of neural networks, focusing on approximation, training dynamics, and generative models.
problem Understanding the statistical properties and training dynamics of neural networks.
method Review of existing literature on neural networks from three perspectives: approximation, training dynamics, and generative models.
result Theoretical insights into neural network training dynamics and generative models.
New framework connects two neural network theories, improving finite-width approximations.
problem Theoretical guarantees for neural network training in general cases.
method Developed a general framework linking mean-field and constant kernel theories.
result Discrete-time MF limit provides better approximation for finite-width nets.
Paper develops approximation and statistical theory for signature-based path regression.
problem Understanding how fast signatures approximate continuous path functionals.
method Develops \(L^2\) approximation rate for smooth functionals of Itô diffusions and establishes consistency of statistical learning procedures.
result Signature-based methods improve prediction over handcrafted features in various real-data applications.
Neural networks approximate high-dimensional functions better than theory predicts.
problem Current theory struggles to explain why small neural networks work well in high-dimensional inverse problems.
method Bounding complexity required for neural networks to approximate Hölder or uniformly continuous functions on high-dimensional sets.
result A general theoretical framework explaining empirical successes of smaller networks in inverse problems.
Paper develops efficient RL algorithm for general value function approximation.
problem Lack of theory for RL with general value function approximation.
method Provable efficient RL algorithm using bounded eluder dimension.
result Achieves a regret bound of O ~ ( p o l y ( d H ) T ) \widetilde{O}(\mathrm{poly}(dH)\sqrt{T}) O ( poly ( d H ) T ) . This paper develops fundamental limits of deep neural network learning by characterizing what is possible if no constraints are imposed on the learning algorithm and on the amount of training data. Concretely, we consider Kolmogorov-optimal approximation through deep neural networks with the guiding theme being a relat…