arXiv research

The paper defines approximate fibrations in higher topos theory.

Unified theory of deep learning from approximation to emergence.

Theory for deep neural network approximation of score function and its derivatives.

Survey on learning Boolean functions in computational theory.

Deep, wide ConvResNets can approximate functions and their smoothness.

Develops a new theory for approximating functions on massive data.

We study sparse approximate solutions to convex optimization problems. It is known that in many engineering applications researchers are interested in an approximate solution of an optimization problem as a linear combination of elements from a given system of elements. There is an increasing interest in building such …

Unified theory for semi-implicit variational inference, bridging approximation and optimization.

We build on the dynamical systems approach to deep learning, where deep residual networks are idealized as continuous-time dynamical systems, from the approximation perspective. In particular, we establish general sufficient conditions for universal approximation using continuous-time deep residual networks, which can …

New game approximates mean curvature flow evolution.

Develops wavelet-based neural network approximation theory.

This research develops approximation theory for OOMs of infinite-dimensional processes.

Simplified proof for approximations of set systems.

Deep ReLU networks can approximate matrix-vector products with error bounds.

The paper develops AMP theory for sparse and robust regression with polynomial iterations.

New Hermite approximations accelerate convergence with adaptive coordinate transformations.

In this paper, from a theoretical perspective, we study how powerful graph neural networks (GNNs) can be for learning approximation algorithms for combinatorial problems. To this end, we first establish a new class of GNNs that can solve a strictly wider variety of problems than existing GNNs. Then, we bridge the gap b…

Improved neural network approximates analytic and L^p functions efficiently.

In this paper, we develop a theory about the relationship between $G$-invariant/equivariant functions and deep neural networks for finite group $G$. Especially, for a given $G$-invariant/equivariant function, we construct its universal approximator by deep neural network whose layers equip $G$-actions and each affine t…

New findings on how convolutional architectures approximate time series data.

Study variation spaces for neural networks, linking them to approximation theory.

Transformers enable in-context learning with guarantees for a wide range of tasks.

New knots found with tough, unsliceable discs.

Deep residual networks can approximate any continuous function using control theory.

Variational inference has become one of the most widely used methods in latent variable modeling. In its basic form, variational inference employs a fully factorized variational distribution and minimizes its KL divergence to the posterior. As the minimization can only be carried out approximately, this approximation i…

Gradient descent trains shallow neural networks to approximate functions in 1D.

Paper develops a new kernel approximation framework.

Asynchronous stochastic approximations (SAs) are an important class of model-free algorithms, tools and techniques that are popular in multi-agent and distributed control scenarios. To counter Bellman's curse of dimensionality, such algorithms are coupled with function approximations. Although the learning/ control pro…

FQE with deep neural networks achieves asymptotic normality and finite-sample bounds.

Study approximates top Lyapunov exponents for surface mapping classes.

Survey discusses new ideas in geometric group theory and their applications.

Almost perimeter-minimizing boundaries in plentiful groups can be approximated by Lipschitz graphs.

We study the Stochastic Gradient Descent (SGD) method in nonconvex optimization problems from the point of view of approximating diffusion processes. We prove rigorously that the diffusion process can approximate the SGD algorithm weakly using the weak form of master equation for probability evolution. In the small ste…

Paper studies Transformer learning theory for Euclidean and Riemannian domains.

Gaussian and bootstrap methods improve ATE estimator accuracy.

We extend the Eliashberg-Thurston theorem on approximations of taut oriented $C^2$-foliations of 3-manifolds by both positive and negative contact structures to a large class of taut oriented $C^{1,0}$-foliations, where by $C^{1,0}$ foliation, we mean a foliation with continuous tangent plane field. These $C^{1,0}$-fol…

New study shows exponential sample growth for ReQU neural networks.

Zap Q-learning is a recent class of reinforcement learning algorithms, motivated primarily as a means to accelerate convergence. Stability theory has been absent outside of two restrictive classes: the tabular setting, and optimal stopping. This paper introduces a new framework for analysis of a more general class of r…

New theory approximates functions between metric spaces using random probability measures.

We propose a general formalism of iterated random functions with semigroup property, under which exact and approximate Bayesian posterior updates can be viewed as specific instances. A convergence theory for iterated random functions is presented. As an application of the general theory we analyze convergence behaviors…

Paper develops neural network for distribution regression.

New algorithms tackle RKHS bandits with reduced complexity and improved performance.

Survey on statistical theories of neural networks, focusing on approximation, training dynamics, and generative models.

New framework connects two neural network theories, improving finite-width approximations.

Paper develops approximation and statistical theory for signature-based path regression.

Neural networks approximate high-dimensional functions better than theory predicts.

Paper develops efficient RL algorithm for general value function approximation.

This paper develops fundamental limits of deep neural network learning by characterizing what is possible if no constraints are imposed on the learning algorithm and on the amount of training data. Concretely, we consider Kolmogorov-optimal approximation through deep neural networks with the guiding theme being a relat…