Uniform diffusion approximation for SGD in non-convex settings.
problem Finite-time diffusion approximation for SGD.
method Establishing uniform-in-time diffusion approximation with strong convexity and mild conditions.
result Uniform-in-time diffusion approximation of SGD without convexity of each loss function.
Paper proves discrete uniformizations converge to continuous for surfaces of genus ≥1.
problem Computing uniformizations for surfaces of genus >1.
method Discrete conformality and uniformization on triangle meshes.
result Discrete uniformizations approximate continuous uniformization for closed surfaces of genus ≥1.
We show that the sets in a family with finite VC dimension can be uniformly approximated within a given error by a finite partition. Immediate corollaries include the fact that VC classes have finite bracketing numbers, satisfy uniform laws of averages under strong dependence, and exhibit uniform mixing. Our results ar…
Solves approximation problems for zonoids and neural networks, closing gaps in dimensions 2 and 3.
problem Approximating zonoids and shallow neural networks in uniform norm.
method Combines techniques to solve both problems, closing gaps in dimensions 2 and 3.
result Completes the solution for zonoid approximation in all dimensions and improves neural network approximation rates.
Quasispheres can be approximated by smooth spheres.
problem Characterizing quasispheres using geometric conditions.
method Proving every quasisphere is a limit of smooth spheres and providing necessary and sufficient conditions for uniform quasispheres.
result Every quasisphere can be approximated by uniform quasispheres that satisfy specific geometric conditions.
Paper generalizes discrete uniformization for genus-zero surfaces.
problem Discrete uniformization for surfaces of genus zero.
method Reduction to planar cases via stereographic projections.
result Generalization of discrete uniformization to genus-zero surfaces.
We approximate the heat kernel h(x,y,t) on a compact connected Riemannian manifold M without boundary uniformly in (x,y,t)∈M×M×[a,b], a>0, by n-fold integrals over Mn of the densities of Brownian bridges. Moreover, we provide an estimate for the uniform convergence rate. As an immediate coro…
New algorithm FLUTE achieves uniform-PAC convergence in RL with linear approx.
problem RL with linear function approximation lacks uniform-PAC guarantees.
method FLUTE algorithm with minimax value function estimator and multi-level partition scheme.
result Uniform-PAC convergence to optimal policy with high probability.
The paper develops time-uniform inference methods for stochastic approximation parameters.
problem Statistical inference for parameters in stochastic approximation problems.
method Analysis of averaged iterates convergence rates and construction of asymptotic confidence sequences.
result Valid asymptotic confidence sequences for parameters in stochastic approximation problems.
Length metrics can be closely approximated by conformally flat metrics.
problem Approximating length metrics with conformally flat metrics.
method Uniform approximation of length metrics by conformally flat Riemannian metrics.
result Any length metric on \(\mathbb{R}^d\) can be uniformly approximated by conformally flat Riemannian metrics.
Random sampling has become a critical tool in solving massive matrix problems. For linear regression, a small, manageable set of data rows can be randomly selected to approximate a tall, skinny data matrix, improving processing time significantly. For theoretical performance guarantees, each row must be sampled with pr…
MPNN improves on UniFL approximation with provable guarantees.
problem Uniform Facility Location (UniFL) optimization problem.
method Graph Neural Network (MPNN) incorporating approximation-algorithmic principles.
result Empirically outperforms standard approximation algorithms.
Counterexamples show failure of uniform laws of large numbers for subdifferentials.
problem Failure of uniform laws of large numbers for subdifferentials under natural assumptions.
method Univariate and bivariate random Lipschitz and convex functions with smooth pieces.
result Counterexamples demonstrate failure of uniform laws of large numbers for subdifferentials.
Recently, artificial neural networks (ANNs) in conjunction with stochastic gradient descent optimization methods have been employed to approximately compute solutions of possibly rather high-dimensional partial differential equations (PDEs). Very recently, there have also been a number of rigorous mathematical results …
Uniform approximations for RHTs improve kernel approximation and distance estimation.
problem Theoretical guarantees for RHTs in low-dimensional applications.
method Proved uniform convergence of average of function over RHTs entries.
result Improved guarantees for kernel approximation and distance estimation.
Smooth DNNs mitigate the curse of dimensionality in uniform convergence for various regression tasks.
problem The curse of dimensionality in uniform convergence of ReLU networks.
method Analysis of smoothly activated deep neural networks (smooth DNNs), establishing pseudo-dimension bounds and non-asymptotic approximation guarantees.
result Smooth DNNs achieve non-asymptotic uniform convergence rates across multiple statistical contexts, mitigating the curse of dimensionality.
New algorithms achieve uniform-PAC guarantees for RL with bounded eluder dimension.
problem Achieving strong performance guarantees in reinforcement learning.
method Proposes algorithms for nonlinear bandits and model-based episodic RL with a bounded eluder dimension.
result Achieves uniform-PAC sample complexity that matches state-of-the-art regret bounds or sample complexity guarantees.
This paper refines homotopy theory for cubical sets and uniform spaces.
problem Classical homotopy theory limitations in cubical sets and uniform spaces.
method Develops a uniform-theoretic refinement for cubical sets and uniform spaces, lifting to a full and faithful embedding.
result Lifts classical homotopy categories to new uniform homotopy categories, generalizing cohomology theories.
Study on curvature invariants near singularities of wavefronts.
problem Conditions for extendibility and boundedness of curvature invariants.
method Investigation of Gaussian curvature, Mean curvature, and principal curvatures near singularities.
result Relationship between convergence to infinity and uniform approximation of fronts.
Uniform TD(0) bound derived for function approximation with Markov noise.
problem Uniform concentration bound for TD(0) with function approximation.
method Contractive stochastic approximation, martingale and Markov noises, Poisson equation, relaxed concentration inequalities.
result Uniform all-time concentration bound for TD(0) with linear function approximation.
Develops a framework for distilling flow models from few steps.
problem Improving few-step sampling in diffusion models for better performance.
method Local approximation errors and dynamical amplification controlled through analytical tractability.
result Deep residual compositions efficiently approximate long-horizon transport with controlled global error.
The paper provides Gaussian approximations for decentralized Federated Learning.
problem Lack of asymptotic statistical guarantees for local SGD in Federated Learning.
method Two generalized Gaussian approximation results for local SGD trajectories.
result Valid multiplier bootstrap procedures and Gaussian bootstrap-based tests for detecting adversarial attacks.
This study uses neural networks to approximate Bayesian filtering problems.
problem Estimating latent time-series signal statistics from observation sequences.
method Formulated a generic recurrent neural network framework to learn recursive mappings directly.
result Approximation error bounds for filtering in non-compact domains and strong time-uniform bounds.
We consider variants of trust-region and cubic regularization methods for non-convex optimization, in which the Hessian matrix is approximated. Under mild conditions on the inexact Hessian, and using approximate solution of the corresponding sub-problems, we provide iteration complexity to achieve ε-approximate seco…
Deep neural nets approximate random dynamical system trajectories uniformly in time.
problem Approximating trajectories of random dynamical systems over infinite time horizons.
method Recurrent neural networks with simple feedback structures.
result Certain random trajectories can be approximated uniformly in time to any desired accuracy.
UMAP (Uniform Manifold Approximation and Projection) is a novel manifold learning technique for dimension reduction. UMAP is constructed from a theoretical framework based in Riemannian geometry and algebraic topology. The result is a practical scalable algorithm that applies to real world data. The UMAP algorithm is c…
Analytic networks with bounded coefficients can't outperform polynomial approximations.
problem Approximation limits of neural networks with analytic activation functions under coefficient constraints.
method Deterministic analysis using comparison argument and Bernstein-type estimates.
result Networks with analytic activation functions and controlled coefficients cannot outperform classical polynomial approximation rates on non-analytic targets.
The paper studies geometric properties of quasi-trees and tree approximations.
problem Geometric properties and tree approximations of quasi-trees.
method Construction of a tree approximating quasi-trees, proving quasi-isometric properties.
result Every quasi-tree is (1,C)-quasi-isometric to a simplicial tree. Diffusion approximation provides weak approximation for stochastic gradient descent algorithms in a finite time horizon. In this paper, we introduce new tools motivated by the backward error analysis of numerical stochastic differential equations into the theoretical framework of diffusion approximation, extending the …
Let U⊆Rn be open and convex. We show that every (not necessarily Lipschitz or strongly) convex function f:U→R can be approximated by real analytic convex functions, uniformly on all of U. In doing so we provide a technique which transfers results on uniform approximation on bounded …
Uniform convergence of metrics on surfaces with bounded curvature measures proved.
problem Proving uniform convergence of metrics on Alexandrov surfaces with bounded integral curvature.
method Weak convergence of measures and analytic approximation of metrics.
result Uniform convergence of metrics on Alexandrov surfaces proved.
Paper introduces Simplet Frequency Distribution (SFD) for SCs.
problem Frequency analysis of simplets in large SCs.
method Developed SFD vector and uniform sampling-based algorithm.
result Validated theoretical bounds with experiments.
Paper introduces a neural network training algorithm for noisy data that achieves optimal parameters and replicates real-world behaviors.
problem Theoretical gap between universal approximation theorems and practical machine learning with noisy data.
method Randomized training algorithm for neural networks trained on noisy data samples.
result Trained neural networks achieve optimal parameters and exhibit real-world behaviors like sub-linear complexity and interpolation.
New learning rule for quantum measurement classes overcomes uniform convergence issues.
problem Characterizing learnability of POVM hypothesis classes in quantum settings.
method Introduced a new learning rule called denoised ERM to address uniform convergence issues.
result Characterized learnability conditions and sample complexity bounds for POVM classes.
Many Markov Chain Monte Carlo (MCMC) methods leverage gradient information of the potential function of target distribution to explore sample space efficiently. However, computing gradients can often be computationally expensive for large scale applications, such as those in contemporary machine learning. Stochastic Gr…
We prove that a compactly supported homeomorphism of a smooth manifold of dimension greater or equal to 5 can be approximated uniformly by compactly supported diffeomorphisms if and only if it is isotopic to a diffeomorphism. If the given homeomorphism is in addition volume preserving, then it can be approximated unifo…
Statistical performance bounds for reinforcement learning (RL) algorithms can be critical for high-stakes applications like healthcare. This paper introduces a new framework for theoretically measuring the performance of such algorithms called Uniform-PAC, which is a strengthening of the classical Probably Approximatel…
Algorithm learns CNF formulas from random solutions under specific conditions.
problem Learning a CNF formula from uniform random solutions.
method Revisits Valiant's algorithm and applies Lovász local lemma conditions.
result Significantly reduces sample complexity for learning CNFs.
Matrix completion has been well studied under the uniform sampling model and the trace-norm regularized methods perform well both theoretically and numerically in such a setting. However, the uniform sampling model is unrealistic for a range of applications and the standard trace-norm relaxation can behave very poorly …
Paper shows how to integrate quantization into neural compression models.
problem Integrating quantization into neural compression models.
method Integrates uniform noise channel at test time using universal quantization.
result Eliminates mismatch between training and test phases while maintaining differentiability.
Unified framework for uniform signal recovery in nonlinear GCS with 1-bit/quantized measurements.
problem Uniform recovery guarantees for nonlinear generative compressed sensing.
method Unified framework using generalized Lasso and Lipschitz approximation.
result Uniform recovery of all signals in the ball up to an error of ε using approximately O(k/ε^2) samples.
Residual networks with block width max(d_x, d_y) approximate all functions.
problem Achieving universal approximation with residual networks.
method Established bounds on block width for different activation functions.
result Minimum block width for universal approximation is max(d_x, d_y) with inner width 1.
Energy-efficient sampling for machine learning using magnetic tunnel junctions.
problem Costly and inefficient random sampling in machine learning.
method Energy-efficient algorithm using stochastic magnetic tunnel junctions for uniform Float16 sampling.
result Higher energy efficiency than state-of-the-art algorithms, with a minimum factor of 9721.
This work establishes uniform convergence of subdifferentials in stochastic optimization.
problem Understanding how empirical stationary points approximate population ones in nonsmooth, nonconvex stochastic optimization.
method Reduction principle for weakly convex stochastic objectives, focusing on subgradient convergence.
result Sharp uniform convergence rates for subdifferential mappings in stochastic convex-composite optimization.
Paper studies shallow ReLU networks' approximation rates for Hölder functions.
problem Understanding shallow ReLU networks' efficiency in approximating Hölder functions.
method Analyzes rates of uniform approximation by ReLU shallow neural networks with m hidden neurons. result Shows ReLU shallow neural networks can uniformly approximate Hölder functions with rates close to optimal.
Three themes of general topology: quotient spaces; absolute retracts; and inverse limits - are reapproached here in the setting of metrizable uniform spaces, with an eye to applications in geometric and algebraic topology. The results include: 1) If f: A -> Y is a uniformly continuous map, where X and Y are metric spac…
This note gives a simple analysis of a randomized approximation scheme for matrix multiplication proposed by Sarlos (2006) based on a random rotation followed by uniform column sampling. The result follows from a matrix version of Bernstein's inequality and a tail inequality for quadratic forms in subgaussian random ve…
We prove a persistence result for noncompact normally hyperbolic invariant manifolds in the setting of Riemannian manifolds of bounded geometry. Bounded geometry of the ambient manifold is a crucial assumption required to control the uniformity of all estimates throughout the proof. The Ck,α-smoothness result is o…