The abstract discusses convergence properties of Lipschitz functions and sets defined by equations.
problem Convergence of Lipschitz functions and sets defined by equations.
method Painlevé-Kuratowski convergence applied to Lipschitz functions and sets defined by equations.
result Generalizations and reverses of classical theorems on convergence of functions and sets.
The objective of this paper is to introduce the notion of generalized almost statistical (briefly, GAS) convergence of bounded real sequences, which generalizes the notion of almost convergence as well as statistical convergence of bounded real sequences. As a special kind of Banach limit functional, we also introduce …
Equivalence shown between two mathematical concepts for hyperbolic surfaces.
problem None explicitly stated, but related to mathematical equivalence of concepts.
method Benjamini-Schramm convergence and zeta functions equivalence demonstration.
result Equivalence of Benjamini-Schramm convergence and zeta functions for compact hyperbolic surfaces.
Entropy-regularized NPG converges linearly with linear function approximation.
problem Analyzing convergence of entropy-regularized NPG with function approximation.
method Established finite-time convergence analyses with entropy regularization and linear function approximation.
result Entropy-regularized NPG achieves linear convergence up to a function approximation error.
Study of convergence in Lorentzian spacetimes using temporal functions.
problem Non-compactness of spacetime isometries and convergence in semi-Riemannian settings.
method Introduced anchored convergence and used Cauchy temporal functions to define convergence for spacetimes.
result Established local and global regularity of Cauchy temporal functions and their properties.
This paper improves convergence guarantees for SGD algorithms in non-convex smooth functions.
problem Theoretical convergence properties of SGD algorithms for non-convex smooth functions.
method Analysis of SGD algorithms with arbitrary data ordering for non-convex smooth functions.
result Enhanced convergence guarantees for incremental gradient and single shuffle SGD, improving the optimization term of convergence guarantee.
Develops uniform convergence guarantees for a broad class of risk functionals in supervised learning.
problem Bounding generalization gaps for various risk functionals beyond the expectation.
method Establishes uniform convergence for Hölder risk functionals, providing guarantees for empirical risk minimization.
result First uniform convergence results for estimating the CDF of loss distributions, applicable to various risk functionals.
Random permutations can offer faster convergence than with-replacement sampling for some functions.
problem Understanding when and how random permutations outperform with-replacement sampling in SGD convergence.
method Analyzing convergence rates for different function classes (1D strongly convex, general strongly convex, quadratic strongly convex).
result The optimal convergence gap between random and permutation-based SGD varies from exponential to nonexistent, depending on the function class.
SONATA algorithm converges to solutions of nonconvex smooth functions with KL property.
problem Decentralized optimization over networks with nonconvex smooth functions and convex constraints.
method Decentralized gradient-tracking algorithm SONATA under the KL property.
result SONATA converges to stationary solutions at R-linear rate for θ ∈ ( 0 , 1 / 2 ] θ\in (0,1/2] θ ∈ ( 0 , 1/2 ] , sublinear rate for θ ∈ ( 1 / 2 , 1 ) θ\in (1/2,1) θ ∈ ( 1/2 , 1 ) , and R-linear rate for θ = 0 θ=0 θ = 0 . The stochastic gradient descent has been widely used for solving composite optimization problems in big data analyses. Many algorithms and convergence properties have been developed. The composite functions were convex primarily and gradually nonconvex composite functions have been adopted to obtain more desirable prop…
We prove the local convergence to minima and estimates on the rate of convergence for the stochastic gradient descent method in the case of not necessarily globally convex nor contracting objective functions. In particular, the results are applicable to simple objective functions arising in machine learning.
Generative adversarial networks (GAN) approximate a target data distribution by jointly optimizing an objective function through a "two-player game" between a generator and a discriminator. Despite their empirical success, however, two very basic questions on how well they can approximate the target distribution remain…
Study shows convergence rate for empirical minimizer of unbounded functions with fast growth.
problem Convergence rate of empirical minimizer for unbounded functions with fast growth.
method Analyzes L 1 L^1 L 1 -distance convergence rate of the empiric minimizer for coercive functions sampled with noise. result Convergence rate is bounded above by a n n − 1 / q a_n n^{-1/q} a n n − 1/ q , where q q q is the dimension and a n = o ( n ε ) a_n = o(n^\varepsilon) a n = o ( n ε ) for every ε > 0 \varepsilon > 0 ε > 0 . SGD converges to global minimum for structured non-convex functions.
problem Optimizing non-convex functions using SGD with slow convergence rates.
method Convergence theorems for SGD on structured non-convex functions, including Quasar and PL conditions.
result SGD converges to global minimum for specific non-convex functions under certain conditions.
Improved convergence speed of principal component analysis through modified learning rules.
problem Slow convergence for covariance matrices with close eigenvalues.
method Introduced an additional term to the objective function to mitigate convergence issues.
result Significantly improved convergence speed confirmed through simulations.
Improved Frank-Wolfe algorithm for generalized self-concordant functions converges quickly.
problem Efficiently solving learning problems with generalized self-concordant objectives.
method Simple Frank-Wolfe variant with open-loop step size strategy γ t = 2 / ( t + 2 ) γ_t = 2/(t+2) γ t = 2/ ( t + 2 ) . result Achieves O ( 1 / t ) \mathcal{O}(1/t) O ( 1/ t ) convergence rate for primal and Frank-Wolfe gaps. Paper analyzes convergence rates of SGD for non-convex functions under various assumptions.
problem Analyzing convergence rates of SGD for non-convex functions.
method Studied convergence properties of Stochastic Gradient Descent (SGD) for invex functions under weaker and stronger hypotheses.
result Derives estimates on the rate of convergence of $J(oldsymbolθ_t)$ to its limit for functions satisfying the Polyak-Lojasiewicz (PL) condition.
Improves online learning algorithms for functional models with capacity assumptions.
problem Convergence rates of online stochastic gradient descent algorithms for functional linear models.
method Characterizations of slope function regularity, kernel space capacity, and sampling process covariance operator.
result Capacity assumptions can alleviate saturation of convergence rates as function regularity increases.
Adam optimizer converges to zeros of a new vector field, not just gradient zeros.
problem Prove convergence rates for Adam optimizer in simple quadratic optimization problems.
method Introduced Adam vector field to analyze Adam optimizer's convergence.
result Established optimal convergence rates for Adam optimizer.
Bayesian method with Gaussian process priors achieves optimal convergence rates for regression function and its derivatives.
problem Estimating the regression function and its derivatives in nonparametric regression.
method Bayesian approach with Gaussian process priors, focusing on convergence rates and plug-in property.
result Equivalence of convergence rates of posterior distributions and Bayes estimators for regression function and its derivatives.
Deep learning method proves convergence for high-dimensional PDEs.
problem Solving high-dimensional nonlinear PDEs for mean field control problems.
method Deep Galerkin method (DGM) for Hamilton-Jacobi-Bellman (HJB) equations.
result DGM converges to the true value function of mean field control problems.
The paper develops a uniform function estimator in RKHS for regression.
problem Reconstructing functions from noisy data at random locations.
method Using reproducing kernel Hilbert spaces and Gaussian random fields.
result The estimator converges uniformly to the conditional expectation.
In this paper, we propose two discontinuous dynamical systems in continuous time with guaranteed prescribed finite-time local convergence to strict local minima of a given cost function. Our approach consists of exploiting a Lyapunov-based differential inequality for differential inclusions, which leads to finite-time …
We prove that for analytic functions in low dimension, the convergence rate of the deep neural network approximation is exponential.
New algorithm speeds up Lasso computation by proving faster convergence.
problem Lasso estimator's slow convergence rate due to ℓ 1 \ell_1 ℓ 1 penalty. method Homotopic approach using surrogate functions.
result Proves O ( [ log ( 1 / ε ) ] 2 ) O([\log(1/ε)]^2) O ([ log ( 1/ ε ) ] 2 ) convergence rate for Lasso computation. Proves continuum limits of Lipschitz learning using Γ-convergence.
problem Semi-supervised learning with graph-based methods and continuum limits of p p p -Laplacian learning. method Proves continuum limits of Lipschitz learning using Γ-convergence.
result Proves Γ Γ Γ -convergence in the L ∞ L^\infty L ∞ -topology to the supremum norm of the gradient. New algorithms accelerate value function convergence in MDPs.
problem Accelerating convergence of value functions in Markov Decision Processes (MDPs).
method Operator Splitting Value Iteration (OS-VI) and OS-Dyna.
result Achieves much faster convergence rate with accurate models.
SGD converges to global minimum for certain non-convex functions.
problem Theoretical challenges in optimizing non-convex functions in machine learning.
method Perturbed SGD on a broad class of non-convex functions.
result SGD converges to global minimum for certain non-convex functions.
Continuous-time distributed mirror descent with integral feedback converges to global optimum.
problem Distributed optimization of a global strongly convex function with local convex components.
method Continuous-time distributed mirror descent with integral feedback.
result Asymptotic convergence to global optimum with constant step-size.
We introduce a natural definition of L p L^p L p -convergence of maps, p ≥ 1 p \ge 1 p ≥ 1 , in the case where the domain is a convergent sequence of measured metric space with respect to the measured Gromov-Hausdorff topology and the target is a Gromov-Hausdorff convergent sequence. With the L p L^p L p -convergence, we establish a theory of …
Temporal difference learning explained through gradient splitting, improving convergence times.
problem Learning value functions in Markov Decision Processes with linear approximations.
method Interpreting TD learning as gradient splitting and applying convergence proofs from gradient descent.
result Improved convergence times for TD learning, especially with a minor variation.
New Hermite approximations accelerate convergence with adaptive coordinate transformations.
problem Accelerating convergence of spectral approximations for Hermite expansions.
method Using normalizing flows for adaptive coordinate transformations and deriving error estimates.
result Error estimates for Hermite expansions under adaptive coordinate transformations.
Paper develops an online learning algorithm for functional data models.
problem Recovering slope functions or predictors in functional data models.
method Online regularized learning algorithm in reproducing kernel Hilbert spaces with polynomially decaying step-size.
result Established fast convergence rates for estimation error without capacity assumption.
While there are convergence guarantees for temporal difference (TD) learning when using linear function approximators, the situation for nonlinear models is far less understood, and divergent examples are known. Here we take a first step towards extending theoretical convergence guarantees to TD learning with nonlinear…
Green functions on stationary varifolds established with inequalities and convergence results.
problem Establishing Green functions on stationary varifolds with inequalities and convergence.
method Extending Grüter and Widman's method, constructing Green functions, using local Harnack inequality.
result Green functions converge for sequences of stationary varifolds converging with multiplicity one.
Proves weak convergence equals mean convergence in GGC.
problem Proving convergence in GGC distributions.
method Using generalized gamma convolution (GGC) and expected utility maximization.
result Weak convergence implies mean convergence in GGC.
Study convergence of simulated annealing in continuous and discrete settings.
problem Analyzing convergence rate of simulated annealing methods.
method Apply Eyring-Kramers law to prove polynomial decay of tail probabilities.
result Explicit rate of convergence for continuous and discrete simulated annealing.
Improved convergence rates for MLE in mixture models using penalized log-likelihood.
problem Convergence rates for MLE in finite mixture models.
method Penalizing log-likelihood to discourage vanishing mixing weights, using Wasserstein distance and new loss functions.
result Improved convergence rates for some mixture components, faster than traditional methods.
SignSVRG improves SignSGD by reducing variance, achieving similar convergence rates.
problem Minimizing finite sums of convex and Lipschitz functions.
method Incorporates variance reduction techniques into SignSGD.
result Achieves convergence rates of O ( 1 / T ) \mathcal{O}(1 / \sqrt{T}) O ( 1/ T ) for expected norm of the gradient and O ( 1 / T ) \mathcal{O}(1/T) O ( 1/ T ) for smooth convex functions. We explore the distinctions between L p L^p L p convergence of metric tensors on a fixed Riemannian manifold versus Gromov-Hausdorff, uniform, and intrinsic flat convergence of the corresponding sequence of metric spaces. We provide a number of examples which demonstrate these notions of convergence do not agree even for two…
Study on how sampling works for complex data functions.
problem Analyzing convergence of sampling algorithms for RKHS functions.
method Minimalistic assumptions on kernel and data, error estimates in RKHS norm, uniform convergence on compact domains.
result New convergence rates for Lipschitz and Hölder continuous kernels.
Improved SGD methods converge faster for nonconvex optimization.
problem Nonconvex optimization challenges in machine learning.
method Adaptive SGD with line-search and Polyak stepsizes.
result Unified convergence rates for various nonconvex functions.
Wide neural networks can be closely approximated by Gaussian processes, with rates depending on the activation function's properties.
problem Approximating the behavior of wide neural networks using Gaussian processes.
method Established convergence rates for the central limit theorem in an infinite-dimensional functional space, using a transportation distance metric.
result Explicit convergence rates for neural networks approximated by Gaussian processes, varying based on the activation function's properties.
The paper analyzes convergence of Langevin dynamics with time-dependent metrics.
problem Analyzing convergence of Langevin dynamics with time-dependent metrics.
method Formulated a modified gradient flow of the Kullback-Leibler divergence, selected a time-dependent relative Fisher information functional, and developed a time-dependent Hessian matrix condition.
result Proved convergence conditions for various Langevin dynamics.
The paper investigates how activation functions impact the training of Neural ODEs, leading to global convergence.
problem Challenges in training Neural ODEs, particularly gradient computation accuracy and convergence analysis.
method Investigates the impact of activation functions on the training dynamics of Neural ODEs.
result Establishes global convergence of Neural ODEs under gradient descent in overparameterized regimes.
Proposes a new loss function for robust learning.
problem Creating a robust loss function for machine learning.
method Extended pseudo Huber loss with log-exp transform and logistic function.
result Linear convergence algorithm for minimizer finding.
Novel algorithm accelerates PnP methods for image deblurring and super-resolution.
problem Efficiently solving inverse problems and imaging with provable convergence guarantees.
method Incorporates quasi-Newton steps into provable PnP framework based on proximal denoisers.
result 2--8x faster convergence compared to other provable PnP methods with similar quality.
New method guarantees global convergence in variational inference.
problem Limited convergence to local optima in variational inference.
method Minimizes inclusive KL divergence using neural networks and neural tangent kernel.
result Gradient descent dynamics converge to a unique solution in function space.