Study smooth convergence of metric flows from F \mathbb{F} F -limits.
problem Smooth convergence of F \mathbb{F} F -limit flows. method Extensively studied metric flows and F \mathbb{F} F -limits, showing smooth convergence at regular points. result Each regular point on the limit is a point of smooth convergence.
This paper improves convergence guarantees for SGD algorithms in non-convex smooth functions.
problem Theoretical convergence properties of SGD algorithms for non-convex smooth functions.
method Analysis of SGD algorithms with arbitrary data ordering for non-convex smooth functions.
result Enhanced convergence guarantees for incremental gradient and single shuffle SGD, improving the optimization term of convergence guarantee.
We establish a fundamental connection between smooth and polygonal knot energies, showing that the Minimum Distance Energy for polygons inscribed in a smooth knot converges to the Moebius Energy of the smooth knot as the polygons converge to the smooth knot. However, the polygons must converge in a ``nice'' way, and th…
New shuffling methods improve convergence without Lipschitz smoothness.
problem Lack of convergence guarantees for shuffling methods under non-Lipschitz conditions.
method Revisit shuffling methods, prove convergence under general bounded variance condition.
result Matched current best-known convergence rates without Lipschitz smoothness.
The study analyzes convergence rates for sparse pivotal estimators in high-dimensional regression.
problem Sparse pivotal estimation in high-dimensional regression problems.
method Theoretical analysis and comparison of non-smooth + non-smooth optimization problems, including smoothing techniques.
result Minimax sup-norm convergence rates for square-root Lasso-type estimators are derived.
The article calculates the F \mathbb{F} F -convergence rate for Ricci flows with closed and smooth tangent flows.
problem Analyzing the convergence rate of Ricci flows with specific tangent flows.
method Calculating the F \mathbb{F} F -convergence rate for Ricci flows with closed and smooth tangent flows. result A Ricci flow with closed and smooth tangent flow is ∣ log λ ∣ − θ |\log λ|^{-θ} ∣ log λ ∣ − θ close to its tangent flow in the F \mathbb{F} F -sense. Smooth convergence shown for curve diffusion flows.
problem Embeddedness and global existence of curves.
method Exponentially fast convergence established.
result Smooth convergence for curve diffusion flows.
Last iterate of Extragradient algorithm converges slower than averaged iterates in saddle point problems.
problem Smooth convex-concave saddle point problems
method Analysis of Extragradient (EG) algorithm convergence rates
result The last iterate of EG converges at a rate of O(1/√T), compared to O(1/T) for averaged iterates
Smooth DNNs mitigate the curse of dimensionality in uniform convergence for various regression tasks.
problem The curse of dimensionality in uniform convergence of ReLU networks.
method Analysis of smoothly activated deep neural networks (smooth DNNs), establishing pseudo-dimension bounds and non-asymptotic approximation guarantees.
result Smooth DNNs achieve non-asymptotic uniform convergence rates across multiple statistical contexts, mitigating the curse of dimensionality.
Stochastic gradient methods are dominant in nonconvex optimization especially for deep models but have low asymptotical convergence due to the fixed smoothness. To address this problem, we propose a simple yet effective method for improving stochastic gradient methods named predictive local smoothness (PLS). First, we …
OSGM uses online learning to adapt stepsize for faster convergence.
problem Improving convergence rates of first-order methods.
method OSGM combines online learning and feedback functions to adjust stepsize.
result OSGM achieves convergence rates asymptotically no worse than optimal.
Study revisits AdaGrad convergence with relaxed noise assumptions.
problem Non-convex smooth optimization problems with general noise.
method General noise model with function value gap and gradient magnitude control.
result Probabilistic convergence rate of ( ilde{\mathcal{O}}(1/\sqrt{T})) under general noise.
This paper analyzes the convergence of Federated Average under relaxed assumptions.
problem Lack of theoretical analysis for Federated Average under assumptions beyond smoothness.
method Relaxing assumptions of strong smoothness to semi-smoothness and semi-Lipschitz properties, and introducing a bound on the gradient.
result Provides a theoretical convergence study on Federated Learning under new assumptions.
New method for faster convergence in non-convex optimization with unbounded smoothness.
problem Finding first-order stationary points of non-convex functions with unbounded smoothness.
method Developed a stopped analysis technique to prove convergence rates for ( L 0 , L 1 ) (L_0,L_1) ( L 0 , L 1 ) -smooth functions. result Achieved O ( p o l y log ( T ) T ) \mathcal{O}(\frac{\mathrm{poly}\log(T)}{\sqrt{T}}) O ( T poly l o g ( T ) ) convergence rates without uniform noise bounds. A new algorithm speeds up sparse-penalized quantile regression solving non-convex penalties.
problem Sparse-penalized quantile regression with non-convex penalties.
method Single-loop smoothing ADMM (SIAD) algorithm for faster convergence.
result SIAD method outperforms existing approaches in solving sparse-penalized quantile regression.
New method improves convergence for smooth games.
problem Improving convergence for smooth games.
method Stochastic Hamiltonian Gradient Methods (SHGD).
result SHGD converges linearly to the neighbourhood of a stationary point.
We study discrete curvatures computed from nets of curvature lines on a given smooth surface, and prove their uniform convergence to smooth principal curvatures. We provide explicit error bounds, with constants depending only on properties of the smooth limit surface and the shape regularity of the discrete net.
Study analyzes label smoothing in deep learning optimization.
problem Understanding label smoothing's impact on deep learning optimization.
method Analysis of stochastic gradient descent with label smoothing for non-convex problems.
result Label smoothing can speed up convergence by reducing variance.
AdaGrad fails to adapt to Hölder-smoothness in composite optimization problems.
problem AdaGrad's convergence rate is suboptimal for composite objectives.
method Exhibited a simple one-dimensional convex problem to highlight AdaGrad's limitations.
result AdaGrad does not achieve the classical convergence rate for Hölder-smooth objectives.
Predictive models can be used on high-dimensional brain images for diagnosis of a clinical condition. Spatial regularization through structured sparsity offers new perspectives in this context and reduces the risk of overfitting the model while providing interpretable neuroimaging signatures by forcing the solution to …
FedProx algorithm improved for non-smooth and heterogeneous data.
problem Theoretical understanding of FedProx for non-convex federated optimization.
method Local dissimilarity invariant convergence theory through algorithmic stability.
result Convergence guarantees for non-smooth FL problems and minibatch size.
This work accelerates gradient descent with anytime convergence guarantees.
problem Improving the convergence rate of gradient descent methods.
method Proposes a stepsize schedule for gradient descent that achieves anytime convergence rates.
result Gradient descent can achieve convergence rates of O ( T − 1.119 ) O(T^{-1.119}) O ( T − 1.119 ) for any stopping time T T T . Improved convergence for Polyak steps with momentum in smooth convex optimization.
problem Optimizing smooth strongly convex functions with limited information.
method Polyak steps with momentum, derived for accelerated gradient method.
result Convergence guarantees for accelerated gradient method with Polyak steps and momentum.
New bounds for agnostic learning with average smoothness.
problem Distribution-free nonparametric regression with average smoothness.
method Distribution-free uniform convergence bounds and agnostic learning algorithm.
result Distribution-free uniform convergence bounds for average-smoothness classes in the agnostic setting.
Armijo line-search speeds up gradient descent for various functions.
problem Improving convergence rate of gradient descent.
method Applying Armijo line-search to adjust step-size in gradient descent.
result GD with Armijo line-search converges faster than GD with a fixed step-size.
New method improves simulation efficiency in high dimensions.
problem Efficiency in estimating functionals of conditional expectations in high dimensions.
method Kernel ridge regression exploiting smoothness of conditional expectation.
result Effective reduction of the curse of dimensionality, bridging convergence rates.
The (global) Lipschitz smoothness condition is crucial in establishing the convergence theory for most optimization methods. Unfortunately, most machine learning and signal processing problems are not Lipschitz smooth. This motivates us to generalize the concept of Lipschitz smoothness condition to the relative smoothn…
Gradient descent with logistic loss can interpolate deep networks with smoothed ReLU activations under certain conditions.
problem Conditions for gradient descent to drive logistic loss to zero in deep networks with smoothed ReLU activations.
method Gradient descent applied to fixed-width deep networks with smoothed ReLU approximations (e.g., Swish, Huberized ReLU).
result Gradient descent can drive logistic loss to zero under specific conditions, providing bounds on convergence rate.
New methods improve convergence in non-convex non-smooth learning problems.
problem Sparse learning from high-dimensional data with non-convex, non-smooth regularizers.
method Stochastic proximal gradient methods with arbitrary sampling.
result Independent sampling improves performance over uniform sampling.
The paper examines convergence of currents and forms under smooth diffeomorphisms.
problem Analyzing convergence of currents and forms under C 0 C^0 C 0 -limits of diffeomorphisms. method Geometric analysis, measure theory, homotopy theory.
result Pushforwards of rectifiable currents converge in the flat norm.
Establishes smooth Ricci flows from convex surfaces in 3D space.
problem Existence and uniqueness of Ricci flow starting from convex surfaces.
method Smooth Ricci flows starting from smooth convex surfaces.
result Uniform convergence of metrics to initial convex surface.
New algorithm for differentially private distributed optimization of smooth, non-convex problems.
problem No differentially private distributed method for smooth, non-convex optimization problems.
method Smoothed normalization integrated with an error-feedback mechanism.
result Achieves superior convergence rate and first differentially private distributed optimization algorithm with provable convergence guarantees.
Paper proposes an algorithm for sampling from complex mixture distributions without requiring smoothness.
problem Sampling from a mixture of weakly smooth potentials.
method Unadjusted Langevin algorithm with Euler discretization for a mixture of weakly smooth distributions.
result Convergence in Kullback-Leibler divergence and L β L_β L β -Wasserstein metric with polynomial dependence on dimension. The paper provides a uniform convergence bound for smooth calibration error and its relationship with functional gradient.
problem Limited theoretical understanding of learning algorithms achieving high accuracy and good calibration.
method Focuses on smooth calibration error, providing a uniform convergence bound and proving the relationship with functional gradient.
result Derives conditions for simultaneous classification and calibration guarantees in gradient boosting trees, kernel boosting, and neural networks.
New algorithm tackles nonconvex machine learning problems with adaptive normalization and independent sampling.
problem Nonconvex machine learning problems with generalized-smoothness.
method Adaptive gradient normalization, independent sampling, and gradient clipping.
result Achieves an O(ε^(-4)) sample complexity for fast convergence.
Given a sequence of properly embedded minimal surfaces in a 3 3 3 -manifold with local bounds on area and genus, we prove subsequential convergence, smooth away from a discrete set, to a smooth embedded limit surface, possibly with multiplicity, and we analyze what happens when one blows up the surfaces near a point where…
Study confirms conjecture about Kähler metrics on smooth minimal models.
problem Behavior of constant scalar curvature Kähler metrics on smooth minimal models.
method Analysis of metrics in a neighborhood of the canonical class.
result Convergence to singular Kähler Einstein metric in the canonical class.
Improved k-NN active learning with local smoothness assumption.
problem Active learning convergence rates under smoothness assumptions.
method Designing an active learning algorithm with better convergence rate using local smoothness assumption for k-NN.
result Better convergence rate than in passive learning.
New convergence guarantees for SGDA and SCO under expected co-coercivity.
problem Solving smooth games with stochastic gradient descent-ascent and consensus optimization.
method Introducing expected co-coercivity and proving convergence guarantees for SGDA and SCO.
result Linear convergence of SGDA and SCO to a neighborhood of the solution with constant step-size, and convergence to the exact solution with stepsize-switching rules.
Step decay schedules improve convergence in non-convex optimization.
problem Improving convergence in non-convex optimization problems.
method Analyzing convergence rates of step decay schedules in non-convex, convex, and strongly convex problems.
result Step decay schedules achieve O ( ln T / T ) \mathcal{O}(\ln T/\sqrt{T}) O ( ln T / T ) convergence rates in various optimization scenarios. Preserves scalar curvature bounds under weak convergence of 3-manifolds.
problem Preserving scalar curvature bounds under weak convergence of 3-manifolds.
method Comparison between μ-bubbles in M_k and M.
result Scalar curvature lower bounds are preserved under weak convergence.
We simplify deep learning convergence analysis using basic math.
problem Ensuring reliable convergence of optimization algorithms in deep networks.
method Elementary arguments and computations to estimate smoothness constants.
result Systematic computation of smoothness constants for first-order algorithms.
Paper proposes ZO-SMD for MERO, achieving optimal convergence rates.
problem Minimizing excess risk across all test distributions.
method Zeroth-order stochastic mirror descent algorithm for both smooth and non-smooth MERO.
result Converges at optimal rates of O ( 1 / t ) \mathcal{O}(1/\sqrt{t}) O ( 1/ t ) for estimates and optimization errors. This paper proposes a novel proximal-gradient algorithm for a decentralized optimization problem with a composite objective containing smooth and non-smooth terms. Specifically, the smooth and nonsmooth terms are dealt with by gradient and proximal updates, respectively. The proposed algorithm is closely related to a p…
The paper provides convergence guarantees for multicalibration gradient boosting.
problem Understanding the convergence properties of multicalibration gradient boosting.
method Computational guarantees for multicalibration gradient boosting algorithms, including adaptive variants.
result The magnitude of successive prediction updates decays at O ( 1 / T ) O(1/\sqrt{T}) O ( 1/ T ) , leading to convergence in empirical multicalibration error. Quantifies scalar curvature under C 0 C^0 C 0 convergence, proving a refined version in all dimensions.
problem Proving a refined quantitative bound for scalar curvature under C 0 C^0 C 0 convergence. method Established the refined quantitative bound in all dimensions using smoothing techniques.
result Established the refined quantitative bound for scalar curvature in all dimensions.
The paper studies the convergence of elastic flows of curves into manifolds, proving smooth convergence under certain conditions.
problem The convergence of elastic flows of curves into manifolds.
method Parabolic estimates and Lojasiewicz-Simon gradient inequality.
result Smooth convergence of the flow to critical points under specific conditions.
Paper provides GOT convergence guarantees for sub-gamma distributions and dependent samples.
problem Estimating GOT distance under general settings.
method Gaussian-smoothed optimal transport (GOT) framework, sub-gamma distributions, dependent samples, kernel MMD distances.
result Convergence guarantees for GOT distance under more general settings.