Study smooth convergence of metric flows from F \mathbb{F} F -limits.
problem Smooth convergence of F \mathbb{F} F -limit flows. method Extensively studied metric flows and F \mathbb{F} F -limits, showing smooth convergence at regular points. result Each regular point on the limit is a point of smooth convergence.
This paper improves convergence guarantees for SGD algorithms in non-convex smooth functions.
problem Theoretical convergence properties of SGD algorithms for non-convex smooth functions.
method Analysis of SGD algorithms with arbitrary data ordering for non-convex smooth functions.
result Enhanced convergence guarantees for incremental gradient and single shuffle SGD, improving the optimization term of convergence guarantee.
We establish a fundamental connection between smooth and polygonal knot energies, showing that the Minimum Distance Energy for polygons inscribed in a smooth knot converges to the Moebius Energy of the smooth knot as the polygons converge to the smooth knot. However, the polygons must converge in a ``nice'' way, and th…
New methods solve non-Lipschitz smooth problems with guaranteed convergence.
problem Non-Lipschitz smooth problems in machine learning and signal processing.
method Bregman-divergence based algorithms for relatively smooth problems.
result Guaranteed convergence to second-order stationary points for any relatively smooth problem.
Improves stochastic gradient methods for faster convergence.
problem Low asymptotic convergence of stochastic gradient methods in nonconvex optimization.
method Predictive Local Smoothness (PLS) method to adaptively adjust learning rates based on local smoothness predictions.
result New variants of SGD, AccSGD, and AMSGrad achieve faster linear convergence.
New shuffling methods improve convergence without Lipschitz smoothness.
problem Lack of convergence guarantees for shuffling methods under non-Lipschitz conditions.
method Revisit shuffling methods, prove convergence under general bounded variance condition.
result Matched current best-known convergence rates without Lipschitz smoothness.
The study analyzes convergence rates for sparse pivotal estimators in high-dimensional regression.
problem Sparse pivotal estimation in high-dimensional regression problems.
method Theoretical analysis and comparison of non-smooth + non-smooth optimization problems, including smoothing techniques.
result Minimax sup-norm convergence rates for square-root Lasso-type estimators are derived.
The article calculates the F \mathbb{F} F -convergence rate for Ricci flows with closed and smooth tangent flows.
problem Analyzing the convergence rate of Ricci flows with specific tangent flows.
method Calculating the F \mathbb{F} F -convergence rate for Ricci flows with closed and smooth tangent flows. result A Ricci flow with closed and smooth tangent flow is ∣ log λ ∣ − θ |\log λ|^{-θ} ∣ log λ ∣ − θ close to its tangent flow in the F \mathbb{F} F -sense. Last iterate of Extragradient algorithm converges slower than averaged iterates in saddle point problems.
problem Smooth convex-concave saddle point problems
method Analysis of Extragradient (EG) algorithm convergence rates
result The last iterate of EG converges at a rate of O(1/√T), compared to O(1/T) for averaged iterates
Smooth convergence shown for curve diffusion flows.
problem Embeddedness and global existence of curves.
method Exponentially fast convergence established.
result Smooth convergence for curve diffusion flows.
Smooth DNNs mitigate the curse of dimensionality in uniform convergence for various regression tasks.
problem The curse of dimensionality in uniform convergence of ReLU networks.
method Analysis of smoothly activated deep neural networks (smooth DNNs), establishing pseudo-dimension bounds and non-asymptotic approximation guarantees.
result Smooth DNNs achieve non-asymptotic uniform convergence rates across multiple statistical contexts, mitigating the curse of dimensionality.
OSGM uses online learning to adapt stepsize for faster convergence.
problem Improving convergence rates of first-order methods.
method OSGM combines online learning and feedback functions to adjust stepsize.
result OSGM achieves convergence rates asymptotically no worse than optimal.
Study revisits AdaGrad convergence with relaxed noise assumptions.
problem Non-convex smooth optimization problems with general noise.
method General noise model with function value gap and gradient magnitude control.
result Probabilistic convergence rate of ( ilde{\mathcal{O}}(1/\sqrt{T})) under general noise.
This paper analyzes the convergence of Federated Average under relaxed assumptions.
problem Lack of theoretical analysis for Federated Average under assumptions beyond smoothness.
method Relaxing assumptions of strong smoothness to semi-smoothness and semi-Lipschitz properties, and introducing a bound on the gradient.
result Provides a theoretical convergence study on Federated Learning under new assumptions.
New method for faster convergence in non-convex optimization with unbounded smoothness.
problem Finding first-order stationary points of non-convex functions with unbounded smoothness.
method Developed a stopped analysis technique to prove convergence rates for ( L 0 , L 1 ) (L_0,L_1) ( L 0 , L 1 ) -smooth functions. result Achieved O ( p o l y log ( T ) T ) \mathcal{O}(\frac{\mathrm{poly}\log(T)}{\sqrt{T}}) O ( T poly l o g ( T ) ) convergence rates without uniform noise bounds. A new algorithm speeds up sparse-penalized quantile regression solving non-convex penalties.
problem Sparse-penalized quantile regression with non-convex penalties.
method Single-loop smoothing ADMM (SIAD) algorithm for faster convergence.
result SIAD method outperforms existing approaches in solving sparse-penalized quantile regression.
New method improves convergence for smooth games.
problem Improving convergence for smooth games.
method Stochastic Hamiltonian Gradient Methods (SHGD).
result SHGD converges linearly to the neighbourhood of a stationary point.
SGD without replacement converges faster for smooth convex functions.
problem Improving convergence rate of SGD for smooth convex functions.
method Using method of exchangeable pairs to bound Wasserstein distance, we provide non-asymptotic results for SGD without replacement.
result SGD without replacement converges at a rate of O(1/K^2) for general smooth, strongly-convex functions.
We study discrete curvatures computed from nets of curvature lines on a given smooth surface, and prove their uniform convergence to smooth principal curvatures. We provide explicit error bounds, with constants depending only on properties of the smooth limit surface and the shape regularity of the discrete net.
Novel active learning algorithm with improved convergence rate under local smoothness condition.
problem Improving convergence rates in active learning under specific smoothness assumptions.
method Developed a novel active learning algorithm with a rate of convergence better than in passive learning, using a local smoothness assumption for k-nearest neighbors.
result The algorithm achieves a better convergence rate than passive learning algorithms, avoiding strong density assumptions.
Paper proves smoothness for variational inference, giving convergence guarantees.
problem Proving convergence guarantees for black-box variational inference.
method Describes gradients in an inner-product space, using Bessel's inequality.
result Objective is M-Lipschitz smooth if target is, excluding entropy.
Study analyzes label smoothing in deep learning optimization.
problem Understanding label smoothing's impact on deep learning optimization.
method Analysis of stochastic gradient descent with label smoothing for non-convex problems.
result Label smoothing can speed up convergence by reducing variance.
Predictive models can be used on high-dimensional brain images for diagnosis of a clinical condition. Spatial regularization through structured sparsity offers new perspectives in this context and reduces the risk of overfitting the model while providing interpretable neuroimaging signatures by forcing the solution to …
AdaGrad fails to adapt to Hölder-smoothness in composite optimization problems.
problem AdaGrad's convergence rate is suboptimal for composite objectives.
method Exhibited a simple one-dimensional convex problem to highlight AdaGrad's limitations.
result AdaGrad does not achieve the classical convergence rate for Hölder-smooth objectives.
FedProx algorithm improved for non-smooth and heterogeneous data.
problem Theoretical understanding of FedProx for non-convex federated optimization.
method Local dissimilarity invariant convergence theory through algorithmic stability.
result Convergence guarantees for non-smooth FL problems and minibatch size.
This work accelerates gradient descent with anytime convergence guarantees.
problem Improving the convergence rate of gradient descent methods.
method Proposes a stepsize schedule for gradient descent that achieves anytime convergence rates.
result Gradient descent can achieve convergence rates of O ( T − 1.119 ) O(T^{-1.119}) O ( T − 1.119 ) for any stopping time T T T . Improved convergence for Polyak steps with momentum in smooth convex optimization.
problem Optimizing smooth strongly convex functions with limited information.
method Polyak steps with momentum, derived for accelerated gradient method.
result Convergence guarantees for accelerated gradient method with Polyak steps and momentum.
New bounds for agnostic learning with average smoothness.
problem Distribution-free nonparametric regression with average smoothness.
method Distribution-free uniform convergence bounds and agnostic learning algorithm.
result Distribution-free uniform convergence bounds for average-smoothness classes in the agnostic setting.
The paper studies convergence of discrete harmonic maps to smooth ones.
problem Discretization of harmonic maps between Riemannian manifolds.
method Introducing triangulations with vertex and edge weights, and studying convergence conditions.
result Suitable conditions on weighted triangulations ensure convergence of discrete harmonic maps to smooth ones.
Armijo line-search speeds up gradient descent for various functions.
problem Improving convergence rate of gradient descent.
method Applying Armijo line-search to adjust step-size in gradient descent.
result GD with Armijo line-search converges faster than GD with a fixed step-size.
Optimizes deep learning pipelines with novel algorithms for smooth and non-smooth functions.
problem Optimizing deep learning pipelines for smooth and non-smooth functions.
method Provided matching lower and upper bounds for smooth convex and non-convex functions, and developed PPRS for non-smooth convex functions.
result PPRS achieves near-linear speed-up and convergence time for non-smooth non-convex problems.
New method improves simulation efficiency in high dimensions.
problem Efficiency in estimating functionals of conditional expectations in high dimensions.
method Kernel ridge regression exploiting smoothness of conditional expectation.
result Effective reduction of the curse of dimensionality, bridging convergence rates.
Gradient descent with logistic loss can interpolate deep networks with smoothed ReLU activations under certain conditions.
problem Conditions for gradient descent to drive logistic loss to zero in deep networks with smoothed ReLU activations.
method Gradient descent applied to fixed-width deep networks with smoothed ReLU approximations (e.g., Swish, Huberized ReLU).
result Gradient descent can drive logistic loss to zero under specific conditions, providing bounds on convergence rate.
New methods improve convergence in non-convex non-smooth learning problems.
problem Sparse learning from high-dimensional data with non-convex, non-smooth regularizers.
method Stochastic proximal gradient methods with arbitrary sampling.
result Independent sampling improves performance over uniform sampling.
The paper examines convergence of currents and forms under smooth diffeomorphisms.
problem Analyzing convergence of currents and forms under C 0 C^0 C 0 -limits of diffeomorphisms. method Geometric analysis, measure theory, homotopy theory.
result Pushforwards of rectifiable currents converge in the flat norm.
This paper improves stochastic approximation for smooth and strongly convex functions.
problem Improving convergence rate of stochastic approximation for smooth and strongly convex functions.
method Utilizes both smoothness and strong convexity conditions to achieve faster convergence rates.
result Demonstrates an O ( 1 / [ λ T α ] + κ F ∗ / T ) O(1/[λT^α] + κF_*/T) O ( 1/ [ λ T α ] + κ F ∗ / T ) risk bound, potentially faster than O ( 1 / [ λ T ] ) O(1/[λT]) O ( 1/ [ λ T ]) . A new decentralized optimization method with independent step-sizes and separated convergence rates.
problem Decentralized optimization with composite objective terms.
method Proximal-gradient algorithm with uncoordinated step-sizes and separated convergence rates.
result Linear convergence for special case without non-smooth terms under strong convexity.
The paper analyzes the convergence rates of smooth message passing algorithms in entropy-regularized MAP inference.
problem Finding the most likely configuration in graphical models with combinatorial optimization.
method Entropy-regularized linear programming relaxations and smooth message passing algorithms.
result The number of iterations sufficient to recover the true integral MAP solution is determined.
Establishes smooth Ricci flows from convex surfaces in 3D space.
problem Existence and uniqueness of Ricci flow starting from convex surfaces.
method Smooth Ricci flows starting from smooth convex surfaces.
result Uniform convergence of metrics to initial convex surface.
New algorithm for differentially private distributed optimization of smooth, non-convex problems.
problem No differentially private distributed method for smooth, non-convex optimization problems.
method Smoothed normalization integrated with an error-feedback mechanism.
result Achieves superior convergence rate and first differentially private distributed optimization algorithm with provable convergence guarantees.
Paper proposes an algorithm for sampling from complex mixture distributions without requiring smoothness.
problem Sampling from a mixture of weakly smooth potentials.
method Unadjusted Langevin algorithm with Euler discretization for a mixture of weakly smooth distributions.
result Convergence in Kullback-Leibler divergence and L β L_β L β -Wasserstein metric with polynomial dependence on dimension. Alternative proof for convergence rate of three operator splitting scheme.
problem Optimizing composite functions with specific operator properties.
method Three operator splitting scheme for optimizing composite functions.
result Sublinear rate of convergence proved for the method.
The paper provides a uniform convergence bound for smooth calibration error and its relationship with functional gradient.
problem Limited theoretical understanding of learning algorithms achieving high accuracy and good calibration.
method Focuses on smooth calibration error, providing a uniform convergence bound and proving the relationship with functional gradient.
result Derives conditions for simultaneous classification and calibration guarantees in gradient boosting trees, kernel boosting, and neural networks.
New method improves convergence rates for stochastic convex optimization.
problem Convergence guarantees for online learning algorithms.
method Black-box modification to online learning algorithms, combined with optimistic and adaptive techniques.
result Achieves optimal accelerated rates of O ( L / T 2 + σ / T ) O(L/T^2 + σ/\sqrt{T}) O ( L / T 2 + σ / T ) . New algorithm tackles nonconvex machine learning problems with adaptive normalization and independent sampling.
problem Nonconvex machine learning problems with generalized-smoothness.
method Adaptive gradient normalization, independent sampling, and gradient clipping.
result Achieves an O(ε^(-4)) sample complexity for fast convergence.
Given a sequence of properly embedded minimal surfaces in a 3 3 3 -manifold with local bounds on area and genus, we prove subsequential convergence, smooth away from a discrete set, to a smooth embedded limit surface, possibly with multiplicity, and we analyze what happens when one blows up the surfaces near a point where…
Improved k-NN active learning with local smoothness assumption.
problem Active learning convergence rates under smoothness assumptions.
method Designing an active learning algorithm with better convergence rate using local smoothness assumption for k-NN.
result Better convergence rate than in passive learning.
Study confirms conjecture about Kähler metrics on smooth minimal models.
problem Behavior of constant scalar curvature Kähler metrics on smooth minimal models.
method Analysis of metrics in a neighborhood of the canonical class.
result Convergence to singular Kähler Einstein metric in the canonical class.