arXiv research

AdaBoost's classifier and margins converge to a known value.

The stochastic gradient descent has been widely used for solving composite optimization problems in big data analyses. Many algorithms and convergence properties have been developed. The composite functions were convex primarily and gradually nonconvex composite functions have been adopted to obtain more desirable prop…

Cubic-regularized Newton's method (CR) is a popular algorithm that guarantees to produce a second-order stationary solution for solving nonconvex optimization problems. However, existing understandings of the convergence rate of CR are conditioned on special types of geometrical properties of the objective function. In…

SONATA algorithm converges to solutions of nonconvex smooth functions with KL property.

This work studies the location estimation problem for a mixture of two rotation invariant log-concave densities. We demonstrate that Least Squares EM, a variant of the EM algorithm, converges to the true location parameter from a randomly initialized point. We establish the explicit convergence rates and sample complex…

We study the convergence properties of the VR-PCA algorithm introduced by \cite{shamir2015stochastic} for fast computation of leading singular vectors. We prove several new results, including a formal analysis of a block version of the algorithm, and convergence from random initialization. We also make a few observatio…

PPGD solves nonconvex nonsmooth optimization problems without KL property.

We extend some properties of random walks on hyperbolic groups to random walks on convergence groups. In particular we prove that if a convergence group $G$ acts on a compact metrizable space $M$ with the convergence property then we can provide $G\cup M$ with a compact topology such that random walks on $G$ converge a…

The paper analyzes spectral properties of connection Laplacian on tori, proving convergence to real torus.

We prove a continuity property for ending invariants of convergent sequences of Kleinian surface groups. We also analyze the bounded curve sets of such groups and show that their projections to non-annular subsurfaces lie a bounded Hausdorff distance from geodesics joining the projections of the ending invariants.

In a recent paper Donaldson defines three operators on a space of Hermitian metrics on a complex projective manifold: $T, T_ν, T_K.$ Iterations of these operators converge to balanced metrics, and these themselves approximate constant scalar curvature metrics. In this paper we investigate the convergence properties of …

The paper investigates how activation functions impact the training of Neural ODEs, leading to global convergence.

This paper improves convergence guarantees for SGD algorithms in non-convex smooth functions.

We provide a general framework to study convergence properties of families of maps. For manifolds $M$ and $N$ where $M$ is equipped with a volume form $\mathcal{V}$ we consider families of maps in the collection $\{(φ, B) : B \subset M, φ:B \rightarrow N\text{ with both measurable}\}$ and we define a distance function …

Study on metric spaces with properties (ETR), (LBD) and their convergence.

We discuss the turnpike property for optimal investment and consumption problems. We find there exists a threshold value that determines the turnpike property for investment policy. The threshold value only depends on the Sharpe ratio, the riskless interest rate and the discount rate. We show that if utilities behave a…

Here we explore a variety of properties of intrinsic flat convergence. We introduce the sliced filling volume and interval sliced filling volume and explore the relationship between these notions, the tetrahedral property and the disappearance of points under intrinsic flat convergence. We prove two new Gromov-Hausdorf…

Study on Adam-family methods for nonsmooth optimization with convergence guarantees.

Studied SGD convergence under weak conditions.

SGD converges to an invariant distribution with sub-Gaussian or sub-exponential properties.

Gradient descent converges with arbitrary stepsize for separable data under Fenchel-Young losses.

We investigate a variety of stability properties of Haezendonck-Goovaerts premium principles on their natural domain, namely Orlicz spaces. We show that such principles always satisfy the Fatou property. This allows to establish a tractable dual representation without imposing any condition on the reference Orlicz func…

Paper analyzes convergence rates of SGD for non-convex functions under various assumptions.

Improved convergence and curvature estimate for parabolic Allen-Cahn equation.

The paper develops a uniform function estimator in RKHS for regression.

Geometric analysis improves convergence of variational inference.

In this paper we investigate the Hausdorff dimension of limit sets of Anosov representations. In this context we revisit and extend the framework of hyperconvex representations and establish a convergence property for them, analogue to a differentiability property. As an application of this convergence, we prove that t…

This paper develops tools for nonreversible MCMC with convergence guarantees.

Improved uniform convergence bound with fat-shattering dimension reduces sample complexity gap.

The rectified flow method is analyzed for its statistical properties.

We present the Tetrahedral Compactness Theorem which states that sequences of Riemannian manifolds with a uniform upper bound on volume and diameter that satisfy a uniform tetrahedral property have a subsequence which converges in the Gromov-Hausdorff sense to a countably $\mathcal{H}^m$ rectifiable metric space of the…

New method improves convergence of spatial filters in neural networks.

Convergence of the Kalman filter is best analyzed by studying the contraction of the Riccati map in the space of positive definite (covariance) matrices. In this paper, we explore how this contraction property relates to a more fundamental non-expansiveness property of filtering maps in the space of probability distrib…

The objective of this paper is to introduce the notion of generalized almost statistical (briefly, GAS) convergence of bounded real sequences, which generalizes the notion of almost convergence as well as statistical convergence of bounded real sequences. As a special kind of Banach limit functional, we also introduce …

Paper studies asymmetric matrix sensing, proving gradient descent converges to low-rank solutions.

New algorithms improve submodular minimization via DC programming.

Nearest neighbor methods are a popular class of nonparametric estimators with several desirable properties, such as adaptivity to different distance scales in different regions of space. Prior work on convergence rates for nearest neighbor classification has not fully reflected these subtle properties. We analyze the b…

This is an intuitive survey of extrinsic and intrinsic notions of convergence of manifolds complete with pictures of key examples and a discussion of the properties associated with each notion. We begin with a description of three extrinsic notions which have been applied to study sequences of submanifolds in Euclidean…

We study Frank-Wolfe methods for nonconvex stochastic and finite-sum optimization problems. Frank-Wolfe methods (in the convex case) have gained tremendous recent interest in machine learning and optimization communities due to their projection-free property and their ability to exploit structured constraints. However,…

The paper explains how ReLU nets converge globally in high dimensions without strict assumptions.

New ODE models show saddle-point optimization methods converge differently, with last-iterate convergence for OGDA.

The study provides statistical guarantees for Bayesian variational boosting.

Consider the problem of learning, from non-experimental data, the causal (Markov equivalence) structure of the true, unknown causal Bayesian network (CBN) on a given, fixed set of (categorical) variables. This learning problem is known to be so hard that there is no learning algorithm that converges to the truth for al…

Uniform counting formulas for orthogeodesics in Kleinian groups converge.

Final part of a series on nonlinear observers on Riemannian metrics, establishing conditions for convergence.

New algorithm solves $\ell_0$-norm constrained multilinear logistic regression for tensor data.

The paper analyzes the training dynamics of a transformer for next-token prediction.

This paper analyzes kNN convergence over feature transformations.