arXiv research

New bound relaxes uniform gradient norm assumptions for PAC-Bayesian bounds.

Paper relaxes stability and generalization assumptions for SGD.

Paper removes bounded gradient assumption for SGD in nonconvex learning.

This paper analyzes the convergence of Federated Average under relaxed assumptions.

Stochastic gradient descent (SGD) is the optimization algorithm of choice in many machine learning applications such as regularized empirical risk minimization and training deep neural networks. The classical convergence analysis of SGD is carried out under the assumption that the norm of the stochastic gradient is uni…

Proposes a new generalization bound for Bayesian deep nets without strict assumptions.

Adam converges to stationary points under relaxed conditions.

In this article, we prove gradient estimates under Bakry-Emery curvature bounds for unbounded graph Laplacians which satisfy an ellipticity assumption. As applications, we study completeness and finiteness of stochastically complete graphs under Bakry-Emery curvature bounds.

In the first part, we derive a sharp gradient estimate for the log of Dirichlet heat kernel and Poisson heat kernel on domains, and a sharpened local Li-Yau gradient estimate that matches the global one. In the second part, without explicit curvature assumptions, we prove a global upper bound for the fundamental soluti…

Study gradient flow of phase transitions with fixed contact angle.

Let $(M, g)$ be an $n$-dimensional complete Riemannian manifold with $Ric(M)\geq-(n-1)Q$, where $Q\geq0$ is a constant. We obtain an interior gradient bound for minimal graphs in $M\times R$ under some technical assumptions. For details, see Theorem 2.

We prove precompactness in an orbifold Cheeger-Gromov sense of complete gradient Ricci shrinkers with a lower bound on their entropy and a local integral Riemann bound. We do not need any pointwise curvature assumptions, volume or diameter bounds. In dimension four, under a technical assumption, we can replace the loca…

Paper provides variance bounds for variational inference.

In this paper we consider a star-shaped hypersurface flow by mean curvature. Without any assumption on the convexity, we give a new proof of gradient estimate for a short time. As an application, we also give a lower bound for the blowing up time.

Haslhofer and Müller proved a compactness Theorem for four-dimensional shrinking gradient Ricci solitons, with the only assumption being that the entropy is uniformly bounded from below. However, the limit in their result could possibly be an orbifold Ricci shrinker. In this paper we prove a compactness theorem for non…

New bounds for SGLD show error decreases with more data.

AdaGrad outperforms SGD in non-convex optimization problems by a factor of d.

Improved bounds for proximal gradient algorithms with computational errors.

Study submanifolds in gradient Ricci solitons with bounded curvature, proving volume growth properties.

New bounds for neural networks without loss boundedness assumption.

AdaGrad-Norm achieves optimal convergence rates for non-convex objectives without tuning.

The paper studies steady solitons with curvature decay and proves their smoothness.

New analysis improves sample complexity for vanilla policy gradient methods.

We show that if a closed hyperbolic 3-manifold has infinitely many finite covers of bounded Heegaard genus, then it is virtually fibered. This generalizes a theorem of Lackenby, removing restrictions needed about the regularity of the covers. Furthermore, we can replace the assumption that the covers have bounded Heega…

A new differentiable UCB algorithm for linear bandits learns adaptive confidence bounds.

SGD achieves a $O(ε^{-4})$ bound for minimizing gradient norm of smooth functions.

We derive lower bounds on the scalar curvature of complete non-compact gradient Yamabe solitons under some integral curvature conditions. Based on this, we prove that the corresponding potential functions have at most quadratic growth in distance. We also obtain a finite topological type property on complete shrinking …

Study large deviations rates for SGD with strongly convex functions.

Study on gradient descent in Hilbert spaces with Markov chains, focusing on mixing coefficients.

New privacy bounds for DP-SGD's last iterate, even with cyclic sampling.

Gradient estimate for harmonic functions with boundary condition proved.

Stochastic gradient methods can converge in expectation under heavy-tailed noise.

New shuffling methods improve convergence without Lipschitz smoothness.

New bounds show faster convergence for learning algorithms.

A new algorithm estimates sparse gradients on graphs with improved risk bounds.

The paper provides convergence guarantees for multicalibration gradient boosting.

New method bounds causal effects in continuous distributions.

For positive $p$-harmonic functions on Riemannian manifolds, we derive a gradient estimate and Harnack inequality with constants depending only on the lower bound of the Ricci curvature, the dimension $n$, $p$ and the radius of the ball on which the function is defined. Our approach is based on a careful application of…

SMG combines shuffling and momentum for non-convex optimization.

Gradient descent and SGD solve nonlinear inverse problems efficiently.

New guarantees for SGD in non-convex optimization without strict noise bounds.

The study proves rotationally symmetric property of certain shrinking gradient Yamabe solitons.

We establish a point-wise gradient estimate for $all$ positive solutions of the conjugate heat equation. This contrasts to Perelman's point-wise gradient estimate which works mainly for the fundamental solution rather than all solutions. Like Perelman's estimate, the most general form of our gradient estimate does not …

Study on Einstein solitons with bounds and asymptotic behavior.

A new algorithm improves convergence rates for convex optimization problems.

In this paper, we will prove a gap theorem for four-dimensional gradient shrinking soliton. More precisely, we will show that any complete four-dimensional gradient shrinking soliton with nonnegative and bounded Ricci curvature, satisfying a pinched Weyl curvature, either is flat, or $λ_1 + λ_2\ge c_0 R>0$ everywhere f…

New bounds show SGD can match deterministic gradient descent's convergence rate.

New stability bounds for GD in overparameterised shallow nets without NTK assumptions.