arXiv research

New algorithms achieve uniform stability for empirical risk minimization.

We give a complete criterion for the existence of generalized Kähler Einstein metrics on toric Fano manifolds from view points of a uniform stability in a sense of GIT and the properness of a functional on the space of Kähler metrics.

The paper derives uniform stability-based coverage bounds for conformal prediction methods.

New method turns optimization algorithms into uniformly stable learning algorithms for non-Euclidean norms.

New stability bounds for SGD on nonsmooth convex losses.

We introduce uniform K-stability and its relationship with the coercivity property of the K-energy functional, for general polarized manifolds. Since the automorphism groups are not necessarily finite, size of the norm measuring uniformity should be reduced with respect to the group action. About this point we explain …

Accelerated gradient method's stability deteriorates exponentially with steps.

New bounds for SGLD show error decreases with more data.

New method improves generalization in deep learning models.

There is accumulating evidence in the literature that stability of learning algorithms is a key characteristic that permits a learning algorithm to generalize. Despite various insightful results in this direction, there seems to be an overlooked dichotomy in the type of stability-based generalization bounds we have in …

Algorithm identifies bilinear dynamical systems from noisy data.

Enhanced stability improves privacy in machine learning.

The paper solves a uniform Yau-Tian-Donaldson conjecture for toric manifolds.

New PAC bound for meta-learning improves generalization guarantees.

The paper establishes a correspondence for projective bundles over curves using test configurations and extremal metrics.

Paper explores generalization of AID-based bi-level optimization methods.

Paper analyzes stability and generalization of SCO algorithms.

The paper relaxes the stability condition to boost confidence in generalization for randomized learning algorithms.

Stability result for a popular algorithm in optimal transport.

We present a study of generalization for data-dependent hypothesis sets. We give a general learning guarantee for data-dependent hypothesis sets based on a notion of transductive Rademacher complexity. Our main result is a generalization bound for data-dependent hypothesis sets expressed in terms of a notion of hypothe…

Let $X$ be any $\mathbb{Q}$-Fano variety and $\mathrm{Aut}(X)_0$ be the identity component of the automorphism group of $X$. Let $\mathbb{G}$ be a connected reductive subgroup of $\mathrm{Aut}(X)_0$ that contains a maximal torus of $\mathrm{Aut}(X)_0$. We prove that $X$ admits a Kähler-Einstein metric if and only if $X…

GRAIN: Group Aggregation via Min-Norm Objective

Mabuchi solitons generalize Kähler-Einstein metrics on Fano manifolds, which constitute a Yau-Tian-Donaldson type correspondence with relative Ding stability. Comparing with Kähler-Ricci solitons, there is a distinct necessary condition for the existence. We show this condition can be implied by the uniformly relative …

The present paper provides a new generic strategy leading to non-asymptotic theoretical guarantees on the Leave-one-Out procedure applied to a broad class of learning algorithms. This strategy relies on two main ingredients: the new notion of $L^q$ stability, and the strong use of moment inequalities. $L^q$ stability e…

Decomposes J-energy into simpler intersection numbers for stability analysis.

Studied how heavy-tailed behavior affects SGD's generalization in quadratic optimization.

Paper confirms conjecture for projective manifolds in supercritical phase.

I prove the bistability of linear evolution equations $x' = A(t)x$ in a Banach space $E$, where the operator-valued function $A$ is of the form $A(t) = f'(t)G(t,f(t))$ for a binary operator-valued function $G$ and a scalar function $f$. The constant that bounds the solutions of the equation is computed explicitly; it i…

SGD-trained deep nets have bounds on their generalization error.

We prove effective uniformization for nearly round 2-spheres and investigate their stability.

For their ability to capture non-linearities in the data and to scale to large training sets, local Support Vector Machines (SVMs) have received a special attention during the past decade. In this paper, we introduce a new local SVM method, called L$^3$-SVMs, which clusters the input space, carries out dimensionality r…

Stochastic methods with coordinate-wise adaptive stepsize (such as RMSprop and Adam) have been widely used in training deep neural networks. Despite their fast convergence, they can generalize worse than stochastic gradient descent. In this paper, by revisiting the design of Adagrad, we propose to split the network par…

Autoencoder-based learning has emerged as a staple for disciplining representations in unsupervised and semi-supervised settings. This paper analyzes a framework for improving generalization in a purely supervised setting, where the target space is high-dimensional. We motivate and formalize the general framework of ta…

This paper proves a Nakai-Moishezon criterion for complex Hessian equations.

In this paper, we propose a novel uniform generalization bound on the time and inverse temperature for stochastic gradient Langevin dynamics (SGLD) in a non-convex setting. While previous works derive their generalization bounds by uniform stability, we use Rademacher complexity to make our generalization bound indepen…

In recent years, the crucial importance of metrics in machine learning algorithms has led to an increasing interest for optimizing distance and similarity functions. Most of the state of the art focus on learning Mahalanobis distances (requiring to fulfill a constraint of positive semi-definiteness) for use in a local …

This paper proposes a new optimization algorithm called Entropy-SGD for training deep neural networks that is motivated by the local geometry of the energy landscape. Local extrema with low generalization error have a large proportion of almost-zero eigenvalues in the Hessian with very few positive or negative eigenval…

Stochastic momentum methods have been widely adopted in training deep neural networks. However, their theoretical analysis of convergence of the training objective and the generalization error for prediction is still under-explored. This paper aims to bridge the gap between practice and theory by analyzing the stochast…

Leveraging algorithmic stability to derive sharp generalization bounds is a classic and powerful approach in learning theory. Since Vapnik and Chervonenkis [1974] first formalized the idea for analyzing SVMs, it has been utilized to study many fundamental learning algorithms (e.g., $k$-nearest neighbors [Rogers and Wag…

The paper explores the generalization of quantum neural networks using stability theory.

We study binary classification algorithms for which the prediction on any point is not too sensitive to individual examples in the dataset. Specifically, we consider the notions of uniform stability (Bousquet and Elisseeff, 2001) and prediction privacy (Dwork and Feldman, 2018). Previous work on these notions shows how…

We introduce a norm on the space of test configurations, which we call the minimum norm. We conjecture that uniform K-stability with respect to this norm is equivalent to the existence of a constant scalar curvature Kähler metric. This notion of uniform K-stability is analogous to coercivity of the Mabuchi functional. …

In this article, we derive concentration inequalities for the cross-validation estimate of the generalization error for stable predictors in the context of risk assessment. The notion of stability has been first introduced by \cite{DEWA79} and extended by \cite{KEA95}, \cite{BE01} and \cite{KUNIY02} to characterize cla…

This paper analyzes the stability and generalization of triplet learning algorithms.

New stability criteria for Fano varieties using generalized b-divisors.

Paper improves generalization bounds for noisy stochastic algorithms.

This paper analyzes stability and generalization of Markov chain stochastic gradient methods.

Uniform stability of a learning algorithm is a classical notion of algorithmic stability introduced to derive high-probability bounds on the generalization error (Bousquet and Elisseeff, 2002). Specifically, for a loss function with range bounded in $[0,1]$, the generalization error of a $γ$-uniformly stable learning a…