arXiv research

Unified framework for isotropic SG noise in posterior sampling.

Stochastic Gradient Langevin Dynamics infuses isotropic gradient noise to SGD to help navigate pathological curvature in the loss landscape for deep networks. Isotropic nature of the noise leads to poor scaling, and adaptive methods based on higher order curvature information such as Fisher Scoring have been proposed t…

We develop a flexible framework for low-rank matrix estimation that allows us to transform noise models into regularization schemes via a simple bootstrap algorithm. Effectively, our procedure seeks an autoencoding basis for the observed matrix that is stable with respect to the specified noise model; we call the resul…

New blurring diffusion models bridge heat dissipation and denoising.

AniDS improves molecular force field modeling by learning anisotropic noise.

This paper proposes an exploration method for deep reinforcement learning based on parameter space noise. Recent studies have experimentally shown that parameter space noise results in better exploration than the commonly used action space noise. Previous methods devised a way to update the diagonal covariance matrix o…

This work obtains novel finite sample guarantees for Principal Component Analysis (PCA). These hold even when the corrupting noise is non-isotropic, and a part (or all of it) is data-dependent. Because of the latter, in general, the noise and the true data are correlated. The results in this work are a significant impr…

Private adaptive methods improve on traditional SGD for convex optimization.

We introduce a new approach for designing computationally efficient learning algorithms that are tolerant to noise, and demonstrate its effectiveness by designing algorithms with improved noise tolerance guarantees for learning linear separators. We consider both the malicious noise model and the adversarial label nois…

Understanding the behavior of stochastic gradient descent (SGD) in the context of deep neural networks has raised lots of concerns recently. Along this line, we study a general form of gradient based optimization dynamics with unbiased noise, which unifies SGD and standard Langevin dynamics. Through investigating this …

Large-batch stochastic gradient descent (SGD) is widely used for training in distributed deep learning because of its training-time efficiency, however, extremely large-batch SGD leads to poor generalization and easily converges to sharp minima, which prevents naive large-scale data-parallel SGD (DP-SGD) from convergin…

New method improves representation learning from non-i.i.d. and non-isotropic data.

We analyze the variance of stochastic gradients along negative curvature directions in certain non-convex machine learning models and show that stochastic gradients exhibit a strong component along these directions. Furthermore, we show that - contrary to the case of isotropic noise - this variance is proportional to t…

Estimates shared linear subspace from noisy data with multiple users.

Optimizes SGLD noise structure for better generalization bounds.

SGD-trained neural networks generalize well even with adversarial label noise.

Strongly polynomial algorithm for approximate Forster transforms and halfspace learning.

Noise in linear networks minimizes sharpness and leads to shrinkage-thresholding.

Simple model finds high correlation in retail crypto returns.

Optimal ridge regularization computed iteratively from generative parameters.

Stochastic gradient descent (SGD) forms the core optimization method for deep neural networks. While some theoretical progress has been made, it still remains unclear why SGD leads the learning dynamics in overparameterized networks to solutions that generalize well. Here we show that for overparameterized networks wit…

The paper examines Randers metrics with isotropic scalar curvature properties.

WS diffusion models handle anisotropic Gaussian noise better than conventional methods.

Paper establishes a relation between Berwald scalar curvature and S-curvature.

Improved sample complexity for learning halfspaces with malicious noise.

In this paper, we find a condition on $(α, β)$-metrics under which the notions of isotropic S-curvature, weakly isotropic S-curvature and isotropic mean Berwald curvature are equivalent.

Develops structured noise for more accurate graph classifier robustness certificates.

Study physical work done by isotropic vector forces along isotropic curves.

We interpret the variational inference of the Stochastic Gradient Descent (SGD) as minimizing a new potential function named the \textit{quasi-potential}. We analytically construct the quasi-potential function in the case when the loss function is convex and admits only one global minimum point. We show in this case th…

Study isotropic Riemannian maps and helices along them.

We study two types of isotropic planes: weakly isotropic and strongly isotropic planes. We prove that a Riemannian manifold of indefinite metric is conformally flat if and only if its curvature tensor vanishes on all the strongly isotropic planes. We specialize the plane axiom for Riemannian manifolds of indefinite met…

Efficiently learns halfspaces with malicious noise, near-optimal label complexity.

Study isotropic curves on complex quadric with geometric relations.

In this paper, we construct a new class of Finsler manifolds called generalized isotropic Berwald manifolds which is an extension of the class of isotropic Berwald manifolds. We prove that every generalized isotropic Berwald manifold is a generalized Douglas-Weyl manifold. On a compact generalized isotropic Berwald man…

Constructs a moment map flow for isotropic maps on surfaces.

Spinor representation in isotropic space via Laguerre geometry.

The paper studies hanging chains and surfaces in degenerate geometries.

We create real-time geodesic rendering for non-isotropic geometries.

Developed a new concept of isometric surfaces in isotropic space.

Paper shows isotropic $E$- and $S$-curvatures are equivalent in warped Finsler metrics.

In this paper we will show that a Lagrangian, Lorentzian surface $M^2_1$ in a complex pseudo space form $\widetilde M^2_1 (4c)$ is pseudo-isotropic if and only if $M$ is minimal. Next we will obtain a complete classification of all Lagrangian, Lorentzian surfaces which are lightlike pseudo-isotropic but not pseudo-isot…

The paper studies Kropina metrics with a specific curvature property.

New approach uses isotropic geometry to solve Euclidean problems.

We show that for $m>n\geq 2$, there are at least two exact isotropic $n$-tori in $\mathbb{C}^m$ which are not Hamiltonian isotopic in $\mathbb{C}^m$, even though they are smoothly isotopic as isotropic $n$-tori. We apply this discovery to obtain more distinct non-exact isotropic tori in $\mathbb{C}^m$.

Study classifies zero mean curvature surfaces with planar curvature lines.

The study of Laguerre isotropic hypersurfaces with rigidity and isoparametric properties.

This paper aims to provide a description of totally isotropic Willmore two-spheres and their adjoint transforms. We first recall the isotropic harmonic maps which are introduced by Hélein, Xia-Shen and Ma for the study of Willmore surfaces. Then we derive a description of the normalized potential (some Lie algebra valu…

Study surfaces with constant ratio of principal curvatures in Euclidean and isotropic geometries.