arXiv research

Improved Langevin algorithms with prior diffusion achieve dimension-independent convergence for non-log-concave distributions.

Robustly estimates mean in incomplete data with corrupted examples.

Deep nets learn structured densities without dimensionality issues.

We establish dimension-independent estimates related to heat operators e^{tL} on manifolds. We first develop a very general contractivity result for Markov kernels which can be applied to diffusion semigroups. Second, we develop estimates on the norm behavior of harmonic and non-negative subharmonic functions. We apply…

Improved subspace recovery algorithm with dimension-independent error and polynomial time.

Develops a computationally tractable high-dimensional differential privacy estimator.

We prove that for any isometric action of a group on a unit sphere of dimension larger than one, the quotient space has diameter zero or larger than a universal dimension-independent positive constant.

DP-GD achieves dimension-independent convergence for unconstrained private GLMs.

This article is an overview of supervised machine learning problems for regression and classification. Topics include: kernel methods, training by stochastic gradient descent, deep learning architecture, losses for classification, statistical learning theory, and dimension independent generalization bounds. Implicit re…

New SGD covering technique yields dimension-independent generalization bounds.

New method proves dimension-free convergence for ULD in KL divergence.

We investigate 1) the rate at which refined properties of the empirical risk---in particular, gradients---converge to their population counterparts in standard non-convex learning tasks, and 2) the consequences of this convergence for optimization. Our analysis follows the tradition of norm-based capacity control. We p…

New algorithm converges to equilibrium in nonconvex-nonconcave optimization problems without dimension dependence.

Note on subgaussian bounds for sign-quantized linear maps.

Study shows torsion homology growth vanishes for certain free-by-cyclic groups.

We show that the logarithmic derivatives of the convolution heat kernels on a uni-modular Lie group are exponentially integrable. This result is then used to prove an "integrated" Harnack inequality for these heat kernels. It is shown that this integrated Harnack inequality is equivalent to a version of Wang's Harnack …

Private learning can perform well in high dimensions, contrary to known results.

This paper proves an abstract theorem addressing in a unified manner two important problems in function approximation: avoiding curse of dimensionality and estimating the degree of approximation for out-of-sample extension in manifold learning. We consider an abstract (shallow) network that includes, for example, neura…

A new method for faster optimization in high dimensions.

This work studies an explicit embedding of the set of probability measures into a Hilbert space, defined using optimal transport maps from a reference probability density. This embedding linearizes to some extent the 2-Wasserstein space, and enables the direct use of generic supervised and unsupervised learning algorit…

In this paper we prove quantitative regularity results for stationary and minimizing extrinsic biharmonic maps. As an application, we determine sharp, dimension independent $L^p$ bounds for $\nabla^k f$ that do not require a small energy hypothesis. In particular, every minimizing biharmonic map is in $W^{4,p}$ for all…

New sparsification theorem for Gaussian processes reduces dimensionality.

We show that a closed, connected and orientable Riemannian manifold of dimension $d$ that admits a quasiregular mapping from $\mathbb R^d$ must have bounded cohomological dimension independent of the distortion of the map. The dimension of the degree $l$ de Rham cohomology of $M$ is bounded above by $\binom{d}{l}$. Thi…

We show that the error probability of reconstructing kernel matrices from Random Fourier Features for the Gaussian kernel function is at most $\mathcal{O}(R^{2/3} \exp(-D))$, where $D$ is the number of random features and $R$ is the diameter of the data domain. We also provide an information-theoretic method-independen…

The bias potential model explains how generative models can generalize or memorize samples.

We analyze the $K$-armed bandit problem where the reward for each arm is a noisy realization based on an observed context under mild nonparametric assumptions. We attain tight results for top-arm identification and a sublinear regret of $\widetilde{O}\Big(T^{\frac{1+D}{2+D}}\Big)$, where $D$ is the context dimension, f…

New DP optimization methods for sparse gradients, improving on existing algorithms.

Study reveals mutual information is crucial for understanding algorithm performance in stochastic convex optimization.

Polynomial-time tester-learner for general halfspaces with Gaussian adversarial noise.

Information concentration of probability measures have important implications in learning theory. Recently, it is discovered that the information content of a log-concave distribution concentrates around their differential entropy, albeit with an unpleasant dependence on the ambient dimension. In this work, we prove th…

Study shows how to approximate and estimate high-dimensional classification functions without the curse of dimensionality.

Universal preconditioning reduces sequential prediction regret.

Gradient descent solves robust mean estimation in high dimensions.

Zigzag sampling algorithm efficiently samples from strongly log-concave distributions with low computational cost.

Elliptical slice sampling converges geometrically, providing reliable sampling for Bayesian learning.

The paper shows neural networks can approximate functions over non-compact domains with non-polynomial activation.

Study on estimating distances between covariance operators and Gaussian processes.

We extend graph neural networks to transfer performance across different input sizes.

This work improves density estimation by characterizing pdf complexity using NL-spectrum.

New algorithm learns mixtures of any constant number of Gaussians robustly.

DPZero fine-tunes large models privately without backpropagation.

Learnable multiclass hypothesis classes don't always have a sample compression scheme of fixed size.

New method controls error in low-dimensional marginals of spatial models.

New algorithm learns Bayesian networks robustly in nearly-linear time.

Deep learning predicts dynamics from sparse data.

It is well known that isoperimetric inequalities imply in a very general measure-metric-space setting appropriate concentration inequalities. The former bound the boundary measure of sets as a function of their measure, whereas the latter bound the measure of sets separated from sets having half the total measure, as a…

We study high-dimensional distribution learning in an agnostic setting where an adversary is allowed to arbitrarily corrupt an $\varepsilon$-fraction of the samples. Such questions have a rich history spanning statistics, machine learning and theoretical computer science. Even in the most basic settings, the only known…

We investigate implicit regularization schemes for gradient descent methods applied to unpenalized least squares regression to solve the problem of reconstructing a sparse signal from an underdetermined system of linear measurements under the restricted isometry assumption. For a given parametrization yielding a non-co…