arXiv research

Study shows DNNs can recover functions with fewer samples than model parameters at overparameterization.

The paper improves conditions for unique recovery in homomorphic sensing of subspaces.

New tensor recovery method uses Riemannian optimization on Segre manifold.

We show that there are no spurious local minima in the non-convex factorized parametrization of low-rank matrix recovery from incoherent linear measurements. With noisy measurements we show all local minima are very close to a global optimum. Together with a curvature bound at saddle points, this yields a polynomial ti…

When the linear measurements of an instance of low-rank matrix recovery satisfy a restricted isometry property (RIP)---i.e. they are approximately norm-preserving---the problem is known to contain no spurious local minima, so exact recovery is guaranteed. In this paper, we show that moderate RIP is not enough to elimin…

The paper analyzes how good initial guesses affect the amount of data needed for low-rank matrix recovery.

In the context of sparse recovery, it is known that most of existing regularizers such as $\ell_1$ suffer from some bias incurred by some leading entries (in magnitude) of the associated vector. To neutralize this bias, we propose a class of models with partial regularizers for recovering a sparse solution of a linear …

The paper analyzes conditions for solving low-rank matrix recovery problems with noisy measurements.

Improved stability for matrix recovery from rank-one measurements.

We consider the dictionary learning problem, where the aim is to model the given data as a linear combination of a few columns of a matrix known as a dictionary, where the sparse weights forming the linear combination are known as coefficients. Since the dictionary and coefficients, parameterizing the linear model are …

We introduce the {\it diffusion $K$-means} clustering method on Riemannian submanifolds, which maximizes the within-cluster connectedness based on the diffusion distance. The diffusion $K$-means constructs a random walk on the similarity graph with vertices as data points randomly sampled on the manifolds and edges as …

This paper puts forth a novel bi-linear modeling framework for data recovery via manifold-learning and sparse-approximation arguments and considers its application to dynamic magnetic-resonance imaging (dMRI). Each temporal-domain MR image is viewed as a point that lies onto or close to a smooth manifold, and landmark …

Nonconvex matrix recovery is known to contain no spurious local minima under a restricted isometry property (RIP) with a sufficiently small RIP constant $δ$. If $δ$ is too large, however, then counterexamples containing spurious local minima are known to exist. In this paper, we introduce a proof technique that is capa…

This paper presents a unified geometric framework for the statistical analysis of a general ill-posed linear inverse model which includes as special cases noisy compressed sensing, sign vector recovery, trace regression, orthogonal matrix estimation, and noisy matrix completion. We propose computationally feasible conv…

Paper proposes CLAIR for efficient LLM fine-tuning across clients.

Constructing an efficient parameterization of a large, noisy data set of points lying close to a smooth manifold in high dimension remains a fundamental problem. One approach consists in recovering a local parameterization using the local tangent plane. Principal component analysis (PCA) is often the tool of choice, as…

In this paper, we generalize Huber's criterion to multichannel sparse recovery problem of complex-valued measurements where the objective is to find good recovery of jointly sparse unknown signal vectors from the given multiple measurement vectors which are different linear combinations of the same known elementary vec…

Study uses Bayes Hilbert framework to recover probability measure flows from sensors.

We study the community detection and recovery problem in partially-labeled stochastic block models (SBM). We develop a fast linearized message-passing algorithm to reconstruct labels for SBM (with $n$ nodes, $k$ blocks, $p,q$ intra and inter block connectivity) when $δ$ proportion of node labels are revealed. The signa…

Nonnegative low-rank matrix recovery can have spurious local minima.

Local approach learns causal structure of linear Gaussian polytree models from interventional data.

In this paper, we consider regression problems with one-hidden-layer neural networks (1NNs). We distill some properties of activation functions that lead to $\mathit{local~strong~convexity}$ in the neighborhood of the ground-truth parameters for the 1NN squared-loss objective. Most popular nonlinear activation function…

Study supports recovery of PDEs from noisy data using a specific regularization method.

New algorithm recovers model coefficients and supports from noisy data.

New method recovers graph latent positions under edge differential privacy.

We propose a flexible method for estimating value functions in reinforcement learning without parametric assumptions.

We propose a unified framework to solve general low-rank plus sparse matrix recovery problems based on matrix factorization, which covers a broad family of objective functions satisfying the restricted strong convexity and smoothness conditions. Based on projected gradient descent and the double thresholding operator, …

The paper tackles noisy combinations of continuous and step functions, providing conditions for their identification.

Efficient algorithms for sparse parameter recovery in mixture models.

Gradient method converges locally linearly for overparameterized Gaussian mixtures.

Guaranteed convergence for tensor factorization using Riemannian gradient descent.

This paper presents the first theoretical results showing that stable identification of overcomplete $μ$-coherent dictionaries $Φ\in \mathbb{R}^{d\times K}$ is locally possible from training signals with sparsity levels $S$ up to the order $O(μ^{-2})$ and signal to noise ratios up to $O(\sqrt{d})$. In particular the di…

Paper presents ABGD for efficient piecewise linear regression in high dimensions.

Random non-linear Fourier features have recently shown remarkable performance in a wide-range of regression and classification applications. Motivated by this success, this article focuses on a sparse non-linear Fourier feature (NFF) model. We provide a characterization of the sufficient number of data points that guar…

Proof of Gaussian ML estimator consistency in linear auto-regressive models.

We study the convergence of the Expectation-Maximization (EM) algorithm for mixtures of linear regressions with an arbitrary number $k$ of components. We show that as long as signal-to-noise ratio (SNR) is $\tildeΩ(k)$, well-initialized EM converges to the true regression parameters. Previous results for $k \geq 3$ hav…

We consider the problem of recovering a complete (i.e., square and invertible) matrix $\mathbf A_0$, from $\mathbf Y \in \mathbb{R}^{n \times p}$ with $\mathbf Y = \mathbf A_0 \mathbf X_0$, provided $\mathbf X_0$ is sufficiently sparse. This recovery problem is central to theoretical understanding of dictionary learnin…

This paper advances FL algorithms for composite optimization and statistical recovery.

In this paper, we consider parameter recovery for non-overlapping convolutional neural networks (CNNs) with multiple kernels. We show that when the inputs follow Gaussian distribution and the sample size is sufficiently large, the squared loss of such CNNs is $\mathit{~locally~strongly~convex}$ in a basin of attraction…

The paper develops a robust signal estimation method for noisy measurements from generative models.

Paper discusses new stochastic algorithms for sparse signal recovery.

This paper improves support recovery in universal one-bit compressed sensing.

Sign information is the key to overcoming the inevitable saturation error in compressive sensing systems, which causes information loss and results in bias. For sparse signal recovery from saturation, we propose to use a linear loss to improve the effectiveness from existing methods that utilize hard constraints/hinge …

We present a mathematical analysis of a non-convex energy landscape for robust subspace recovery. We prove that an underlying subspace is the only stationary point and local minimizer in a specified neighborhood under a deterministic condition on a dataset. If the deterministic condition is satisfied, we further show t…

FLOP algorithm speeds up causal structure learning for linear models.

New method improves learning from multiple correlated data trajectories.

Stochastic (sub)gradient methods require step size schedule tuning to perform well in practice. Classical tuning strategies decay the step size polynomially and lead to optimal sublinear rates on (strongly) convex problems. An alternative schedule, popular in nonconvex optimization, is called \emph{geometric step decay…

We reveal a model rank that predicts successful recovery of target functions at overparameterization.