arXiv research

New techniques for compressive sensing without noise or signal statistics.

Deep learning improves solving medical imaging problems with sparse data.

Half-AVAE enhances VAE for underdetermined ICA with adversarial training.

Improved audio source separation for underdetermined conditions.

We study the implicit bias of generic optimization methods, such as mirror descent, natural gradient descent, and steepest descent with respect to different potentials and norms, when optimizing underdetermined linear regression or separable linear classification problems. We explore the question of whether the specifi…

Sparse linear regression, which entails finding a sparse solution to an underdetermined system of linear equations, can formally be expressed as an $l_0$-constrained least-squares problem. The Orthogonal Least-Squares (OLS) algorithm sequentially selects the features (i.e., columns of the coefficient matrix) to greedil…

We give sufficient conditions for some underdetermined elliptic PDE of any order to construct smooth compactly supported solutions. In particular we show that two smooth elements in the kernel of certain underdetermined linear elliptic operators $P$ can be glued in a chosen region in order to obtain a new smooth soluti…

The paper argues for prioritizing identifying structure over complex models for scientific discovery.

This paper tackles traffic volume estimation challenges with a deep learning method.

Hard Thresholding Pursuit (HTP) is an iterative greedy selection procedure for finding sparse solutions of underdetermined linear systems. This method has been shown to have strong theoretical guarantee and impressive numerical performance. In this paper, we generalize HTP from compressive sensing to a generic problem …

We calculate relations on characteristic classes which are obstructions preventing closed Kähler manifolds from carrying holomorphic Cartan geometries. We apply these relations to give global constraints on the phase spaces of complex analytic determined and underdetermined systems of differential equations.

GEMSS discovers multiple sparse solutions in high-dimensional data.

High signal to noise ratio (SNR) consistency of model selection criteria in linear regression models has attracted a lot of attention recently. However, most of the existing literature on high SNR consistency deals with model order selection. Further, the limited literature available on the high SNR consistency of subs…

The paper explores why a specific type of predictor works well in noisy data.

We study implicit regularization when optimizing an underdetermined quadratic objective over a matrix $X$ with gradient descent on a factorization of $X$. We conjecture and provide empirical and theoretical evidence that with small enough step sizes and initialization close enough to the origin, gradient descent on a f…

Paper proposes an algorithm for robust estimation using Huber's criterion.

A new NMF variant tackles underdetermined problems with sparse and separable assumptions.

New method constructs solution operators for PDEs with prescribed support properties.

The paper studies sparse solutions for underdetermined systems of linear equations.

New method identifies network structure without regularization for sparse teacher couplings.

This work proposes optimal decision rules for hierarchical classifiers to better align with evaluation metrics.

Double descent in condition number shows peak at equal dimensions.

A conventional wisdom in statistical learning is that large models require strong regularization to prevent overfitting. Here we show that this rule can be violated by linear regression in the underdetermined $n\ll p$ situation under realistic conditions. Using simulations and real-life high-dimensional data sets, we d…

Finding sparse solutions of underdetermined systems of linear equations is a fundamental problem in signal processing and statistics which has become a subject of interest in recent years. In general, these systems have infinitely many solutions. However, it may be shown that sufficiently sparse solutions may be identi…

A-DLISTA and VLISTA learn dictionaries and sparse representations under varying sensing matrices.

New tensor formulation reveals gradient flow's bias in linear neural networks.

When solving data analysis problems it is important to integrate prior knowledge and/or structural invariances. This paper contributes by a novel framework for incorporating algebraic invariance structure into kernels. In particular, we show that algebraic properties such as sign symmetries in data, phase independence,…

Gradient descent with specific initialization and step size achieves optimal sparse signal recovery.

Study compares L1 and VG sparsity priors in inverse problems.

Causal deep learning tackles causal inference using tensor factor analysis.

Our paper characterizes how ReLU affects GD's implicit bias in high-dimensional neural networks.

Muon replaces matrix gradient with polar factor, optimizing flat spectrum updates

We propose a Bayesian expectation-maximization (EM) algorithm for reconstructing Markov-tree sparse signals via belief propagation. The measurements follow an underdetermined linear model where the regression-coefficient vector is the sum of an unknown approximately sparse signal and a zero-mean white Gaussian noise wi…

In this work, we address the problem of solving a series of underdetermined linear inverse problems subject to a sparsity constraint. We generalize the spike-and-slab prior distribution to encode a priori correlation of the support of the solution in both space and time by imposing a transformed Gaussian process on the…

The performance of sparse signal recovery from noise corrupted, underdetermined measurements can be improved if both sparsity and correlation structure of signals are exploited. One typical correlation structure is the intra-block correlation in block sparse signals. To exploit this structure, a framework, called block…

Suppose that we observe noisy linear measurements of an unknown signal that can be modeled as the sum of two component signals, each of which arises from a nonlinear sub-manifold of a high dimensional ambient space. We introduce SPIN, a first order projected gradient method to recover the signal components. Despite the…

We give a geometric interpretation of all the $m$-th elliptic integrable systems associated to a $k'$-symmetric space $N=G/G_0$ (in the sense of C.L. Terng). It turns out that we have to introduce the integer $m_{k'}$ defined by m_{1}=0 and m_{k'}= [(k'+1)/2]. Then the general problem splits into three cases : the prim…

Sparse elasticity reconstruction from local displacements reduces error.

The traditional sparse modeling approach, when applied to inverse problems with large data such as images, essentially assumes a sparse model for small overlapping data patches. While producing state-of-the-art results, this methodology is suboptimal, as it does not attempt to model the entire global signal in any mean…

Unified geometric approach to image reconstruction from incomplete data.

In magnetoencephalography (MEG) the conventional approach to source reconstruction is to solve the underdetermined inverse problem independently over time and space. Here we present how the conventional approach can be extended by regularizing the solution in space and time by a Gaussian process (Gaussian random field)…

The goal of compressed sensing is to estimate a vector from an underdetermined system of noisy linear measurements, by making use of prior knowledge on the structure of vectors in the relevant domain. For almost all results in this literature, the structure is represented by sparsity in a well-chosen basis. We show how…

Fourier PCA is Principal Component Analysis of a matrix obtained from higher order derivatives of the logarithm of the Fourier transform of a distribution.We make this method algorithmic by developing a tensor decomposition method for a pair of tensors sharing the same vectors in rank-$1$ decompositions. Our main appli…

Proposes using equivariant generative models for compressed sensing with unknown orientations.

We use matricial free energy to regularize autoencoders, producing Gaussian-like codes.

A new method for reconstructing flows from perturbed distributions.

Recent results in Compressive Sensing have shown that, under certain conditions, the solution to an underdetermined system of linear equations with sparsity-based regularization can be accurately recovered by solving convex relaxations of the original problem. In this work, we present a novel primal-dual analysis on a …

GenMod uses generative models to approximate high-dimensional PDE solutions with limited evaluations.