arXiv research

Improved audio source separation for underdetermined conditions.

Half-AVAE enhances VAE for underdetermined ICA with adversarial training.

We give sufficient conditions for some underdetermined elliptic PDE of any order to construct smooth compactly supported solutions. In particular we show that two smooth elements in the kernel of certain underdetermined linear elliptic operators $P$ can be glued in a chosen region in order to obtain a new smooth soluti…

Deep learning improves solving medical imaging problems with sparse data.

New method constructs solution operators for PDEs with prescribed support properties.

Double descent in condition number shows peak at equal dimensions.

New techniques for compressive sensing without noise or signal statistics.

The paper argues for prioritizing identifying structure over complex models for scientific discovery.

This paper tackles traffic volume estimation challenges with a deep learning method.

The paper studies sparse solutions for underdetermined systems of linear equations.

A-DLISTA and VLISTA learn dictionaries and sparse representations under varying sensing matrices.

We calculate relations on characteristic classes which are obstructions preventing closed Kähler manifolds from carrying holomorphic Cartan geometries. We apply these relations to give global constraints on the phase spaces of complex analytic determined and underdetermined systems of differential equations.

We study implicit regularization when optimizing an underdetermined quadratic objective over a matrix $X$ with gradient descent on a factorization of $X$. We conjecture and provide empirical and theoretical evidence that with small enough step sizes and initialization close enough to the origin, gradient descent on a f…

A new NMF variant tackles underdetermined problems with sparse and separable assumptions.

The traditional sparse modeling approach, when applied to inverse problems with large data such as images, essentially assumes a sparse model for small overlapping data patches. While producing state-of-the-art results, this methodology is suboptimal, as it does not attempt to model the entire global signal in any mean…

This work proposes optimal decision rules for hierarchical classifiers to better align with evaluation metrics.

High signal to noise ratio (SNR) consistency of model selection criteria in linear regression models has attracted a lot of attention recently. However, most of the existing literature on high SNR consistency deals with model order selection. Further, the limited literature available on the high SNR consistency of subs…

Fourier PCA is Principal Component Analysis of a matrix obtained from higher order derivatives of the logarithm of the Fourier transform of a distribution.We make this method algorithmic by developing a tensor decomposition method for a pair of tensors sharing the same vectors in rank-$1$ decompositions. Our main appli…

Finding sparse solutions of underdetermined systems of linear equations is a fundamental problem in signal processing and statistics which has become a subject of interest in recent years. In general, these systems have infinitely many solutions. However, it may be shown that sufficiently sparse solutions may be identi…

When solving data analysis problems it is important to integrate prior knowledge and/or structural invariances. This paper contributes by a novel framework for incorporating algebraic invariance structure into kernels. In particular, we show that algebraic properties such as sign symmetries in data, phase independence,…

We study the implicit bias of generic optimization methods, such as mirror descent, natural gradient descent, and steepest descent with respect to different potentials and norms, when optimizing underdetermined linear regression or separable linear classification problems. We explore the question of whether the specifi…

Study compares L1 and VG sparsity priors in inverse problems.

Recent results in Compressive Sensing have shown that, under certain conditions, the solution to an underdetermined system of linear equations with sparsity-based regularization can be accurately recovered by solving convex relaxations of the original problem. In this work, we present a novel primal-dual analysis on a …

Joint blind source separation (J-BSS) is an emerging data-driven technique for multi-set data-fusion. In this paper, J-BSS is addressed from a tensorial perspective. We show how, by using second-order multi-set statistics in J-BSS, a specific double coupled canonical polyadic decomposition (DC-CPD) problem can be formu…

The study identifies exceptions to fiber-preserving symmetry in ODEs and systems.

In sensing applications, sensors cannot always measure the latent quantity of interest at the required resolution, sometimes they can only acquire a blurred version of it due the sensor's transfer function. To recover latent signals when only noisy mixed measurements of the signal are available, we propose the Gaussian…

A conventional wisdom in statistical learning is that large models require strong regularization to prevent overfitting. Here we show that this rule can be violated by linear regression in the underdetermined $n\ll p$ situation under realistic conditions. Using simulations and real-life high-dimensional data sets, we d…

GEMSS discovers multiple sparse solutions in high-dimensional data.

Study optimal investment under imitation of decision-changing rates.

In this work, we address the problem of solving a series of underdetermined linear inverse problems subject to a sparsity constraint. We generalize the spike-and-slab prior distribution to encode a priori correlation of the support of the solution in both space and time by imposing a transformed Gaussian process on the…

The performance of sparse signal recovery from noise corrupted, underdetermined measurements can be improved if both sparsity and correlation structure of signals are exploited. One typical correlation structure is the intra-block correlation in block sparse signals. To exploit this structure, a framework, called block…

Paper identifies tensor ranks via prior predictive matching, solving system of equations.

Sparse linear regression, which entails finding a sparse solution to an underdetermined system of linear equations, can formally be expressed as an $l_0$-constrained least-squares problem. The Orthogonal Least-Squares (OLS) algorithm sequentially selects the features (i.e., columns of the coefficient matrix) to greedil…

Suppose that we observe noisy linear measurements of an unknown signal that can be modeled as the sum of two component signals, each of which arises from a nonlinear sub-manifold of a high dimensional ambient space. We introduce SPIN, a first order projected gradient method to recover the signal components. Despite the…

The non-negative solution to an underdetermined linear system can be uniquely recovered sometimes, even without imposing any additional sparsity constraints. In this paper, we derive conditions under which a unique non-negative solution for such a system can exist, based on the theory of polytopes. Furthermore, we deve…

Hard Thresholding Pursuit (HTP) is an iterative greedy selection procedure for finding sparse solutions of underdetermined linear systems. This method has been shown to have strong theoretical guarantee and impressive numerical performance. In this paper, we generalize HTP from compressive sensing to a generic problem …

A theoretical framework that supports automated construction of dynamic prime models purely from experimental time series data has been invented and developed, which can automatically generate (construct) data-driven models of any time series data in seconds. This has resulted in the formulation and formalisation of ne…

We give a geometric interpretation of all the $m$-th elliptic integrable systems associated to a $k'$-symmetric space $N=G/G_0$ (in the sense of C.L. Terng). It turns out that we have to introduce the integer $m_{k'}$ defined by m_{1}=0 and m_{k'}= [(k'+1)/2]. Then the general problem splits into three cases : the prim…

Sparse elasticity reconstruction from local displacements reduces error.

Two approaches improve conformal Bayes for label shift, one post-hoc and one in-training.

Unified geometric approach to image reconstruction from incomplete data.

In this paper we show that very large mixtures of Gaussians are efficiently learnable in high dimension. More precisely, we prove that a mixture with known identical covariance matrices whose number of components is a polynomial of any fixed degree in the dimension n is polynomially learnable as long as a certain non-d…

In magnetoencephalography (MEG) the conventional approach to source reconstruction is to solve the underdetermined inverse problem independently over time and space. Here we present how the conventional approach can be extended by regularizing the solution in space and time by a Gaussian process (Gaussian random field)…

Effective training of neural networks requires much data. In the low-data regime, parameters are underdetermined, and learnt networks generalise poorly. Data Augmentation alleviates this by using existing data more effectively. However standard data augmentation produces only limited plausible alternative data. Given t…

The goal of compressed sensing is to estimate a vector from an underdetermined system of noisy linear measurements, by making use of prior knowledge on the structure of vectors in the relevant domain. For almost all results in this literature, the structure is represented by sparsity in a well-chosen basis. We show how…

Proposes using equivariant generative models for compressed sensing with unknown orientations.

Muon replaces matrix gradient with polar factor, optimizing flat spectrum updates

We use matricial free energy to regularize autoencoders, producing Gaussian-like codes.