arXiv research

In compressed sensing, we wish to reconstruct a sparse signal $x$ from observed data $y$. In sparse coding, on the other hand, we wish to find a representation of an observed signal $y$ as a sparse linear combination, with coefficients $x$, of elements from an overcomplete dictionary. While many algorithms are competit…

Sparse neural encoding can store more memories as targets become sparser.

GrateTile optimizes CNN feature map storage for efficient data access.

Random projections help in representing sparse graphs efficiently.

New model estimates sparse transport maps for high-dimensional data.

Attention mechanisms and non-local mean operations in general are key ingredients in many state-of-the-art deep learning techniques. In particular, the Transformer model based on multi-head self-attention has recently achieved great success in natural language processing and computer vision. However, the vanilla algori…

AEN-SAEs address feature starvation in sparse autoencoders by stabilizing the geometric alignment of sparse coding.

We consider a Bayesian framework for estimating a high-dimensional sparse precision matrix, in which adaptive shrinkage and sparsity are induced by a mixture of Laplace priors. Besides discussing our formulation from the Bayesian standpoint, we investigate the MAP (maximum a posteriori) estimator from a penalized likel…

DistGP models multi-robot mapping with distributed Gaussian process learning.

PixL2R maps natural language to pixel-based rewards for RL, improving sample efficiency.

Functional neuroimaging can measure the brain?s response to an external stimulus. It is used to perform brain mapping: identifying from these observations the brain regions involved. This problem can be cast into a linear supervised learning task where the neuroimaging data are used as predictors for the stimulus. Brai…

Comprehending complex systems by simplifying and highlighting important dynamical patterns requires modeling and mapping higher-order network flows. However, complex systems come in many forms and demand a range of representations, including memory and multilayer networks, which in turn call for versatile community-det…

While matrix factorisation models are ubiquitous in large scale recommendation and search, real time application of such models requires inner product computations over an intractably large set of item factors. In this manuscript we present a novel framework that uses the inverted index representation to exploit struct…

Recovering sparse conditional independence graphs from data is a fundamental problem in machine learning with wide applications. A popular formulation of the problem is an $\ell_1$ regularized maximum likelihood estimation. Many convex optimization algorithms have been designed to solve this formulation to recover the …

Structured prediction requires searching over a combinatorial number of structures. To tackle it, we introduce SparseMAP: a new method for sparse structured inference, and its natural loss function. SparseMAP automatically selects only a few global structures: it is situated between MAP inference, which picks a single …

New method uses neural maps to efficiently sample lattice QCD distributions.

New entropy-based objective for sparse coding improves learning.

We develop a 2D travel time tomography method which regularizes the inversion by modeling groups of slowness pixels from discrete slowness maps, called patches, as sparse linear combinations of atoms from a dictionary. We propose to use dictionary learning during the inversion to adapt dictionaries to specific slowness…

A new model improves CT image quality from low-dose scans.

Sparse PCA selects variables with FDR control for improved performance.

New method discovers causal relationships in sparse linear data.

Graph neural networks improve AMG convergence for sparse systems.

Adaptive sparseness enhances robust regression using MCC and ARD.

Continuous-time mirror descent solves sparse phase retrieval efficiently.

Many data-driven approaches exist to extract neural representations of functional magnetic resonance imaging (fMRI) data, but most of them lack a proper probabilistic formulation. We propose a group level scalable probabilistic sparse factor analysis (psFA) allowing spatially sparse maps, component pruning using automa…

Proposes EM for sparse horseshoe estimation.

Transformers exhibit sparse activation maps, reducing computational load and improving robustness.

Bayesian optimization with RPCE reduces MAP estimation for structural dynamics models.

The paper explores graphons of line graphs from sparse finite graphs.

Sparse neural networks can match dense models on Lipschitz functions.

ERM performs well in feature learning with minimal feature maps.

This paper begins with considering the identification of sparse linear time-invariant networks described by multivariable ARX models. Such models possess relatively simple structure thus used as a benchmark to promote further research. With identifiability of the network guaranteed, this paper presents an identificatio…

We propose to align distributional data from the perspective of Wasserstein means. We raise the problem of regularizing Wasserstein means and propose several terms tailored to tackle different problems. Our formulation is based on the variational transportation to distribute a sparse discrete measure into the target do…

Classifiers based on sparse representations have recently been shown to provide excellent results in many visual recognition and classification tasks. However, the high cost of computing sparse representations at test time is a major obstacle that limits the applicability of these methods in large-scale problems, or in…

Efficiently prices American options with multiple assets using sparse grids.

A fast method combines deep mixtures of sparse GPs for flexible modeling.

New method discovers concepts in hidden feature layers using sparse subspace clustering.

Olshausen and Field (OF) proposed that neural computations in the primary visual cortex (V1) can be partially modeled by sparse dictionary learning. By minimizing the regularized representation error they derived an online algorithm, which learns Gabor-filter receptive fields from a natural image ensemble in agreement …

Big T-Rex solves FDR-controlled sparse regression on laptops with millions of variables.

Tree-based regularization improves latent variable inference from related datasets.

For the problem of multi-class linear classification and feature selection, we propose approximate message passing approaches to sparse multinomial logistic regression (MLR). First, we propose two algorithms based on the Hybrid Generalized Approximate Message Passing (HyGAMP) framework: one finds the maximum a posterio…

We develop a projected Nesterov's proximal-gradient (PNPG) approach for sparse signal reconstruction that combines adaptive step size with Nesterov's momentum acceleration. The objective function that we wish to minimize is the sum of a convex differentiable data-fidelity (negative log-likelihood (NLL)) term and a conv…

Sparse mapping has been a key methodology in many high-dimensional scientific problems. When multiple tasks share the set of relevant features, learning them jointly in a group drastically improves the quality of relevant feature selection. However, in practice this technique is used limitedly since such grouping infor…

We consider machine learning techniques to develop low-latency approximate solutions to a class of inverse problems. More precisely, we use a probabilistic approach for the problem of recovering sparse stochastic signals that are members of the $\ell_p$-balls. In this context, we analyze the Bayesian mean-square-error …

Recent literature on unsupervised learning focused on designing structural priors with the aim of learning meaningful features, but without considering the description length of the representations. In this thesis, first we introduce the $\varphi$ metric that evaluates unsupervised models based on their reconstruction …

The group of $\mathcal C^1$-diffeomorphisms of any sparse Cantor subset of a manifold is countable and discrete (possibly trivial). Thompson's groups come out of this construction when we consider central ternary Cantor subsets of an interval. Brin's higher dimensional generalizations $nV$ of Thompson's group $V$ arise…

We present an algorithm to identify sparse dependence structure in continuous and non-Gaussian probability distributions, given a corresponding set of data. The conditional independence structure of an arbitrary distribution can be represented as an undirected graph (or Markov random field), but most algorithms for lea…

New method combines domain changes and sparse mixing for better latent variable learning.