arXiv research

NIMFA is an open-source Python library that provides a unified interface to nonnegative matrix factorization algorithms. It includes implementations of state-of-the-art factorization methods, initialization approaches, and quality scoring. It supports both dense and sparse matrix representation. NIMFA's component-based…

Automates supervised learning pipeline design with matrix and tensor factorization.

Novel method for efficient low-rank matrix estimation and bandit algorithms.

Linear regression models depend directly on the design matrix and its properties. Techniques that efficiently estimate model coefficients by partitioning rows of the design matrix are increasingly popular for large-scale problems because they fit well with modern parallel computing architectures. We propose a simple me…

High-dimensional settings, where the data dimension ($d$) far exceeds the number of observations ($n$), are common in many statistical and machine learning applications. Methods based on $\ell_1$-relaxation, such as Lasso, are very popular for sparse recovery in these settings. Restricted Eigenvalue (RE) condition is a…

Short proof shows how ridge regression works with random data.

The Lasso is suboptimal in sparse linear regression due to design matrix constraints.

Sparse matrix decomposition identifies key design variables for ICF experiments.

Deep learning improves covariance matrix estimation for better portfolio risk management.

The LASSO is a recent technique for variable selection in the regression model \bean y & = & Xβ+ z, \eean where $X\in \R^{n\times p}$ and $z$ is a centered gaussian i.i.d. noise vector $\mathcal N(0,σ^2I)$. The LASSO has been proved to achieve remarkable properties such as exact support recovery of sparse vectors when …

Differential privacy mechanism design has traditionally been tailored for a scalar-valued query function. Although many mechanisms such as the Laplace and Gaussian mechanisms can be extended to a matrix-valued query function by adding i.i.d. noise to each element of the matrix, this method is often suboptimal as it for…

This work establishes always-valid risk bounds for online matrix completion.

Sparse superposition codes were recently introduced by Barron and Joseph for reliable communication over the AWGN channel at rates approaching the channel capacity. The codebook is defined in terms of a Gaussian design matrix, and codewords are sparse linear combinations of columns of the matrix. In this paper, we prop…

Unified approach for learning quantum operations from measurements.

Paper develops a decoder for sparse codes without encoder matrix, achieving optimal recovery.

Large-scale generalized linear array models (GLAMs) can be challenging to fit. Computation and storage of its tensor product design matrix can be impossible due to time and memory constraints, and previously considered design matrix free algorithms do not scale well with the dimension of the parameter vector. A new des…

Collaborative filtering (CF) is a popular technique in today's recommender systems, and matrix approximation-based CF methods have achieved great success in both rating prediction and top-N recommendation tasks. However, real-world user-item rating matrices are typically sparse, incomplete and noisy, which introduce ch…

PSI-LinUCB improves scalability for large recommender systems.

Matrix completion is a classical problem in data science wherein one attempts to reconstruct a low-rank matrix while only observing some subset of the entries. Previous authors have phrased this problem as a nuclear norm minimization problem. Almost all previous work assumes no explicit structure of the matrix and uses…

Private ALS method improves matrix completion with tighter rates and better privacy.

We propose computationally efficient encoders and decoders for lossy compression using a Sparse Regression Code. The codebook is defined by a design matrix and codewords are structured linear combinations of columns of this matrix. The proposed encoding algorithm sequentially chooses columns of the design matrix to suc…

This paper deals with the explicit design of strategy formulations to make the best strategic choices from a conventional matrix form of representing strategic choices. The explicit strategy formulation is an analytical model which is targeted to provide a mathematical strategy framework to find the best moment for str…

This paper studies non-asymptotic model selection for the general case of arbitrary design matrices and arbitrary nonzero entries of the signal. In this regard, it generalizes the notion of incoherence in the existing literature on model selection and introduces two fundamental measures of coherence---termed as the wor…

This paper deals with the design of a sensing matrix along with a sparse recovery algorithm by utilizing the probability-based prior information for compressed sensing system. With the knowledge of the probability for each atom of the dictionary being used, a diagonal weighted matrix is obtained and then the sensing ma…

We introduce an application of the group lasso to design of experiments. Note that we are NOT trying to explain experimental design for the group lasso. Conversely, we explain how we can use the idea of the group lasso in experimental design, showing that the problem of constructing an optimal design matrix can be tran…

We consider the problem of matrix completion on an $n \times m$ matrix. We introduce the problem of Interpretable Matrix Completion that aims to provide meaningful insights for the low-rank matrix using side information. We show that the problem can be reformulated as a binary convex optimization problem. We design Opt…

Large-scale regression problems where both the number of variables, $p$, and the number of observations, $n$, may be large and in the order of millions or more, are becoming increasingly more common. Typically the data are sparse: only a fraction of a percent of the entries in the design matrix are non-zero. Neverthele…

New Bayesian matrix completion method using Stiefel manifolds.

We study the stability vis a vis adversarial noise of matrix factorization algorithm for matrix completion. In particular, our results include: (I) we bound the gap between the solution matrix of the factorization method and the ground truth in terms of root mean square error; (II) we treat the matrix factorization as …

This paper proposes a new evaluation metric and boosting method for weight separability in neural network design. In contrast to general visual recognition methods designed to encourage both intra-class compactness and inter-class separability of latent features, we focus on estimating linear independence of column vec…

Shrunk sample covariance matrix is a factor model of a special form combining some (typically, style) risk factor(s) and principal components with a (block-)diagonal factor covariance matrix. As such, shrinkage, which essentially inherits out-of-sample instabilities of the sample covariance matrix, is not an alternativ…

New methods improve online matrix optimization with reduced computational cost.

We propose and study a row-and-column affine measurement scheme for low-rank matrix recovery. Each measurement is a linear combination of elements in one row or one column of a matrix $X$. This setting arises naturally in applications from different domains. However, current algorithms developed for standard matrix rec…

Although nonnegative matrix factorization (NMF) is NP-hard in general, it has been shown very recently that it is tractable under the assumption that the input nonnegative data matrix is close to being separable (separability requires that all columns of the input matrix belongs to the cone spanned by a small subset of…

In many signal processing and machine learning applications, datasets containing private information are held at different locations, requiring the development of distributed privacy-preserving algorithms. Tensor and matrix factorizations are key components of many processing pipelines. In the distributed setting, diff…

A new method for feature selection robust to noise and design variability.

We present two simple ways of reducing the number of parameters and accelerating the training of large Long Short-Term Memory (LSTM) networks: the first one is "matrix factorization by design" of LSTM matrix into the product of two smaller matrices, and the second one is partitioning of LSTM matrix, its inputs and stat…

A Semi-Hidden Markov Model (SHMM) for bursty error channels is defined by a state transition probability matrix $A$, a prior probability vector $Π$, and the state dependent output symbol error probability matrix $B$. Several processes are utilized for estimating $A$, $Π$ and $B$ from a given empirically obtained or sim…

We formulate the problem of matrix completion with and without side information as a non-convex optimization problem. We design fastImpute based on non-convex gradient descent and show it converges to a global minimum that is guaranteed to recover closely the underlying matrix while it scales to matrices of sizes beyon…

We solve matrix denoising with both row and column correlations, setting limits and designing optimal methods.

EDAs with matrix transpose improve Bayesian structure learning performance.

Designs neural networks on matrix manifolds for improved performance in tasks like human action recognition.

The paper develops concentration inequalities for structured random data, extending beyond independent terms.

Autoencoders are popular among neural-network-based matrix completion models due to their ability to retrieve potential latent factors from the partially observed matrices. Nevertheless, when training data is scarce their performance is significantly degraded due to overfitting. In this paper, we mit- igate overfitting…

We introduce negative binomial matrix factorization (NBMF), a matrix factorization technique specially designed for analyzing over-dispersed count data. It can be viewed as an extension of Poisson matrix factorization (PF) perturbed by a multiplicative term which models exposure. This term brings a degree of freedom fo…

Advances neural tri-factorization for clustering and discordance analysis of multi-typed data.

New findings show some designs can't be robustly recovered in linear regression.

Graph neural networks speed up nonnegative matrix factorization.