arXiv research

Lower bounds show linear complexity for linear regression.

Matrix completion aims to predict missing elements in a partially observed data matrix which in typical applications, such as collaborative filtering, is large and extremely sparsely observed. A standard solution is matrix factorization, which predicts unobserved entries as linear combinations of latent variables. We g…

CoLA automates efficient numerical linear algebra for complex matrix structures.

Improved singular value approximation for convolutional layers.

New method reduces linear regret in high-dimensional bandit problems.

In this paper, we investigate the recovery of a sparse weight vector (parameters vector) from a set of noisy linear combinations. However, only partial information about the matrix representing the linear combinations is available. Assuming a low-rank structure for the matrix, one natural solution would be to first app…

New method tackles high-dimensional SBL without covariance matrices.

Study proposes memory-efficient backpropagation for linear layers in neural networks.

Derives adjoint formulas for matrix operations and applies them to specific cases.

Develops a fast algorithm for fitting multilevel factor models.

Paper develops a decoder for sparse codes without encoder matrix, achieving optimal recovery.

We revisit the inductive matrix completion problem that aims to recover a rank-$r$ matrix with ambient dimension $d$ given $n$ features as the side prior information. The goal is to make use of the known $n$ features to reduce sample and computational complexities. We present and analyze a new gradient-based non-convex…

This paper analyzes error in SKI for Gaussian Processes, providing conditions for linear time inference.

NGRC shows numerical instabilities with short lags and high-degree polynomials.

We provide a proof of backpropagation algorithm in matrix notation.

Mixed-precision CA-SGD for generalized linear models on GPUs

Efficient approximations reduce computation of matrix-based Renyi's entropy.

Large-scale generalized linear array models (GLAMs) can be challenging to fit. Computation and storage of its tensor product design matrix can be impossible due to time and memory constraints, and previously considered design matrix free algorithms do not scale well with the dimension of the parameter vector. A new des…

We consider the problem of finding anomalies in high-dimensional data using popular PCA based anomaly scores. The naive algorithms for computing these scores explicitly compute the PCA of the covariance matrix which uses space quadratic in the dimensionality of the data. We give the first streaming algorithms that use …

Global stability bounds for matrix frames in phase retrieval problems.

Reservoir computing's success depends on mapping different input time series to separable states.

Linear regression models depend directly on the design matrix and its properties. Techniques that efficiently estimate model coefficients by partitioning rows of the design matrix are increasingly popular for large-scale problems because they fit well with modern parallel computing architectures. We propose a simple me…

Multiresolution Matrix Factorization (MMF) was recently introduced as a method for finding multiscale structure and defining wavelets on graphs/matrices. In this paper we derive pMMF, a parallel algorithm for computing the MMF factorization. Empirically, the running time of pMMF scales linearly in the dimension for spa…

A new method to measure neural network expressiveness using tighter upper bounds.

We study the problem of estimating low-rank matrices from linear measurements (a.k.a., matrix sensing) through nonconvex optimization. We propose an efficient stochastic variance reduced gradient descent algorithm to solve a nonconvex optimization problem of matrix sensing. Our algorithm is applicable to both noisy and…

This paper describes a new approach, based on linear programming, for computing nonnegative matrix factorizations (NMFs). The key idea is a data-driven model for the factorization where the most salient features in the data are used to express the remaining features. More precisely, given a data matrix X, the algorithm…

We propose an algebraic combinatorial method for solving large sparse linear systems of equations locally - that is, a method which can compute single evaluations of the signal without computing the whole signal. The method scales only in the sparsity of the system and not in its size, and allows to provide error estim…

Non-linear shrinkage isn't optimal for portfolio optimization, especially when asset dependence is non-stationary.

Consider a noisy linear observation model with an unknown permutation, based on observing $y = Π^* A x^* + w$, where $x^* \in \mathbb{R}^d$ is an unknown vector, $Π^*$ is an unknown $n \times n$ permutation matrix, and $w \in \mathbb{R}^n$ is additive Gaussian noise. We analyze the problem of permutation recovery in a …

Last-layer approximation improves UQ performance without sacrificing computational efficiency.

New methods improve solving linear systems and preconditioning with reduced complexity.

Kernel clustering algorithm improved for large datasets using incomplete Cholesky factorization.

Robust method learns nonlinear structures robustly to noise.

Random sampling has become a critical tool in solving massive matrix problems. For linear regression, a small, manageable set of data rows can be randomly selected to approximate a tall, skinny data matrix, improving processing time significantly. For theoretical performance guarantees, each row must be sampled with pr…

Poyiadjis et al. (2011) show how particle methods can be used to estimate both the score and the observed information matrix for state space models. These methods either suffer from a computational cost that is quadratic in the number of particles, or produce estimates whose variance increases quadratically with the am…

Simple matrix formulas for Grassmannian curvatures.

We consider the problem of estimation of a low-rank matrix from a limited number of noisy rank-one projections. In particular, we propose two fast, non-convex \emph{proper} algorithms for matrix recovery and support them with rigorous theoretical analysis. We show that the proposed algorithms enjoy linear convergence a…

We develop matrix models for Grassmann, flag, and Stiefel manifolds.

LDA-GO improves LDA for high-dimensional data via gradient optimization.

A new method speeds up computation of Sinkhorn divergences to linear time.

Matrix approximation method for Bachelier option pricing and Greeks under stochastic volatility models

Study shows double descent curve in high-dimensional linear regression with random projections.

We propose a parallelizable sparse inverse formulation Gaussian process (SpInGP) for temporal models. It uses a sparse precision GP formulation and sparse matrix routines to speed up the computations. Due to the state-space formulation used in the algorithm, the time complexity of the basic SpInGP is linear, and becaus…

Two Fisher information matrix estimators are analyzed for neural networks, focusing on their variances and trade-offs.

Probabilistic approach to Boolean matrix factorization can provide solutions robustagainst noise and missing values with linear computational complexity. However,the assumption about latent factors can be problematic in real world applications.This study proposed a new probabilistic algorithm free of assumptions of lat…

The covariance matrix is formulated in the framework of a linear multivariate ARCH process with long memory, where the natural cross product structure of the covariance is generalized by adding two linear terms with their respective parameter. The residuals of the linear ARCH process are computed using historical data …

In matrix factorization, available graph side-information may not be well suited for the matrix completion problem, having edges that disagree with the latent-feature relations learnt from the incomplete data matrix. We show that removing these $\textit{contested}$ edges improves prediction accuracy and scalability. We…

New algorithm speeds up sampling from log-concave distributions over polytopes.