arXiv research

An algorithm for computing positive semidefinite factorizations of matrices.

We address the rectangular matrix completion problem by lifting the unknown matrix to a positive semidefinite matrix in higher dimension, and optimizing a nonconvex objective over the semidefinite factor using a simple gradient descent scheme. With $O( μr^2 κ^2 n \max(μ, \log n))$ random observations of a $n_1 \times n…

New PSDMF algorithms derived from PR and ARM methods.

This article provides the mathematical foundation for stochastically continuous affine processes on the cone of positive semidefinite symmetric matrices. This analysis has been motivated by a large and growing use of matrix-valued affine processes in finance, including multi-asset option pricing with stochastic volatil…

New inequalities for matrix supermartingales converge under various conditions.

Factor analysis, a classical multivariate statistical technique is popularly used as a fundamental tool for dimensionality reduction in statistics, econometrics and data science. Estimation is often carried out via the Maximum Likelihood (ML) principle, which seeks to maximize the likelihood under the assumption that t…

This paper considers inference over distributed linear Gaussian models using factor graphs and Gaussian belief propagation (BP). The distributed inference algorithm involves only local computation of the information matrix and of the mean vector, and message passing between neighbors. Under broad conditions, it is show…

Most existing word embedding methods can be categorized into Neural Embedding Models and Matrix Factorization (MF)-based methods. However some models are opaque to probabilistic interpretation, and MF-based methods, typically solved using Singular Value Decomposition (SVD), may incur loss of corpus information. In addi…

This work studies low-rank approximation of a positive semidefinite matrix from partial entries via nonconvex optimization. We characterized how well local-minimum based low-rank factorization approximates a fixed positive semidefinite matrix without any assumptions on the rank-matching, the condition number or eigensp…

Several important applications, such as streaming PCA and semidefinite programming, involve a large-scale positive-semidefinite (psd) matrix that is presented as a sequence of linear updates. Because of storage limitations, it may only be possible to retain a sketch of the psd matrix. This paper develops a new algorith…

Improved guarantees for nonconvex matrix factorization with rank overparameterization.

Unified framework for hyperbolic embeddings from mixed data types.

We consider stochastic partial differential equations appearing as Markovian lifts of matrix valued (affine) Volterra type processes from the point of view of the generalized Feller property (see e.g., \cite{doetei:10}). We introduce in particular Volterra Wishart processes with fractional kernels and values in the con…

RPCholesky approximates kernel matrices with few evaluations.

We are concerned with an approximation problem for a symmetric positive semidefinite matrix due to motivation from a class of nonlinear machine learning methods. We discuss an approximation approach that we call {matrix ridge approximation}. In particular, we define the matrix ridge approximation as an incomplete matri…

We consider semidefinite programs (SDPs) of size n with equality constraints. In order to overcome scalability issues, Burer and Monteiro proposed a factorized approach based on optimizing over a matrix Y of size $n$ by $k$ such that $X = YY^*$ is the SDP variable. The advantages of such formulation are twofold: the di…

The SCMU algorithm computes cone factorizations for symmetric cones, improving upon existing methods.

We propose a simple, scalable, and fast gradient descent algorithm to optimize a nonconvex objective for the rank minimization problem and a closely related family of semidefinite programs. With $O(r^3 κ^2 n \log n)$ random measurements of a positive semidefinite $n \times n$ matrix of rank $r$ and condition number $κ$…

We present a hybrid algorithm for optimizing a convex, smooth function over the cone of positive semidefinite matrices. Our algorithm converges to the global optimal solution and can be used to solve general large-scale semidefinite programs and hence can be readily applied to a variety of machine learning problems. We…

Introduces a new model for mapping matrices to matrices, subsuming linear regression.

Nonnegative matrix factorization (NMF) under the separability assumption can provably be solved efficiently, even in the presence of noise, and has been shown to be a powerful technique in document classification and hyperspectral unmixing. This problem is referred to as near-separable NMF and requires that there exist…

New algorithm improves clustering accuracy without sacrificing scalability.

New approach to convex hulls for low-rank problems.

A new matrix concentration inequality for random products of matrices.

Matrix completion is a basic machine learning problem that has wide applications, especially in collaborative filtering and recommender systems. Simple non-convex optimization algorithms are popular and effective in practice. Despite recent progress in proving various non-convex algorithms converge from a good initial …

A fast method estimates correlations in hybrid systems using observable market data.

This paper considers the matrix completion problem. We show that it is not necessary to assume joint incoherence, which is a standard but unintuitive and restrictive condition that is imposed by previous studies. This leads to a sample complexity bound that is order-wise optimal with respect to the incoherence paramete…

Over the past few years, trace regression models have received considerable attention in the context of matrix completion, quantum state tomography, and compressed sensing. Estimation of the underlying matrix from regularization-based approaches promoting low-rankedness, notably nuclear norm regularization, have enjoye…

Equivalent formulations for low-rank matrix optimization are proven.

New methods improve solving linear systems and preconditioning with reduced complexity.

Denise learns a function to quickly decompose covariance matrices robustly.

We show that the gradient descent algorithm provides an implicit regularization effect in the learning of over-parameterized matrix factorization models and one-hidden-layer neural networks with quadratic activations. Concretely, we show that given $\tilde{O}(dr^{2})$ random linear measurements of a rank $r$ positive s…

Testing whether a probability distribution is compatible with a given Bayesian network is a fundamental task in the field of causal inference, where Bayesian networks model causal relations. Here we consider the class of causal structures where all correlations between observed quantities are solely due to the influenc…

Global stability bounds for matrix frames in phase retrieval problems.

The paradigm of multi-task learning is that one can achieve better generalization by learning tasks jointly and thus exploiting the similarity between the tasks rather than learning them independently of each other. While previously the relationship between tasks had to be user-defined in the form of an output kernel, …

Sharp condition found for Burer-Monteiro method to work for MaxCut-type SDPs.

Improved covariance matrix estimation for portfolio optimization with guaranteed PSD and controlled conditioning.

Efficient knockoffs for large-scale feature selection.

Low-rank factorization is a standard way to make structured optimization problems in machine learning more tractable by replacing matrix variables with compact factors. For positive semidefinite (PSD) variables, the symmetric Burer--Monteiro factorization (sBMF) writes $Z=XX^\top$ with a single low-rank factor $X$. A r…

Accelerated RPCholesky speeds up kernel matrix approximations.

New method improves efficiency of non-convex matrix reconstruction.

Mirror descent algorithm recovers low-rank matrices in matrix sensing.

Semidefinite programs (SDP) are important in learning and combinatorial optimization with numerous applications. In pursuit of low-rank solutions and low complexity algorithms, we consider the Burer--Monteiro factorization approach for solving SDPs. We show that all approximate local optima are global optima for the pe…

This paper describes a suite of algorithms for constructing low-rank approximations of an input matrix from a random linear image of the matrix, called a sketch. These methods can preserve structural properties of the input matrix, such as positive-semidefiniteness, and they can produce approximations with a user-speci…

We propose a new input perturbation mechanism for publishing a covariance matrix to achieve $(ε,0)$-differential privacy. Our mechanism uses a Wishart distribution to generate matrix noise. In particular, We apply this mechanism to principal component analysis. Our mechanism is able to keep the positive semi-definitene…

New geometric framework for positive semidefinite matrices of fixed rank.

This paper improves linear system solving by optimizing matrix diagonal scaling.

Paper tackles multi-label learning by improving SVR for positive semidefinite metrics.