arXiv research

New geometric framework for positive semidefinite matrices of fixed rank.

Paper tackles multi-label learning by improving SVR for positive semidefinite metrics.

This article provides the mathematical foundation for stochastically continuous affine processes on the cone of positive semidefinite symmetric matrices. This analysis has been motivated by a large and growing use of matrix-valued affine processes in finance, including multi-asset option pricing with stochastic volatil…

Study finds polynomial convergence rate for Farey sequences linked to Riemann hypothesis.

We propose a simple, scalable, and fast gradient descent algorithm to optimize a nonconvex objective for the rank minimization problem and a closely related family of semidefinite programs. With $O(r^3 κ^2 n \log n)$ random measurements of a positive semidefinite $n \times n$ matrix of rank $r$ and condition number $κ$…

We present a hybrid algorithm for optimizing a convex, smooth function over the cone of positive semidefinite matrices. Our algorithm converges to the global optimal solution and can be used to solve general large-scale semidefinite programs and hence can be readily applied to a variety of machine learning problems. We…

New inequalities for matrix supermartingales converge under various conditions.

A new method speeds up community detection in graphs.

Several important applications, such as streaming PCA and semidefinite programming, involve a large-scale positive-semidefinite (psd) matrix that is presented as a sequence of linear updates. Because of storage limitations, it may only be possible to retain a sketch of the psd matrix. This paper develops a new algorith…

Proves Gerber statistic is always non-negative.

Testing whether a probability distribution is compatible with a given Bayesian network is a fundamental task in the field of causal inference, where Bayesian networks model causal relations. Here we consider the class of causal structures where all correlations between observed quantities are solely due to the influenc…

The paradigm of multi-task learning is that one can achieve better generalization by learning tasks jointly and thus exploiting the similarity between the tasks rather than learning them independently of each other. While previously the relationship between tasks had to be user-defined in the form of an output kernel, …

We consider stochastic partial differential equations appearing as Markovian lifts of matrix valued (affine) Volterra type processes from the point of view of the generalized Feller property (see e.g., \cite{doetei:10}). We introduce in particular Volterra Wishart processes with fractional kernels and values in the con…

An algorithm for computing positive semidefinite factorizations of matrices.

Denise learns a function to quickly decompose covariance matrices robustly.

New PSDMF algorithms derived from PR and ARM methods.

We prove that all currently known examples of manifolds with nonnegative sectional curvature satisfy a stronger condition: their curvature operator can be modified with a 4-form to become positive-semidefinite.

A fast method estimates correlations in hybrid systems using observable market data.

We consider a short rate model, driven by a stochastic process on the cone of positive semidefinite matrices. We derive sufficient conditions ensuring that the model replicates normal, inverse or humped yield curves.

Recent research in off-the-grid compressed sensing (CS) has demonstrated that, under certain conditions, one can successfully recover a spectrally sparse signal from a few time-domain samples even though the dictionary is continuous. In particular, atomic norm minimization was proposed in \cite{tang2012csotg} to recove…

We address the rectangular matrix completion problem by lifting the unknown matrix to a positive semidefinite matrix in higher dimension, and optimizing a nonconvex objective over the semidefinite factor using a simple gradient descent scheme. With $O( μr^2 κ^2 n \max(μ, \log n))$ random observations of a $n_1 \times n…

A broad class of convex optimization problems can be formulated as a semidefinite program (SDP), minimization of a convex function over the positive-semidefinite cone subject to some affine constraints. The majority of classical SDP solvers are designed for the deterministic setting where problem data is readily availa…

We put forward a complete theory on moment explosion for fairly general state-spaces. This includes a characterization of the validity of the affine transform formula in terms of minimal solutions of a system of generalized Riccati differential equations. Also, we characterize the class of positive semidefinite process…

A new matrix concentration inequality for random products of matrices.

Over the past few years, trace regression models have received considerable attention in the context of matrix completion, quantum state tomography, and compressed sensing. Estimation of the underlying matrix from regularization-based approaches promoting low-rankedness, notably nuclear norm regularization, have enjoye…

Introduces a new model for mapping matrices to matrices, subsuming linear regression.

Matrix completion is a basic machine learning problem that has wide applications, especially in collaborative filtering and recommender systems. Simple non-convex optimization algorithms are popular and effective in practice. Despite recent progress in proving various non-convex algorithms converge from a good initial …

The success of kernel methods has initiated the design of novel positive semidefinite functions, in particular for structured data. A leading design paradigm for this is the convolution kernel, which decomposes structured objects into their parts and sums over all pairs of parts. Assignment kernels, in contrast, are ob…

A streaming algorithm estimates quadratic covariation from financial data efficiently.

This paper considers inference over distributed linear Gaussian models using factor graphs and Gaussian belief propagation (BP). The distributed inference algorithm involves only local computation of the information matrix and of the mean vector, and message passing between neighbors. Under broad conditions, it is show…

Most existing word embedding methods can be categorized into Neural Embedding Models and Matrix Factorization (MF)-based methods. However some models are opaque to probabilistic interpretation, and MF-based methods, typically solved using Singular Value Decomposition (SVD), may incur loss of corpus information. In addi…

New methods improve solving linear systems and preconditioning with reduced complexity.

In recent years, optimization theory has been greatly impacted by the advent of sum of squares (SOS) optimization. The reliance of this technique on large-scale semidefinite programs however, has limited the scale of problems to which it can be applied. In this paper, we introduce DSOS and SDSOS optimization as linear …

This work studies low-rank approximation of a positive semidefinite matrix from partial entries via nonconvex optimization. We characterized how well local-minimum based low-rank factorization approximates a fixed positive semidefinite matrix without any assumptions on the rank-matching, the condition number or eigensp…

Factor analysis, a classical multivariate statistical technique is popularly used as a fundamental tool for dimensionality reduction in statistics, econometrics and data science. Estimation is often carried out via the Maximum Likelihood (ML) principle, which seeks to maximize the likelihood under the assumption that t…

Global stability bounds for matrix frames in phase retrieval problems.

New research disproves a key conjecture in optimization.

New scalable geometric framework for SPD matrices.

The paper trains neural networks with robustness guarantees using semidefinite constraints.

Efficiently solves MRF inference problems with semidefinite programming.

We consider semidefinite programs (SDPs) of size n with equality constraints. In order to overcome scalability issues, Burer and Monteiro proposed a factorized approach based on optimizing over a matrix Y of size $n$ by $k$ such that $X = YY^*$ is the SDP variable. The advantages of such formulation are twofold: the di…

Improved covariance matrix estimation for portfolio optimization with guaranteed PSD and controlled conditioning.

We glue two manifolds which have curvature operators at least k (in the sense of eigenvalues) along their common boundary. We show that if the sum of the second fundamental forms of the boundary is positive semidefinite, then the curvature operator of the resulting manifold is at least k up to an arbitrarily small erro…

Let $G$ be a compact connected Lie group and $H$ a closed subgroup of $G$. Suppose the homogeneous space $G/H$ is effective and has dimension 3 or higher. Consider a $G$-invariant, symmetric, positive-semidefinite, nonzero (0,2)-tensor field $T$ on $G/H$. Assume that $H$ is a maximal connected Lie subgroup of $G$. We p…

In this paper, we propose a low-rank coordinate descent approach to structured semidefinite programming with diagonal constraints. The approach, which we call the Mixing method, is extremely simple to implement, has no free parameters, and typically attains an order of magnitude or better improvement in optimization pe…

Trace norm regularization is a popular method of multitask learning. We give excess risk bounds with explicit dependence on the number of tasks, the number of examples per task and properties of the data distribution. The bounds are independent of the dimension of the input space, which may be infinite as in the case o…

In this paper we demonstrate that under general conditions there exists a metric in the conformal class of an arbitrary metric on a smooth, closed Riemannian manifold of dimension greater than four such that the $Q$-curvature of the metric is a constant. Existence of solutions is obtained through the combination of var…

The Hessian of neural networks can be decomposed into a sum of two matrices: (i) the positive semidefinite generalized Gauss-Newton matrix G, and (ii) the matrix H containing negative eigenvalues. We observe that for wider networks, minimizing the loss with the gradient descent optimization maneuvers through surfaces o…