arXiv research

Tensor decomposition recovers Gaussian mixtures from moments.

This work develops efficient methods for computing moments of Gaussian mixtures.

In this work, we show an injectivity result and support theorems for integral moments of a m-tensor field on a simple, real analytic, Riemannian manifold. Integral moments of m-tensor field were first introduced by Sharafutdinov. At first we generalize a Helgason type support theorem proven by Krishnan and Stefanov in …

Optimizes mixture models without parametrizing distributions using tensor decomposition.

Machine learning models accurately predict molecular magnetic anisotropy tensors.

Paper identifies tensor ranks via prior predictive matching, solving system of equations.

We consider moment matching techniques for estimation in Latent Dirichlet Allocation (LDA). By drawing explicit links between LDA and discrete versions of independent component analysis (ICA), we first derive a new set of cumulant-based tensors, with an improved sample complexity. Moreover, we reuse standard ICA techni…

Develops efficient algorithms for learning latent-variable models using implicit moment tensor computation.

Tensor decomposition methods are popular tools for learning latent variables given only lower-order moments of the data. However, the standard assumption is that we have sufficient data to estimate these moments to high accuracy. In this work, we consider the case in which certain dimensions of the data are not always …

Study on estimating rank-one tensors in noisy data with heavy tails.

Spectral methods of moments provide a powerful tool for learning the parameters of latent variable models. Despite their theoretical appeal, the applicability of these methods to real data is still limited due to a lack of robustness to model misspecification. In this paper we present a hierarchical approach to methods…

Spectral learning extends matrix methods to tensors for better latent variable modeling.

This study shows the moment-SOS hierarchy converges in polynomial optimization over product of spheres.

We introduce three novel semi-parametric extensions of probabilistic canonical correlation analysis with identifiability guarantees. We consider moment matching techniques for estimation in these models. For that, by drawing explicit links between the new models and a discrete version of independent component analysis …

Efficiently factorize tensors in streaming data with coreset selection.

Estimates MLDS using tensor decomposition, improving upon existing methods.

Motivated by the sampling problems and heterogeneity issues common in high- dimensional big datasets, we consider a class of discordant additive index models. We propose method of moments based procedures for estimating the indices of such discordant additive index models in both low and high-dimensional settings. Our …

Latent variable models with hidden binary units appear in various applications. Learning such models, in particular in the presence of noise, is a challenging computational problem. In this paper we propose a novel spectral approach to this problem, based on the eigenvectors of both the second order moment matrix and t…

This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models---including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation---which exploits a certain tensor structure in their low-order observable moments (typically…

We study the problem of learning a distribution from samples, when the underlying distribution is a mixture of product distributions over discrete domains. This problem is motivated by several practical applications such as crowd-sourcing, recommendation systems, and learning Boolean functions. The existing solutions e…

New algorithm learns ReLU networks efficiently using Schur polynomials.

A tensor model for meta-learning adapts to task-specific features.

SPIDER uses deep neural networks for streaming tensor factorization.

We first give a constructive answer to the attenuated tensor tomography problem on simple surfaces. We then use this result to propose two approaches to produce vector-valued integral transforms which are fully injective over tensor fields. The first approach is by construction of appropriate weights which vary along t…

Paper studies statistical-computational trade-offs in tensor PCA and related problems.

Study of hyperkähler reduction on abelian varieties and toric manifolds.

We provide guarantees for learning latent variable models emphasizing on the overcomplete regime, where the dimensionality of the latent space can exceed the observed dimensionality. In particular, we consider multiview mixtures, spherical Gaussian mixtures, ICA, and sparse coding models. We provide tight concentration…

The paper explores tail diversification in financial markets using entropy and mutual information.

In this paper, we propose a general framework for sparse and low-rank tensor estimation from cubic sketchings. A two-stage non-convex implementation is developed based on sparse tensor decomposition and thresholded gradient descent, which ensures exact recovery in the noiseless case and stable recovery in the noisy cas…

Develops a graphical calculus for stable curvature invariants.

Algorithm tackles large-scale portfolio optimization with higher moments, improving computational efficiency.

Polynomial-time algorithm learns ReLU networks without assumptions.

Tensor-EM method learns MoLDS from complex, noisy data.

The paper optimizes estimating high-dimensional Gaussian mixtures without separation conditions.

Study of asymmetric rank-one tensor models with non-Gaussian noise.

We present an efficient algorithm for learning mixed membership models when the number of variables $p$ is much larger than the number of hidden components $k$. This algorithm reduces the computational complexity of state-of-the-art tensor methods, which require decomposing an $O\left(p^3\right)$ tensor, to factorizing…

New algorithm for tensor decomposition and Gaussian mixture models.

We propose an efficient meta-algorithm for Bayesian estimation problems that is based on low-degree polynomials, semidefinite programming, and tensor decomposition. The algorithm is inspired by recent lower bound constructions for sum-of-squares and related to the method of moments. Our focus is on sample complexity bo…

DGMM improves Gaussian mixture modeling efficiency and stability.

We present a novel analysis of the dynamics of tensor power iterations in the overcomplete regime where the tensor CP rank is larger than the input dimension. Finding the CP decomposition of an overcomplete tensor is NP-hard in general. We consider the case where the tensor components are randomly drawn, and show that …

Improved image learning using elliptically contoured tensor-variate distributions.

Corrected whitening restores orthogonality in high-dimensional spherical Gaussian mixtures.

Paper identifies latent factors from noisy measurements using tensor decomposition.

A new memory-efficient Adam variant reduces second moments when feasible.

Tensor methods have emerged as a powerful paradigm for consistent learning of many latent variable models such as topic models, independent component analysis and dictionary learning. Model parameters are estimated via CP decomposition of the observed higher order input moments. However, in many domains, additional inv…

Tensor decomposition methods allow us to learn the parameters of latent variable models through decomposition of low-order moments of data. A significant limitation of these algorithms is that there exists no general method to regularize them, and in the past regularization has mostly been performed using bespoke modif…

We introduce polynomial processes taking values in an arbitrary Banach space $B$ via their infinitesimal generator $L$ and the associated martingale problem. We obtain two representations of the (conditional) moments in terms of solutions of a system of ODEs on the truncated tensor algebra of dual respectively bidual s…

Research explores the space-like embeddings in pseudo-hyperbolic space, finding geometric frames and actions.