arXiv research

The paper develops sum-of-squares relaxations for computing $f$-divergences.

We study a statistical model for the tensor principal component analysis problem introduced by Montanari and Richard: Given a order-$3$ tensor $T$ of the form $T = τ\cdot v_0^{\otimes 3} + A$, where $τ\geq 0$ is a signal-to-noise ratio, $v_0$ is a unit vector, and $A$ is a random noise tensor, the goal is to recover th…

New algorithms learn graph structures privately, matching best results.

This work improves graph inference using the degree-4 sum-of-squares hierarchy.

This paper establishes a statistical versus computational trade-off for solving a basic high-dimensional machine learning problem via a basic convex relaxation method. Specifically, we consider the {\em Sparse Principal Component Analysis} (Sparse PCA) problem, and the family of {\em Sum-of-Squares} (SoS, aka Lasserre/…

New algorithm solves large cardinality-constrained clustering problems.

Many high dimensional sparse learning problems are formulated as nonconvex optimization. A popular approach to solve these nonconvex optimization problems is through convex relaxations such as linear and semidefinite programming. In this paper, we study the statistical limits of convex relaxations. Particularly, we con…

New algorithm learns mixtures of any constant number of Gaussians robustly.

We study tensor completion in the agnostic setting. In the classical tensor completion problem, we receive $n$ entries of an unknown rank-$r$ tensor and wish to exactly complete the remaining entries. In agnostic tensor completion, we make no assumption on the rank of the unknown tensor, but attempt to predict unknown …

We develop efficient algorithms for estimating low-degree moments of unknown distributions in the presence of adversarial outliers. The guarantees of our algorithms improve in many cases significantly over the best previous ones, obtained in recent works of Diakonikolas et al, Lai et al, and Charikar et al. We also sho…

Paper improves variational inference on Boolean hypercube using quantum methods.

In the noisy tensor completion problem we observe $m$ entries (whose location is chosen uniformly at random) from an unknown $n_1 \times n_2 \times n_3$ tensor $T$. We assume that $T$ is entry-wise close to being rank $r$. Our goal is to fill in its missing entries using as few observations as possible. Let $n = \max(n…

New algorithms recover signals robustly against outliers and heavy-tailed noise.

New tools in nonlinear random matrices improve understanding of the Sum of Squares hierarchy.

Maximum A posteriori Probability (MAP) inference in graphical models amounts to solving a graph-structured combinatorial optimization problem. Popular inference algorithms such as belief propagation (BP) and generalized belief propagation (GBP) are intimately related to linear programming (LP) relaxation within the She…

Estimation is the computational task of recovering a hidden parameter $x$ associated with a distribution $D_x$, given a measurement $y$ sampled from the distribution. High dimensional estimation problems arise naturally in statistics, machine learning, and complexity theory. Many high dimensional estimation problems ca…

Efficiently estimates covariance matrix for elliptical distributions under strong contamination.

New method certifies anti-concentration for various non-Gaussian distributions.

New CRB derived for curved models using extrinsic geometry.

A new fuzzy clustering method using hyperbolic smoothing for large datasets.

Hilbert's 17th problem asks that whether every nonnegative polynomial can be a sum of squares of rational functions. It has been answered affirmatively by Artin. However, the question as to whether a given nonnegative polynomial is a sum of squares of polynomials is still a central question in real algebraic geometry. …

Polynomial-time private algorithm for robust estimation of mean and covariance in the presence of outliers.

Paper solves outlier robust mean estimation near breakdown point.

Given a large data matrix $A\in\mathbb{R}^{n\times n}$, we consider the problem of determining whether its entries are i.i.d. with some known marginal distribution $A_{ij}\sim P_0$, or instead $A$ contains a principal submatrix $A_{{\sf Q},{\sf Q}}$ whose entries have marginal distribution $A_{ij}\sim P_1\neq P_0$. As …

New framework reduces sum-of-squares proof degree, speeding up clustering and robust moment estimation.

In recent years, optimization theory has been greatly impacted by the advent of sum of squares (SOS) optimization. The reliance of this technique on large-scale semidefinite programs however, has limited the scale of problems to which it can be applied. In this paper, we introduce DSOS and SDSOS optimization as linear …

Polynomial-time algorithm estimates edge density of random graphs with privacy and robustness.

Paper introduces a new kernel model for PSD-valued functions with theoretical guarantees and applications.

In order to model entanglements of polymers in a confined region, we consider the linking numbers and writhes of cycles in random linear embeddings of complete graphs in a cube. Our main results are that for a random linear embedding of $K_n$ in a cube, the mean sum of squared linking numbers and the mean sum of square…

New algorithm recovers sparse signals robustly against Gaussian noise and adaptive adversaries.

We consider two problems that arise in machine learning applications: the problem of recovering a planted sparse vector in a random linear subspace and the problem of decomposing a random low-rank overcomplete 3-tensor. For both problems, the best known guarantees are based on the sum-of-squares method. We develop new …

New algorithm estimates transport maps with nearly optimal error.

We obtain the first polynomial-time algorithm for exact tensor completion that improves over the bound implied by reduction to matrix completion. The algorithm recovers an unknown 3-tensor with $r$ incoherent, orthogonal components in $\mathbb R^n$ from $r\cdot \tilde O(n^{1.5})$ randomly observed entries of the tensor…

In this paper, we generalize the Cao-Yau's gradient estimate for the sum of squares of vector fields up to higher step under assumption of the generalized curvature-dimension inequality. With its applications, by deriving a curvature-dimension inequality, we are able to obtain the Li-Yau gradient estimate for the CR he…

Explicitly constructed 3XOR instances hard for Sum-of-Squares hierarchy.

Efficiently estimates prediction error in regression with Gaussian covariates under privacy constraints.

Nearly all Gaussian points in high dimensions lie on a common ellipsoid.

KSOS improves kernel learning for dynamical systems via global optimization.

This note optimizes distributions using kernel mean embeddings with a new parameterization.

In this paper we prove local analytic hypoellipticity for a degenerate sum of squares of complex vector fields generalizing those of Kohn in "Hypoellipticity and Loss of Derivatives". Kohn's article is to appear in the Annals of Mathematics with an appendix by Derridj and Tartakoff proving local analyticity in that cas…

SOS programming verifies MTW tensor non-negativity for optimal transport maps.

Three algorithms improve starting solutions for clustering problems.

Polynomial-time algorithm estimates mean with bounded covariance using differential privacy.

New method clusters non-spherical Gaussian mixtures with fewer samples and time.

Efficiently constructs prediction bands with minimal assumptions.

Triangular map is a recent construct in probability theory that allows one to transform any source probability density function to any target density function. Based on triangular maps, we propose a general framework for high-dimensional density estimation, by specifying one-dimensional transformations (equivalently co…

Sum-of-Squares lower bound shows NGCA requires more samples than known algorithms.

Algorithm estimates mixtures of arbitrary Gaussians robustly in presence of corruptions.