arXiv research

Sum-of-Squares lower bound shows NGCA requires more samples than known algorithms.

Nearly all Gaussian points in high dimensions lie on a common ellipsoid.

This paper establishes a statistical versus computational trade-off for solving a basic high-dimensional machine learning problem via a basic convex relaxation method. Specifically, we consider the {\em Sparse Principal Component Analysis} (Sparse PCA) problem, and the family of {\em Sum-of-Squares} (SoS, aka Lasserre/…

Estimation is the computational task of recovering a hidden parameter $x$ associated with a distribution $D_x$, given a measurement $y$ sampled from the distribution. High dimensional estimation problems arise naturally in statistics, machine learning, and complexity theory. Many high dimensional estimation problems ca…

Polynomial-time algorithm estimates edge density of random graphs with privacy and robustness.

New algorithm estimates transport maps with nearly optimal error.

New tools in nonlinear random matrices improve understanding of the Sum of Squares hierarchy.

Explicitly constructed 3XOR instances hard for Sum-of-Squares hierarchy.

We develop efficient algorithms for estimating low-degree moments of unknown distributions in the presence of adversarial outliers. The guarantees of our algorithms improve in many cases significantly over the best previous ones, obtained in recent works of Diakonikolas et al, Lai et al, and Charikar et al. We also sho…

Given a large data matrix $A\in\mathbb{R}^{n\times n}$, we consider the problem of determining whether its entries are i.i.d. with some known marginal distribution $A_{ij}\sim P_0$, or instead $A$ contains a principal submatrix $A_{{\sf Q},{\sf Q}}$ whose entries have marginal distribution $A_{ij}\sim P_1\neq P_0$. As …

The paper develops sum-of-squares relaxations for computing $f$-divergences.

New method clusters non-spherical Gaussian mixtures with fewer samples and time.

We study the problem of high-dimensional sparse mean estimation in the presence of an $ε$-fraction of adversarial outliers. Prior work obtained sample and computationally efficient algorithms for this task for identity-covariance subgaussian distributions. In this work, we develop the first efficient algorithms for rob…

Polynomial-time algorithm finds planted hypercube vectors in Gaussian mixtures.

In the noisy tensor completion problem we observe $m$ entries (whose location is chosen uniformly at random) from an unknown $n_1 \times n_2 \times n_3$ tensor $T$. We assume that $T$ is entry-wise close to being rank $r$. Our goal is to fill in its missing entries using as few observations as possible. Let $n = \max(n…

Efficiently estimates prediction error in regression with Gaussian covariates under privacy constraints.

This work improves graph inference using the degree-4 sum-of-squares hierarchy.

Algorithm distinguishes Gaussian mixtures from pure Gaussians in quasi-polynomial time.

We obtain the first polynomial-time algorithm for exact tensor completion that improves over the bound implied by reduction to matrix completion. The algorithm recovers an unknown 3-tensor with $r$ incoherent, orthogonal components in $\mathbb R^n$ from $r\cdot \tilde O(n^{1.5})$ randomly observed entries of the tensor…

New algorithm solves large cardinality-constrained clustering problems.

In this paper, we generalize the Cao-Yau's gradient estimate for the sum of squares of vector fields up to higher step under assumption of the generalized curvature-dimension inequality. With its applications, by deriving a curvature-dimension inequality, we are able to obtain the Li-Yau gradient estimate for the CR he…

New algorithms recover signals robustly against outliers and heavy-tailed noise.

Paper introduces a new kernel model for PSD-valued functions with theoretical guarantees and applications.

We propose an efficient meta-algorithm for Bayesian estimation problems that is based on low-degree polynomials, semidefinite programming, and tensor decomposition. The algorithm is inspired by recent lower bound constructions for sum-of-squares and related to the method of moments. Our focus is on sample complexity bo…

Survey on using low-degree polynomials to assess statistical tasks complexity.

New algorithm estimates edge density of random graphs robustly, achieving optimal breakdown point.

New algorithms learn graph structures privately, matching best results.

Neural networks minimize error with shallow ReLU models for function estimation.

Many high dimensional sparse learning problems are formulated as nonconvex optimization. A popular approach to solve these nonconvex optimization problems is through convex relaxations such as linear and semidefinite programming. In this paper, we study the statistical limits of convex relaxations. Particularly, we con…

Polynomial-time algorithm estimates mean with bounded covariance using differential privacy.

This work addresses privacy in Bayesian estimation, achieving near-optimal error rates.

Hilbert's 17th problem asks that whether every nonnegative polynomial can be a sum of squares of rational functions. It has been answered affirmatively by Artin. However, the question as to whether a given nonnegative polynomial is a sum of squares of polynomials is still a central question in real algebraic geometry. …

New algorithm learns mixtures of any constant number of Gaussians robustly.

Neuc-MDS extends MDS for non-Euclidean data.

Paper solves outlier robust mean estimation near breakdown point.

This is the fourth article of our series. Here, we study weighted norm inequalities for the Riesz transform of the Laplace-Beltrami operator on Riemannian manifolds and of subelliptic sum of squares on Lie groups, under the doubling volume property and Gaussian upper bounds.

New CRB derived for curved models using extrinsic geometry.

New algorithm recovers sparse signals robustly against Gaussian noise and adaptive adversaries.

New framework reduces sum-of-squares proof degree, speeding up clustering and robust moment estimation.

In recent years, optimization theory has been greatly impacted by the advent of sum of squares (SOS) optimization. The reliance of this technique on large-scale semidefinite programs however, has limited the scale of problems to which it can be applied. In this paper, we introduce DSOS and SDSOS optimization as linear …

Tensor rank and low-rank tensor decompositions have many applications in learning and complexity theory. Most known algorithms use unfoldings of tensors and can only handle rank up to $n^{\lfloor p/2 \rfloor}$ for a $p$-th order tensor in $\mathbb{R}^{n^p}$. Previously no efficient algorithm can decompose 3rd order ten…

Estimation under missing data shows computational and statistical limits for Gaussian data.

We propose a mixed integer programming (MIP) model and iterative algorithms based on topological orders to solve optimization problems with acyclic constraints on a directed graph. The proposed MIP model has a significantly lower number of constraints compared to popular MIP models based on cycle elimination constraint…

New estimator avoids overfitting in convex regression.

Christoffel function characterizes the corruption a bounded-degree certificate cannot remove in robust halfspace learning.

New findings on tensor decomposition complexity, showing polynomial functions can estimate the largest component under certain conditions.

Algorithm learns Gaussian mixtures robust to outliers.

In order to model entanglements of polymers in a confined region, we consider the linking numbers and writhes of cycles in random linear embeddings of complete graphs in a cube. Our main results are that for a random linear embedding of $K_n$ in a cube, the mean sum of squared linking numbers and the mean sum of square…