Paper connects free-energy and low-degree hardness in high-dimensional statistics.
problem High-dimensional statistical inference problems are computationally hard.
method Defines a free-energy criterion and connects it to low-degree hardness.
result Establishes connection between free-energy and low-degree hardness for Gaussian models.
New method uses almost orthonormal bases to prove low-degree lower bounds in complex statistical models.
problem Proving statistical-computational gaps in high-dimensional models with planted structures.
method Constructing an almost orthonormal polynomial basis under the planted distribution.
result Established new low-degree lower bounds for various complex models.
Paper proves first non-trivial PTF testing lower bounds for NGCA.
problem Proving lower bounds against PTF tests is challenging.
method Developed tools to prove PTF testing lower bounds for NGCA.
result First non-trivial PTF testing lower bounds for NGCA.
Survey on using low-degree polynomials to assess statistical tasks complexity.
problem Understanding the complexity of statistical tasks using polynomial functions.
method Applying low-degree polynomials to measure the complexity of statistical tasks, including detection, recovery, and estimation.
result Low-degree polynomials provide a framework to predict and explain statistical-computational tradeoffs.
The resilience of low-degree Rademacher chaos is studied, providing probabilistic lower bounds.
problem Understanding how much a Rademacher chaos can withstand adversarial sign-flips without significant probability changes.
method Probabilistic lower-bound guarantees for the resilience of Rademacher chaos of arbitrary degree.
result Probabilistic lower-bound guarantees for the resilience of Rademacher chaos of arbitrary degree, especially meaningful for constant degree.
New method explains computational barriers in high-dimensional statistical models.
problem Understanding detection-recovery gaps in high-dimensional inference.
method Combining algorithmic contiguity and cross-validation reduction to obtain conditional computational lower bounds.
result Mild control of low-degree advantage is sufficient to explain computational barriers for recovery.
Statistical query algorithms and low-degree tests are nearly equivalent in high-dimensional hypothesis testing.
problem High-dimensional hypothesis testing and information-computation gaps.
method Analysis of statistical query framework and low-degree polynomials.
result Statistical query algorithms and low-degree polynomials are almost equivalent in power under mild conditions.
New lower bounds show challenges in clustering in moderate dimensions.
problem Clustering points from mixtures of isotropic Gaussians in moderate dimensions.
method Established low-degree polynomial lower bounds and developed a novel non-spectral algorithm.
result New lower bounds reveal a 'non-parametric rate' in moderate dimensions.
New evidence shows computational barriers in graphon estimation using low-degree polynomials.
problem Estimating graphons efficiently and accurately.
method Low-degree polynomials to analyze computational limits.
result Low-degree polynomial estimators cannot significantly outperform USVT in graphon estimation.
New computational lower bounds for clustering and related problems.
problem Statistical-computational gaps in high-dimensional clustering problems.
method Investigation of low-degree polynomials in latent space models to derive lower bounds.
result New and sharper computational lower bounds for clustering, sparse clustering, and biclustering.
Study disproves conjecture about low-degree polynomials in hypothesis testing.
problem Conjecture about limitations of polynomial-time algorithms in hypothesis testing.
method Used counterexamples to refute the conjecture and modified the conjecture to rule out the counterexample.
result Disproved conjecture about limitations of low-degree polynomials in hypothesis testing.
Sublinear algorithms detect cliques in graphs with high probability.
problem Detecting a planted clique in random graphs efficiently.
method Non-adaptive low-degree polynomial queries of adjacency matrix entries.
result Sublinear time detection is possible for a specific range of clique sizes.
Study shows a tradeoff between sample complexity and computational efficiency for learning halfspaces with random noise.
problem PAC learning γ-margin halfspaces with Random Classification Noise.
method Established an information-computation tradeoff and provided a simple efficient algorithm with sample complexity O(1/(γ^2 ε^2)). Also, proved lower bounds for SQ algorithms and low-degree polynomial tests.
result Inherent gap between sample complexity and computational efficiency for learning halfspaces with random noise.
Low-degree method fails to predict robust subspace recovery problem.
problem Predicting computational tractability of robust subspace recovery problem.
method Low-degree polynomial framework, anti-concentration properties.
result Low-degree method fails to predict computational tractability of robust subspace recovery problem even up to high degree.
New findings on computational limits for estimating hidden structures.
problem Estimating hidden structures in noisy data.
method Use of low-degree polynomials as a restricted model of computation.
result Established low-degree hardness of recovery problems for easy detection problems.
These notes survey and explore an emerging method, which we call the low-degree method, for predicting and understanding statistical-versus-computational tradeoffs in high-dimensional inference problems. In short, the method posits that a certain quantity -- the second moment of the low-degree likelihood ratio -- gives…
New work shows FP potential monotonicity equals low-degree polynomial estimators limits.
problem Establishing a precise mathematical relationship between statistical physics and polynomial estimators limits.
method Analyzing Gaussian additive models (GAMs) to show FP potential monotonicity equals low-degree polynomial estimators limits.
result For a broad family of Gaussian additive models, the power of low-degree polynomials is equivalent to the monotonicity of the annealed FP potential.
New algorithm achieves optimal clustering for sparse centers with high dimensions.
problem Statistical and computational limits of clustering sparse centers with high dimensions.
method Sparse clustering algorithm based on sparse PCA.
result Achieves minimax optimal misclustering rate under certain conditions.
Paper proves computational hardness for graph matching and detection problems.
problem Computational hardness for graph matching and detection problems in correlated random graphs.
method Algorithmic contiguity and low-degree advantage bounds.
result No efficient algorithms exist for certain graph matching and detection problems.
We propose an efficient meta-algorithm for Bayesian estimation problems that is based on low-degree polynomials, semidefinite programming, and tensor decomposition. The algorithm is inspired by recent lower bound constructions for sum-of-squares and related to the method of moments. Our focus is on sample complexity bo…
New SQ lower bounds show learning mixtures of bounded covariance Gaussians is hard.
problem Learning mixtures of Gaussians with bounded covariance matrices is hard.
method Statistical Query (SQ) lower bounds.
result Any SQ algorithm requires complexity at least d Ω ( 1 / ε ) d^{Ω(1/ε)} d Ω ( 1/ ε ) for learning mixtures of bounded covariance Gaussians. Study on estimating Gaussian mean with missing data in high dimensions.
problem Estimating Gaussian mean in high dimensions with missing data due to realizable contamination.
method Statistical Query model, Low-Degree Polynomials, PTF tests, and algorithms.
result Established information-computation gap and developed efficient algorithms.
Study near-optimal bounds for learning Gaussian halfspaces with random noise.
problem Learning general halfspaces with Gaussian distribution and random classification noise.
method Established nearly-matching algorithmic and SQ lower bounds, developed a computationally efficient learning algorithm.
result Sample complexity of learning algorithm is O ( d / ε + d / ( max { p , ε } ) 2 ) O(d/ε + d/(\max\{p, ε\})^2) O ( d / ε + d / ( max { p , ε } ) 2 ) , SQ lower bound is Ω ( d 1 / 2 / ( max { p , ε } ) 2 ) Ω(d^{1/2}/(\max\{p, ε\})^2) Ω ( d 1/2 / ( max { p , ε } ) 2 ) . New study shows low-degree polynomial algorithms struggle at clause densities close to Fix's.
problem Finding satisfying assignments in random k-SAT formulas at high clause densities.
method Analysis of low-degree polynomial algorithms and a new many-way overlap gap property.
result No efficient algorithms can find satisfying assignments at clause densities close to Fix's.
New algorithms learn multi-index models via harmonic analysis, achieving statistical and computational trade-offs.
problem Learning multi-index models with unknown projections of input data.
method Exploiting the equivariance of the problem under the orthogonal group, we derive lower bounds and construct spectral algorithms based on harmonic tensor unfolding.
result Achieve statistical and computational trade-offs between sample and runtime complexity.
Learn low-degree functions with few random queries.
problem Learning low-degree functions from limited random queries.
method Learn bounded functions f : { − 1 , 1 } n o [ − 1 , 1 ] f:\{-1,1\}^n o[-1,1] f : { − 1 , 1 } n o [ − 1 , 1 ] of degree at most d d d with L 2 L_2 L 2 -accuracy ε \varepsilon ε and confidence 1 − δ 1-δ 1 − δ from log ( f r a c n δ ) ε − d − 1 C d 3 / 2 log d \log( frac{n}δ)\,\varepsilon^{-d-1} C^{d^{3/2}\sqrt{\log d}} log ( f r a c n δ ) ε − d − 1 C d 3/2 l o g d random queries. result Learn low-degree functions efficiently with logarithmic number of random queries.
Let Γ Γ Γ be a discrete group. Assuming rational injectivity of the Baum-Connes assembly map, we provide new lower bounds on the rank of the positive scalar curvature bordism group and the relative group in Stolz' positive scalar curvature sequence for B Γ \mathrm{B} Γ B Γ . The lower bounds are formulated in terms of the part …
Polynomial-time algorithm finds planted hypercube vectors in Gaussian mixtures.
problem Clustering d-dimensional Gaussian mixtures with unknown covariance.
method Lattice-based methods using Lenstra--Lenstra--Lovasz reduction.
result Achieves statistically-optimal sample complexity of d+1 samples.
New study shows limits of low-degree algorithms in finding large independent sets in sparse hypergraphs.
problem Finding large independent sets in sparse random hypergraphs.
method Low-degree polynomial algorithms are analyzed to determine their limits.
result Low-degree algorithms can find independent sets of density up to \(\left(\frac{\log d}{(r-1)d}
ight)^{1/(r-1)}\), but no larger.
New SQ lower bound shows complexity nearly matches known upper bound for smoothed agnostic learning.
problem Smoothed agnostic learning of halfspaces under subgaussian distributions.
method Statistical Query (SQ) lower bound using moment-matching hard distribution and linear programming duality.
result First non-trivial lower bound on complexity nearly matches known upper bound.
Optimized Franz-Parisi criterion matches SQ lower bounds for various statistical models.
problem Understanding computational hardness in statistical inference.
method Proposed and refined Franz-Parisi criterion, established equivalence with SQ lower bounds.
result Optimized Franz-Parisi criterion is equivalent to Statistical Query (SQ) lower bounds.
Study shows it's impossible to count communities without finding them.
problem Determining the number and sizes of communities in random graph models.
method Hypothesis testing between models with different community structures, using low-degree polynomial framework.
result Testing between two different planted distributions is as hard as finding the communities.
Improved efficient robust regression with near-linear time and subquadratic samples.
problem Robust linear regression with unknown covariance matrix under Gaussian covariates.
method Near-linear time algorithm using subquadratic samples, complemented by SQ and polynomial lower bounds.
result Achieves prediction error O ( ε κ ) O(\sqrt{εκ}) O ( ε κ ) for ε κ ≲ 1 εκ\lesssim 1 ε κ ≲ 1 , improving over prior works. High-dimensional kernel regression struggles due to rotational invariance.
problem Kernel ridge regression struggles in high dimensions due to rotational invariance.
method Analysis of kernel properties and their impact on high-dimensional data.
result Lower bound on generalization error for high-dimensional kernel regression.
New findings show that common optimization algorithms struggle with random problems.
problem Finding near-optimal solutions to random optimization problems.
method Low-degree polynomials, Boolean circuits, and Langevin dynamics.
result These algorithms fail to produce nearly optimal solutions with high probability.
New method for estimating sparse means in noisy data.
problem Estimating the mean of a sparse distribution in the presence of outliers.
method Difference-of-Pairs Filtering technique for list-decodable sparse mean estimation.
result First sample and computationally efficient algorithm for list-decodable sparse mean estimation.
Sum-of-Squares lower bound shows NGCA requires more samples than known algorithms.
problem Finding a non-Gaussian direction in a high-dimensional dataset.
method Sum-of-Squares (SoS) framework to prove lower bounds.
result First super-constant degree SoS lower bound for NGCA.
We develop efficient algorithms for estimating low-degree moments of unknown distributions in the presence of adversarial outliers. The guarantees of our algorithms improve in many cases significantly over the best previous ones, obtained in recent works of Diakonikolas et al, Lai et al, and Charikar et al. We also sho…
An algorithm learns from multiple models to match an oracle's risk.
problem Learning from multiple noisy models to estimate a target parameter.
method Elimination rounds algorithm for adaptive learning.
result Risk of weak-oracle learner matches that of an oracle in multiple source case.
New findings on tensor decomposition complexity, showing polynomial functions can estimate the largest component under certain conditions.
problem The complexity of tensor decomposition, especially for low-degree polynomials.
method Modeling a slightly larger component in a random tensor decomposition and using polynomial functions to estimate it.
result Polynomial functions can accurately estimate the largest component when r ≪ n 3 / 2 r \ll n^{3/2} r ≪ n 3/2 but fail when r ≫ n 3 / 2 r \gg n^{3/2} r ≫ n 3/2 . We introduce the problem of learning mixtures of k k k subcubes over { 0 , 1 } n \{0,1\}^n { 0 , 1 } n , which contains many classic learning theory problems as a special case (and is itself a special case of others). We give a surprising n O ( log k ) n^{O(\log k)} n O ( l o g k ) -time learning algorithm based on higher-order multilinear moments. It is not possible to l…
It is proved that the continuous bounded cohomology of SL_2(k) vanishes in all positive degrees whenever k is a non-Archimedean local field. This holds more generally for boundary-transitive groups of tree automorphisms and implies low degree vanishing for SL_2 over S-integers.
Unified approach to tensor PCA and related problems using tensor cumulants.
problem Statistical inference on invariant distributions, particularly tensor PCA.
method Definition and analysis of tensor cumulants to unify and extend previous results.
result Unified explanation of hardness and subexponential-time algorithms for tensor PCA.
Improved agnostic learning time via Gaussian surface area analysis.
problem Learning polynomial threshold functions under Gaussian marginals.
method Improvement of polynomial degree required for approximation.
result Near optimal bounds on agnostic learning complexity.
We study the problem of high-dimensional sparse mean estimation in the presence of an ε ε ε -fraction of adversarial outliers. Prior work obtained sample and computationally efficient algorithms for this task for identity-covariance subgaussian distributions. In this work, we develop the first efficient algorithms for rob…
Hard problem of learning simple generative models from i.i.d. samples.
problem Learning simple neural network distributions from samples.
method Statistical query model, ODE-based construction of piecewise-linear functions.
result No polynomial-time algorithm can solve this problem even with one-hidden-layer ReLU networks.
Improved robust regression with clean covariates achieves better rates than Huber's model.
problem Robust regression under adaptive contamination of responses with clean covariates.
method Exploiting clean covariates to construct an estimator achieving better rates than Huber's model.
result Improved estimation rate even with constant contamination, achieving consistency.
Semi-supervised learning improves classification in high dimensions.
problem Combining labeled and unlabeled data for high-dimensional classification.
method Information theoretic and computational lower bounds analysis for feature selection.
result Semi-supervised learning is advantageous for classification in high dimensions.