arXiv research

Survey of statistical queries and their applications.

Statistical query algorithms and low-degree tests are nearly equivalent in high-dimensional hypothesis testing.

New algorithm learns disjunctions faster than previous methods.

New bounds show complex neural networks need many queries to learn.

Statistical query (SQ) algorithms are algorithms that have access to an {\em SQ oracle} for the input distribution $D$ instead of i.i.d.~ samples from $D$. Given a query function $φ:X \rightarrow [-1,1]$, the oracle returns an estimate of ${\bf E}_{ x\sim D}[φ(x)]$ within some tolerance $τ_φ$ that roughly corresponds t…

Unified approach for quantum and classical learning from evaluation oracles.

Active learning aims to obtain a classifier of high accuracy by using fewer label requests in comparison to passive learning by selecting effective queries. Many active learning methods have been developed in the past two decades, which sample queries based on informativeness or representativeness of unlabeled data poi…

Study shows limitations and possibilities of learning quantum circuit output distributions.

We study active learning where the labeler can not only return incorrect labels but also abstain from labeling. We consider different noise and abstention conditions of the labeler. We propose an algorithm which utilizes abstention responses, and analyze its statistical consistency and query complexity under fairly nat…

Paper develops online statistical inference methods for stochastic optimization using Kiefer-Wolfowitz algorithms.

Overlapping clusters are common in models of many practical data-segmentation applications. Suppose we are given $n$ elements to be clustered into $k$ possibly overlapping clusters, and an oracle that can interactively answer queries of the form "do elements $u$ and $v$ belong to the same cluster?" The goal is to recov…

Statistical Query lower bound shows difficulty in list-decodable linear regression.

Quantum circuits are hard to learn on average.

Study finds a method to discover causal relationships that are invariant to marginal distributions.

Adapts data analysis for growing data, improving generalization guarantees.

Paper develops PAC verification for hypothesis classes and statistical algorithms.

Algorithm learns arbitrary ReLU neurons under Gaussian inputs.

Three new oracle-efficient algorithms for private synthetic data release.

Optimal locally private hypothesis selection with interactive rounds.

Estimating the normalizing constant of an unnormalized probability distribution has important applications in computer science, statistical physics, machine learning, and statistics. In this work, we consider the problem of estimating the normalizing constant $Z=\int_{\mathbb{R}^d} e^{-f(x)}\,\mathrm{d}x$ to within a m…

Tensor PCA problem analyzed with statistical query lower bounds.

Unified model for interactive estimation with improved learnability measure.

Near-optimal SQ hardness shows learning halfspaces with Massart noise is hard.

A new method for private query release using Johnson-Lindenstrauss projection.

New framework limits SGD for multi-index models, addressing SQ framework shortcomings.

Study private query release with public data, reducing sample sizes.

This paper investigates and evaluates support vector machine active learning algorithms for use with imbalanced datasets, which commonly arise in many applications such as information extraction applications. Algorithms based on closest-to-hyperplane selection and query-by-committee selection are combined with methods …

Improved efficient robust regression with near-linear time and subquadratic samples.

The study sets limits on how well halfspaces can be learned when labels are corrupted.

Study replicability in high-dimensional statistics, resolving open problems.

Datasets are often reused to perform multiple statistical analyses in an adaptive way, in which each analysis may depend on the outcomes of previous analyses on the same dataset. Standard statistical guarantees do not account for these dependencies and little is known about how to provably avoid overfitting and false d…

Study shows computational and statistical gaps in Gaussian Single-Index Models.

Lower bound shows super-polynomial gap for estimating truncated Gaussian means.

Paper solves k-sparse parity problem with sign SGD, matching SQ lower bound.

Study near-optimal bounds for learning Gaussian halfspaces with random noise.

Study shows a tradeoff between sample complexity and computational efficiency for learning halfspaces with random noise.

Lower bounds show density estimation requires linear samples or query time.

FAQ efficiently evaluates LLMs with statistical guarantees using adaptive query selection.

We give algorithms for estimating the expectation of a given real-valued function $φ:X\to {\bf R}$ on a sample drawn randomly from some unknown distribution $D$ over domain $X$, namely ${\bf E}_{{\bf x}\sim D}[φ({\bf x})]$. Our algorithms work in two well-studied models of restricted access to data samples. The first o…

Why are classifiers in high dimension vulnerable to "adversarial" perturbations? We show that it is likely not due to information theoretic limitations, but rather it could be due to computational constraints. First we prove that, for a broad set of classification tasks, the mere existence of a robust classifier implie…

Gradient descent fails to learn simple neural networks efficiently.

New algorithms learn multi-index models via harmonic analysis, achieving statistical and computational trade-offs.

In this paper, we revisit the problem of private stochastic convex optimization. We propose an algorithm based on noisy mirror descent, which achieves optimal rates both in terms of statistical complexity and number of queries to a first-order stochastic oracle in the regime when the privacy parameter is inversely prop…

New algorithm tackles dynamic query routing to multiple embedding models.

New algorithm for learning ReLU networks with Gaussian noise, improving previous results.

Causal discovery from empirical data is a fundamental problem in many scientific domains. Observational data allows for identifiability only up to Markov equivalence class. In this paper we first propose a polynomial time algorithm for learning the exact correctly-oriented structure of the transitive reduction of any c…

In this paper we study the adaptive learnability of decision trees of depth at most $d$ from membership queries. This has many applications in automated scientific discovery such as drugs development and software update problem. Feldman solves the problem in a randomized polynomial time algorithm that asks $\tilde O(2^…

Efficient algorithms learn from coarse labels instead of fine grained ones.