arXiv research

Model shows feature learning can improve neural scaling laws for hard tasks.

Paper investigates hardness of learning neural networks under manifold hypothesis.

SLR tackles sparse linear regression problems, showing hardness for efficient algorithms.

The paper analyzes the sample complexity of offline RL with linear approximations, identifying a hard regime and providing an algorithm.

We continue the study of statistical/computational tradeoffs in learning robust classifiers, following the recent work of Bubeck, Lee, Price and Razenshteyn who showed examples of classification tasks where (a) an efficient robust classifier exists, in the small-perturbation regime; (b) a non-robust classifier can be l…

The restricted isometry property (RIP) for design matrices gives guarantees for optimal recovery in sparse linear models. It is of high interest in compressed sensing and statistical learning. This property is particularly important for computationally efficient recovery methods. As a consequence, even though it is in …

The paper explores when linear system identification is hard or easy, especially for under-actuated systems.

New algorithms improve privacy in bandit problems with partial information.

Detects dense subhypergraphs in random hypergraphs using low-degree polynomials.

Combinatorial auctions are formulated as frustrated lattice gases on sparse random graphs, allowing the determination of the optimal revenue by methods of statistical physics. Transitions between computationally easy and hard regimes are found and interpreted in terms of the geometric structure of the space of solution…

The paper proposes an efficient estimator for linear regression with shuffled labels.

Study shows registration regime improves IPO pricing efficiency in China.

Efficient method for generating adversarial examples with limited query budget.

Quantum algorithm approximates Khovanov homology ranks.

This paper investigates the average-case time complexity of certifying RIP matrices.

Many optimization problems can be cast into the maximum satisfiability (MAX-SAT) form, and many solvers have been developed for tackling such problems. To evaluate a MAX-SAT solver, it is convenient to generate hard MAX-SAT instances with known solutions. Here, we propose a method of generating weighted MAX-2-SAT insta…

New insights into statistical and computational limits for mixed sparse linear regression.

In this paper we take a problem of unsupervised nodes clustering on graphs and show how recent advances in attention models can be applied successfully in a "hard" regime of the problem. We propose an unsupervised algorithm that encodes Bethe Hessian embeddings by optimizing soft modularity loss and argue that our mode…

By simulating the easy-to-hard learning manners of humans/animals, the learning regimes called curriculum learning~(CL) and self-paced learning~(SPL) have been recently investigated and invoked broad interests. However, the intrinsic mechanism for analyzing why such learning regimes can work has not been comprehensivel…

We consider online learning problems where the aim is to achieve regret which is efficient in the sense that it is the same order as the lowest regret amongst K experts. This is a substantially stronger requirement that achieving $O(\sqrt{n})$ or $O(\log n)$ regret with respect to the best expert and standard algorithm…

The paper examines the optimality of kernel methods in high-dimensional clustering.

New research shows the maximum ℓ1-margin classifier doesn't adapt to sparse ground truths.

Paper proposes DG-ETC for online submodular maximization with stochastic bandit feedback.

Market competition depends on computational complexity, P != NP makes it impossible.

The paper tackles intervention generalization using factor graph models.

Detection of dense cycles in graphs reveals a gap between easy detection and hard recovery.

Inverse problems correspond to a certain type of optimization problems formulated over appropriate input distributions. Recently, there has been a growing interest in understanding the computational hardness of these optimization problems, not only in the worst case, but in an average-complexity sense under this same i…

Classification and regression tasks in overparameterized models show different generalization properties.

Two-layer networks learn hard GLMs with SGD in high dimensions.

We consider two closely related problems: planted clustering and submatrix localization. The planted clustering problem assumes that a random graph is generated based on some underlying clusters of the nodes; the task is to recover these clusters given the graph. The submatrix localization problem concerns locating hid…

Financial markets are not random, but hard to predict due to hidden causes and strategic use.

Study limits of testing algorithms without assumptions, finding key performance bounds.

We consider the problem of sparsity-constrained $M$-estimation when both explanatory and response variables have heavy tails (bounded 4-th moments), or a fraction of arbitrary corruptions. We focus on the $k$-sparse, high-dimensional regime where the number of variables $d$ and the sample size $n$ are related through $…

We present a selective sampling method designed to accelerate the training of deep neural networks. To this end, we introduce a novel measurement, the minimal margin score (MMS), which measures the minimal amount of displacement an input should take until its predicted classification is switched. For multi-class linear…

Study privacy and robustness in learning halfspaces, proving hard trade-offs.

The paper studies privacy-protected BAI with fixed confidence, deriving lower bounds and proposing an adaptive algorithm.

Stochastic Gradient Descent phases explained for deep networks.

CPCR mitigates bias in PCR for overparameterized models.

Given a graphical model (GM), computing its partition function is the most essential inference task, but it is computationally intractable in general. To address the issue, iterative approximation algorithms exploring certain local structure/consistency of GM have been investigated as popular choices in practice. Howev…

In intractable, undirected graphical models, an intuitive way of creating structured mean field approximations is to select an acyclic tractable subgraph. We show that the hardness of computing the objective function and gradient of the mean field objective qualitatively depends on a simple graph property. If the tract…

Study how firm liquidation regimes affect shareholder value and stability.

We propose a nonconvex estimator for joint multivariate regression and precision matrix estimation in the high dimensional regime, under sparsity constraints. A gradient descent algorithm with hard thresholding is developed to solve the nonconvex estimator, and it attains a linear rate of convergence to the true regres…

New algorithm learns sparse GLMs for binary outcomes efficiently.

We study the problem of finding a universal (image-agnostic) perturbation to fool machine learning (ML) classifiers (e.g., neural nets, decision tress) in the hard-label black-box setting. Recent work in adversarial ML in the white-box setting (model parameters are known) has shown that many state-of-the-art image clas…

Study online control of unknown time-varying systems with negative and positive results.

Predicting labels of nodes in a network, such as community memberships or demographic variables, is an important problem with applications in social and biological networks. A recently-discovered phase transition puts fundamental limits on the accuracy of these predictions if we have access only to the network topology…

Optimal learning rates decay to zero in easy tasks and maintain a warmup phase in hard tasks.

Paper improves robustness certification by integrating ML and logical reasoning.