arXiv research

Paper proves hardness of learning various complex models under local pseudorandom generators.

This work connects hardness of approximation and learning.

Hardness proven for learning neural networks with polynomial size and Gaussian inputs.

New unsupervised method selects hard negative samples for contrastive learning.

Hard instances, which require a long time for a specific algorithm to solve, help (1) analyze the algorithm for accelerating it and (2) build a good benchmark for evaluating the performance of algorithms. There exist several efforts for automatic generation of hard instances. For example, evolutionary algorithms have b…

Study shows exponential sample complexity for stabilizing certain linear systems.

DC3 uses deep learning to solve hard-constrained optimization problems efficiently.

Paper investigates hardness of learning neural networks under manifold hypothesis.

Hardness proven for neural networks with natural weights.

Study shows challenges in reinforcement learning math problems, proposing enhancements and a hardness measure.

New findings show learning deeper neural networks is hard even with Gaussian inputs and non-degenerate weights.

Hardness proof for agnostically learning halfspaces from worst-case lattice problems.

Meta-learning strategy improves few-shot classification performance.

Paper improves deep learning for solving evolutionary equations with trainable hard constraints.

Model shows feature learning can improve neural scaling laws for hard tasks.

Researchers prove NP-hardness of learning parameter-bounded Bayes nets.

There has recently been significant interest in hard attention models for tasks such as object recognition, visual captioning and speech recognition. Hard attention can offer benefits over soft attention such as decreased computational cost, but training hard attention models can be difficult because of the discrete la…

Paper characterizes causal graphs from hard interventions and proposes a learning algorithm.

The paper explores when linear system identification is hard or easy, especially for under-actuated systems.

Reduces learning periodic neural networks to lattice problems, proving hardness under cryptographic assumptions.

Study on learning halfspaces under adversarial perturbations, finding computational hardness.

In this article supervised learning problems are solved using soft rule ensembles. We first review the importance sampling learning ensembles (ISLE) approach that is useful for generating hard rules. The soft rules are then obtained with logistic regression from the corresponding hard rules. In order to deal with the p…

The paper explores when to prioritize easy or hard samples in learning tasks.

Optimal CL requires perfect memory and is NP-hard.

Paper introduces scalable neural architecture for solving NP-hard problems.

Introduces a continuous version of LWE problem.

Study shows optimal RL with transition look-ahead is NP-hard for $\ell \geq 2$.

MAML outperforms NAL in diverse task landscapes.

This paper shows how optimizing with hard negative examples improves image retrieval.

APT-Gen generates tasks to help RL learn in hard problems.

Theoretical comparison of three invariance approaches in deep linear networks.

We continue the study of statistical/computational tradeoffs in learning robust classifiers, following the recent work of Bubeck, Lee, Price and Razenshteyn who showed examples of classification tasks where (a) an efficient robust classifier exists, in the small-perturbation regime; (b) a non-robust classifier can be l…

Gradient descent struggles to learn equivariant neural networks, even with symmetries.

Binary classification is a common statistical learning problem in which a model is estimated on a set of covariates for some outcome indicating the membership of one of two classes. In the literature, there exists a distinction between hard and soft classification. In soft classification, the conditional class probabil…

RL pipeline simplifies knot diagrams, including very hard unknots.

HardNet adds hard constraints to neural networks without sacrificing performance.

Iterative hard thresholding (IHT) is a projected gradient descent algorithm, known to achieve state of the art performance for a wide range of structured estimation problems, such as sparse inference. In this work, we consider IHT as a solution to the problem of learning sparse discrete distributions. We study the hard…

Transformers struggle to learn Markovian dynamics, showing NP-hard optimization challenges.

New lower bounds show learning intersections of halfspaces is hard even for a few halfspaces.

It is becoming increasingly important to understand the vulnerability of machine learning models to adversarial attacks. In this paper we study the feasibility of robust learning from the perspective of computational learning theory, considering both sample and computational complexity. In particular, our definition of…

This work tackles learning Markov games with adversarial opponents and achieves both average reward and exploitation.

Hard visual attention is a promising approach to reduce the computational burden of modern computer vision methodologies. Hard attention mechanisms are typically non-differentiable. They can be trained with reinforcement learning but the high-variance training this entails hinders more widespread application. We show h…

Detecting adversarial examples is as hard as classifying them.

Catastrophic forgetting occurs when a neural network loses the information learned in a previous task after training on subsequent tasks. This problem remains a hurdle for artificial intelligence systems with sequential learning capabilities. In this paper, we propose a task-based hard attention mechanism that preserve…

Over recent years, devising classification algorithms that are robust to adversarial perturbations has emerged as a challenging problem. In particular, deep neural nets (DNNs) seem to be susceptible to small imperceptible changes over test instances. However, the line of work in provable robustness, so far, has been fo…

Single T-gate makes distribution learning hard for deep circuits.

Several learning applications require solving high-dimensional regression problems where the relevant features belong to a small number of (overlapping) groups. For very large datasets and under standard sparsity constraints, hard thresholding methods have proven to be extremely efficient, but such methods require NP h…

Study on hard Legendrian unknots using normal rulings.