This work connects hardness of approximation and learning.
problem Hardness of approximation and learnability in machine learning.
method Shows a single hardness property implying both approximation and learning hardness.
result Obtains new results on hardness of approximation and learnability of specific functions.
We show that it is NP-hard to approximate the hyperspherical radius of a triangulated manifold up to an almost-polynomial factor.
It is well known that Sparse PCA (Sparse Principal Component Analysis) is NP-hard to solve exactly on worst-case instances. What is the complexity of solving Sparse PCA approximately? Our contributions include: 1) a simple and efficient algorithm that achieves an n−1/3-approximation; 2) NP-hardness of approximatio…
In this thesis I explore challenging discrete energy minimization problems that arise mainly in the context of computer vision tasks. This work motivates the use of such "hard-to-optimize" non-submodular functionals, and proposes methods and algorithms to cope with the NP-hardness of their optimization. Consequently, t…
DC3 uses deep learning to solve hard-constrained optimization problems efficiently.
problem Hard constraints in optimization problems make classical solvers slow and infeasible.
method DC3 employs a differentiable procedure to enforce feasibility and unrolls corrections for inequality constraints.
result DC3 achieves near-optimal solutions while maintaining feasibility in both synthetic and real-world tasks.
HARFE approximates sparse additive functions using random features and ridge regression.
problem Approximating high-dimensional sparse additive functions.
method Hard-ridge random feature expansion with sparse ridge regression and hard-thresholding pursuit.
result HARFE method converges with a given error bound and achieves lower error than other algorithms.
Hard to approximate critical points for simple nonconvex functions.
problem Approximating critical points of nonconvex functions.
method Proving hardness results for polynomial-time approximation of critical points.
result Proving that approximating critical points is intractable for simple nonconvex functions.
Paper proves MDS NP-hard and provides a PTAS.
problem Theoretical limitations of MDS objective function.
method Proves NP-hardness and provides a PTAS approximation algorithm.
result Minimizing Kamada-Kawai objective is NP-hard.
Quantum algorithm approximates Khovanov homology ranks.
problem Efficient computation of Khovanov homology ranks.
method Novel quantum algorithm with pre-thermalization procedure.
result Additive approximations to Khovanov homology ranks are hard problems.
This paper explores the computational hardness of generating latent vectors for generative models.
problem Computational hardness of generating latent vectors for generative models.
method Established lower bounds for exact and approximate model inversion under strong exponential time hypothesis (SETH) and exponential time hypothesis (ETH).
result Lower bounds for computational complexity of exact and approximate model inversion.
HardNet adds hard constraints to neural networks without sacrificing performance.
problem Ensuring adherence to input-dependent constraints in neural networks.
method Appends a differentiable enforcement layer to neural networks for end-to-end training with hard constraint guarantees.
result HardNet retains neural networks' universal approximation capabilities and enables efficient optimization.
Hardness proven for learning neural networks with polynomial size and Gaussian inputs.
problem Learning one hidden layer ReLU neural networks with polynomial size and Gaussian inputs.
method Based on the hardness of the Continuous Learning with Errors (CLWE) problem.
result Hardness of learning neural networks is proven under standard cryptographic assumptions.
New polynomial-time solutions found for training ReLU networks, mirroring Max-Cut complexity.
problem Training two-layer ReLU neural networks with weight decay regularization.
method Developed a convex formulation and randomized algorithm to find approximate global optimizers.
result First polynomial-time approximation guarantees and hardness of approximation results for regularized ReLU networks.
The paper analyzes the sample complexity of offline RL with linear approximations, identifying a hard regime and providing an algorithm.
problem Sample complexity of policy evaluation in infinite-horizon offline reinforcement learning with linear function approximation.
method Identification of a hard regime and construction of hard instances; algorithm with sample complexity bound.
result An algorithm that guarantees approximation to the value function up to an additive error of ε with high probability.
Improved hardness results for clearing payments in financial networks with CDSs.
problem Determining clearing payments in financial networks with CDSs after financial shocks.
method Analyzing computational complexity of clearing problems, showing PPAD-hardness and FIXP-completeness improvements.
result PPAD-hardness of clearing problem significantly improved to ε ≈ 0.101.
We give a reduction from {\sc clique} to establish that sparse PCA is NP-hard. The reduction has a gap which we use to exclude an FPTAS for sparse PCA (unless P=NP). Under weaker complexity assumptions, we also exclude polynomial constant-factor approximation algorithms.
Study on computing and estimating calibration distance, showing hardness and efficiency.
problem Computing and estimating calibration distance under different assumptions.
method Efficient algorithm for exact computation, polynomial-time approximation scheme; sample-based estimation for upper bounds.
result The problem becomes NP-hard when assumptions are removed, but efficient algorithms exist under certain conditions.
New analysis shows Thompson Sampling can work with greedy approximations in combinatorial bandits.
problem Thompson Sampling's theoretical limits with greedy approximations in combinatorial semi-bandits.
method Study with greedy oracle, providing lower and upper bounds on regret.
result First theoretical results showing TS can work with greedy approximations, breaking misconceptions.
Sharp results link DLN gradient flow to basis pursuit optimization and GHA phase transitions.
problem Understanding implicit regularization in Diagonal Linear Networks.
method Sharp convergence bounds and characterization of ℓ1 minimizers. result Gradient flow of DLNs with tiny initialization approximates minimizers of basis pursuit optimization problem.
Efficient algorithm approximates discrete random variables with minimal Kolmogorov distance.
problem Estimating the probability of missing deadlines in series-parallel schedules.
method An efficient algorithm that computes a random variable with minimal Kolmogorov distance to a given discrete random variable.
result The algorithm efficiently approximates the probability of missing deadlines with minimal Kolmogorov distance.
Optimal intervention in economic networks modeled as influence maximization, with hard computational problems.
problem Optimal intervention in economic networks modeled as influence maximization.
method Transformed into influence maximization-like form, with theoretical and practical implications.
result Optimal intervention is NP-hard and cannot be approximated to a constant factor in polynomial time.
Optimal transport is #P-hard when components are independent, even with approximate solutions.
problem Computational complexity of optimal transport with independent marginals.
method Proved #P-hardness and developed a pseudo-polynomial time approximation algorithm.
result Optimal transport is #P-hard even with independent components and approximate solutions.
Bailouts in financial networks are hard to optimize due to NP-hardness.
problem Optimizing bailouts in a network of insolvent banks.
method Modeling bailouts as an optimization problem, proving NP-hardness and inapproximability.
result Banks can strategically alter debt contracts to increase their market value in the event of a bailout.
Novel active learning framework using sparse approximation for efficient model training.
problem Efficient model training with limited labeled data.
method Formulates batch active learning as sparsity-constrained discontinuous optimization problems, using greedy or proximal iterative hard thresholding algorithms.
result Achieves competitive performance with lower computational complexity across different settings.
New method controls gradient error for sparse MRFs.
problem Efficient learning for sparse discrete MRFs with NP-hard inference.
method Stochastic proximal gradient (SPG) with controlled gradient approximation error.
result Novel bounds control gradient approximation quality.
Hard visual attention is a promising approach to reduce the computational burden of modern computer vision methodologies. Hard attention mechanisms are typically non-differentiable. They can be trained with reinforcement learning but the high-variance training this entails hinders more widespread application. We show h…
Mutual Information (MI) is often used for feature selection when developing classifier models. Estimating the MI for a subset of features is often intractable. We demonstrate, that under the assumptions of conditional independence, MI between a subset of features can be expressed as the Conditional Mutual Information (…
Given a graphical model (GM), computing its partition function is the most essential inference task, but it is computationally intractable in general. To address the issue, iterative approximation algorithms exploring certain local structure/consistency of GM have been investigated as popular choices in practice. Howev…
Efficient algorithms for online learning with changing action sets, achieving no-approximate-regret guarantees.
problem Online learning with sleeping experts/bandits, where only a subset of actions are available each time.
method Developed computationally efficient algorithms providing no-approximate-regret guarantees for the general problem and better approximation ratios for special cases.
result Achieved no-approximate-regret guarantees for the general sleeping expert/bandit problems and better approximation ratios for specific cases.
Iterative hard thresholding (IHT) is a projected gradient descent algorithm, known to achieve state of the art performance for a wide range of structured estimation problems, such as sparse inference. In this work, we consider IHT as a solution to the problem of learning sparse discrete distributions. We study the hard…
An algorithmically hard phase was described in a range of inference problems: even if the signal can be reconstructed with a small error from an information theoretic point of view, known algorithms fail unless the noise-to-signal ratio is sufficiently small. This hard phase is typically understood as a metastable bran…
New model approximates sparse mean-CVaR portfolio optimization efficiently.
problem NP-hard ℓ0-constrained mean-CVaR optimization. method Proximal alternating linearized minimization algorithm with nested fixed-point proximity.
result The model offers a guaranteed approximation of the ℓ0-constrained mean-CVaR model. Paper presents a randomized algorithm for SPCA with high probability approximation.
problem Sparse Principal Component Analysis (SPCA) is NP-hard.
method Based on basic SDP relaxation, the algorithm constructs deterministic and randomized solutions.
result The algorithm achieves an approximation ratio of at most the sparsity constant with high probability.
Breaks the hardness conjecture for batch RL with a novel tournament-based approach.
problem Sample-efficient reinforcement learning from exploratory data.
method BVFT algorithm using pairwise comparison and state-action partition.
result Solves the learning problem in a setting previously thought impossible.
New lower bounds show learning intersections of halfspaces is hard even for a few halfspaces.
problem Learning intersections of halfspaces in polynomial time under standard assumptions.
method Unified connection to parallel pancakes distribution for proving hardness.
result Learning ω(loglogN) halfspaces in dimension N requires super-polynomial time under standard assumptions. EPMF factorizes matrices by adjusting their entries to match a specified power.
problem Factorizing matrices with adjusted entries to match a specified power.
method Analyzes the computational complexity of exact and approximate EPMF problems.
result Exact EPMF is strongly NP-hard, but can be solved in polynomial time when rank is fixed.
We solve ReLU regression with efficient approximations for various distributions.
problem Finding the best fitting ReLU function with square loss from unknown distributions.
method Introduced efficient constant-factor approximation algorithm and polynomial-time approximation scheme.
result First constant-factor approximation algorithm for ReLU regression with weak concentration conditions.
Coresets are efficient representations of data sets such that models trained on the coreset are provably competitive with models trained on the original data set. As such, they have been successfully used to scale up clustering models such as K-Means and Gaussian mixture models to massive data sets. However, until now,…
Paper solves NP-hard sparse mixed linear regression problem with provable guarantees.
problem Sparse mixed linear regression on unlabeled data.
method Invex relaxation for intractable problem with theoretical guarantees.
result Exact recovery of data labels and close approximation of regression parameters.
OTSS learns personalized decision weights from logged decisions and outputs.
problem Learning context-specific decision weights from logged decisions and outputs.
method Output-targeted soft-segmentation model that deploys personalized decision-ready weight vectors.
result OTSS achieves the lowest mean regret in benchmark settings.
Reduces learning periodic neural networks to lattice problems, proving hardness under cryptographic assumptions.
problem Learning single periodic neurons in noisy environments.
method Reduction to worst-case lattice problems, using LLL algorithm.
result Polynomial-time algorithms for learning these functions are hard under cryptographic assumptions.
We develop an efficient algorithm to find confidence ellipsoids with volume guarantees in high dimensions.
problem Finding robust confidence ellipsoids in high-dimensional data.
method Polynomial time algorithm using primal-dual structure and geometric Brascamp-Lieb inequality.
result Algorithm finds ellipsoids within a O(β)γd volume factor of best β-conditioned ellipsoid. We show that the existence of a computationally efficient calibration algorithm, with a low weak calibration rate, would imply the existence of an efficient algorithm for computing approximate Nash equilibria - thus implying the unlikely conclusion that every problem in PPAD is solvable in polynomial time.
We study the problem of nonnegative rank-one approximation of a nonnegative tensor, and show that the globally optimal solution that minimizes the generalized Kullback-Leibler divergence can be efficiently obtained, i.e., it is not NP-hard. This result works for arbitrary nonnegative tensors with an arbitrary number of…
APT-Gen generates tasks to help RL learn in hard problems.
problem Learning in hard exploration problems.
method APT-Gen uses a task generator to create tasks from a parameterized space, balancing performance and similarity to target tasks.
result APT-Gen outperforms baselines in grid world and robotic manipulation tasks.
New algorithms optimize a soft-robust criterion in reinforcement learning, reducing conservatism.
problem Computing robust policies for high-stakes decisions with limited data.
method Soft-robust criterion using risk measures, two algorithms for optimization.
result Our algorithms produce less conservative solutions than existing methods.
ARHT algorithm improves sparsity guarantees in convex optimization.
problem Optimizing convex functions with sparsity constraints.
method Adaptively Regularized Hard Thresholding (ARHT) algorithm.
result ARHT achieves sparsity bound of γ=O(κ), matching theoretical limits.
Temporal Point Processes (TPP) with partial likelihoods involving a latent structure often entail an intractable marginalization, thus making inference hard. We propose a novel approach to Maximum Likelihood Estimation (MLE) involving approximate inference over the latent variables by minimizing a tight upper bound on …