Paper improves robust PCA with feature information using iterative hard-thresholding.
problem Separating low-rank and sparse components in data with feature information.
method Iterative hard-thresholding algorithm for robust PCA under weaker assumptions.
result Global convergence and faster convergence rate of the proposed algorithm.
Guarantees sparse recovery for neural networks with iterative hard thresholding.
problem Recovering sparse network weights in neural networks.
method Structural properties of sparse network weights and iterative hard thresholding algorithm.
result Simple iterative hard thresholding algorithm recovers sparse network weights exactly using linear memory.
Improved iterative hard thresholding for faster, sparser solutions.
problem Finding sparser solutions without sacrificing runtime.
method Adaptive regularization framework applied to iterative hard thresholding.
result Returns solutions with sparsity O(sκ), improving over existing methods. IHT improves sparse distribution learning.
problem Learning sparse discrete distributions.
method Iterative hard thresholding as a solution, with a greedy approximate projection.
result IHT achieves state of the art results for sparse distribution learning.
Hard Thresholding Pursuit (HTP) is an iterative greedy selection procedure for finding sparse solutions of underdetermined linear systems. This method has been shown to have strong theoretical guarantee and impressive numerical performance. In this paper, we generalize HTP from compressive sensing to a generic problem …
New algorithm resists contamination in high-dimensional regression with optimal performance.
problem Adversarial and measurement errors in high-dimensional data.
method Adversarial Contamination-resistant Iterative Hard Thresholding (AC-IHT) algorithm.
result Achieves minimax near-optimal estimation and signal-adaptive support recovery.
The use of M-estimators in generalized linear regression models in high dimensional settings requires risk minimization with hard L0 constraints. Of the known methods, the class of projected gradient descent (also known as iterative hard thresholding (IHT)) methods is known to offer the fastest and most scalable sol…
Paper extends tensor recovery method for low CP-rank tensors.
problem Recovery of low-rank tensors from few measurements.
method Iterative Hard Thresholding with tensor version of RIP.
result Exact recovery of tensors with low CP-rank is guaranteed.
AIHT improves online high-dimensional quantile regression by separating support discovery and refinement.
problem Online high-dimensional quantile regression with structural sparsity.
method Adaptive Iterative Hard Thresholding (AIHT) alternates stochastic updates with adaptive hard-thresholding steps.
result AIHT achieves logarithmic regret for the sliding-window objective in high-dimensional settings.
In this paper we consider l0 regularized convex cone programming problems. In particular, we first propose an iterative hard thresholding (IHT) method and its variant for solving l0 regularized box constrained convex programming. We show that the sequence generated by these methods converges to a local minimizer.…
New RGraSP framework for efficient non-convex optimization.
problem Large-scale non-convex sparsity-constrained optimization problems.
method Relaxed gradient support pursuit with semi-stochastic gradient hard thresholding.
result Our algorithms converge faster with lower per-iteration cost.
Paper develops algorithms to maximize AUC in imbalanced classification.
problem Maximizing AUC in imbalanced classification problems.
method Developed stochastic hard thresholding algorithms to reformulate U-statistics as ERM.
result Proposed algorithm achieves linear convergence rate.
Paper analyzes BIHT for noisy 1-bit CS, improving results with up to τ-fraction of incorrect measurements.
problem Estimating sparse vectors from noisy sign measurements in 1-bit compressed sensing.
method Binary Iterative Hard Thresholding (BIHT) algorithm, using Gaussian matrices and high-dimensional geometry analysis.
result BIHT provides estimates within ε+τ error with τ-fraction of incorrect measurements, maintaining universality of measurements.
This paper resolves BIHT convergence, showing normalization is not necessary in noiseless settings but crucial for robustness.
problem Analyzing convergence and robustness of BIHT for 1-bit compressed sensing.
method Characterizes BIHT convergence and robustness, proving necessity of normalization for robustness under sign corruptions.
result Per-iteration normalization is not necessary for optimal recovery in noiseless settings but is crucial for robustness under sign corruptions.
Improved IHT with momentum accelerates convex optimization with non-convex constraints.
problem Optimizing convex criteria with non-convex constraints.
method Modified iterative hard thresholding with momentum.
result Acceleration leads to significant improvements over state-of-the-art methods.
New algorithm learns sparse GLMs for binary outcomes efficiently.
problem Sparse modeling of binary outcomes in high-dimensional data.
method Iterative hard thresholding algorithm (BIHT) for sparse GLMs.
result BIHT achieves statistical optimality for logistic regression.
Paper analyzes IHT's performance in sparse recovery problems.
problem Generalization performance of Iterative Hard Thresholding (IHT).
method Sparse generalization theory under algorithmic stability.
result IHT achieves convergence rates in sparse excess risk.
Optimal iterative thresholding algorithms improve upon hard and soft thresholding.
problem Optimizing sparsity or rank constraints in optimization problems.
method Developed the notion of relative concavity for thresholding operators, finding a new class of operators that are optimal.
result A new class of thresholding operators, including ℓq thresholding and reciprocal thresholding, achieves the strongest convergence guarantee. In this paper, we consider the problem of compressed sensing where the goal is to recover almost all the sparse vectors using a small number of fixed linear measurements. For this problem, we propose a novel partial hard-thresholding operator that leads to a general family of iterative algorithms. While one extreme of …
New algorithm recovers signals from low-precision data in interferometry and imaging.
problem Signal loss in data compression for interferometry and medical imaging.
method Normalized Iterative Hard Thresholding with aggressive quantization.
result Recovery guarantees for low-precision data in compressive sensing.
Novel active learning framework using sparse approximation for efficient model training.
problem Efficient model training with limited labeled data.
method Formulates batch active learning as sparsity-constrained discontinuous optimization problems, using greedy or proximal iterative hard thresholding algorithms.
result Achieves competitive performance with lower computational complexity across different settings.
New methods solve sparse linear regression with limited attribute observation.
problem Sparse linear regression with limited attribute observation.
method Stochastic gradient methods using hard thresholding and adaptive combination of exploration and exploitation.
result Achieves sample complexity of O(1/ε) for error ε under restricted eigenvalue condition.
This paper is concerned with the hard thresholding operator which sets all but the k largest absolute elements of a vector to zero. We establish a {\em tight} bound to quantitatively characterize the deviation of the thresholded solution from a given signal. Our theoretical result is universal in the sense that it ho…
The problem of recovering a low n-rank tensor is an extension of sparse recovery problem from the low dimensional space (matrix space) to the high dimensional space (tensor space) and has many applications in computer vision and graphics such as image inpainting and video inpainting. In this paper, we consider a new …
Binary Iterative Hard Thresholding converges with optimal number of 1-bit measurements.
problem Recovering sparse signals from 1-bit compressed measurements.
method Binary Iterative Hard Thresholding (BIHT) algorithm.
result BIHT converges with only O(k/ε) measurements, optimal for recovery.
ARHT algorithm improves sparsity guarantees in convex optimization.
problem Optimizing convex functions with sparsity constraints.
method Adaptively Regularized Hard Thresholding (ARHT) algorithm.
result ARHT achieves sparsity bound of γ=O(κ), matching theoretical limits.
Novel algorithm recovers sparse parameters in high-dimensional data with constant corruption.
problem Sparse regression with high dimensionality and constant fraction of corruptions.
method Robust Iterative Hard Thresholding, filtering algorithm for outlier removal.
result Near information-theoretically optimal error guarantee with sub-linear sample complexity.
Hard thresholding remains efficient for DNN pruning, but smart pruning offers faster accuracy recovery.
problem Efficiently pruning deep neural networks while minimizing accuracy loss.
method Proposes a novel smart pruning algorithm based on difference of convex functions optimization.
result Smart pruning is often orders of magnitude faster than competing approaches while achieving low accuracy degradation.
IntHT solves sparse quadratic regression in sub-quadratic time and space.
problem Sparse quadratic regression in high-dimensional problems.
method Interaction Hard Thresholding (IntHT) is a variant of Iterative Hard Thresholding tailored for quadratic structures.
result IntHT provably converges to a consistent estimate under high-dimensional sparse recovery assumptions.
A genome-wide association study (GWAS) correlates marker variation with trait variation in a sample of individuals. Each study subject is genotyped at a multitude of SNPs (single nucleotide polymorphisms) spanning the genome. Here we assume that subjects are unrelated and collected at random and that trait values are n…
In this paper we study the performance of the Projected Gradient Descent(PGD) algorithm for ℓp-constrained least squares problems that arise in the framework of Compressed Sensing. Relying on the Restricted Isometry Property, we provide convergence guarantees for this algorithm for the entire range of $0\leq p\…
We develop mask iterative hard thresholding algorithms (mask IHT and mask DORE) for sparse image reconstruction of objects with known contour. The measurements follow a noisy underdetermined linear model common in the compressive sampling literature. Assuming that the contour of the object that we wish to reconstruct i…
Dual IHT algorithm solves NP-hard non-convex sparse minimization problems.
problem Non-convex sparse minimization with ℓ2-regularized loss function. method Developed a dual IHT algorithm for maximizing the non-smooth dual objective.
result Sparse recovery performance is invariant to RIP, superior to primal IHT algorithms.
Developed a new thresholding method that connects soft and hard thresholding.
problem Connecting soft and hard thresholding methods in data analysis.
method Scaled soft thresholding method with empirical scaling values.
result Found two sources of over-fitting in the scaled soft thresholding method.
SCOPE iteratively optimizes sparsity-constrained problems without tuning hyperparameters.
problem Optimizing sparsity-constrained problems in signal processing, statistics, and machine learning.
method SCOPE (Sparsity-Constrained Optimization via sPlicing itEration) replaces gradient steps with a splicing operation guided by the objective value.
result SCOPE achieves linear convergence and superior support recovery performance.
New methods solve graph sparsity optimization problems faster.
problem Complex graph sparsity optimization problems in disease outbreak monitoring and social network analysis.
method Stochastic variance-reduced gradient-based methods GraphSVRG-IHT and GraphSCSG-IHT.
result Our methods achieve linear convergence speed.
New PSDMF algorithms derived from PR and ARM methods.
problem Positive semidefinite matrix factorization (PSDMF) challenges.
method Design PSDMF algorithms based on phase retrieval (PR) and affine rank minimization (ARM) methods.
result New PSDMF algorithms inherit numerical properties from PR and ARM methods.
The paper shows exchanging estimates over networks is effective for learning sparse signals.
problem Learning sparse signals over networks with limited communication.
method Iterative algorithm exchanging intermediate estimates over a network, with theoretical and simulation analysis.
result The iterative algorithm provides competitive performance in learning sparse signals.
New method for robust regression with near-optimal performance even with high corruption rates.
problem Robust linear regression with response variable corruptions.
method Adaptive hard thresholding for consistent estimation.
result Near-optimal consistent estimation of the true regression vector with 1−o(1) fraction of corruptions. New method solves graph-structured sparsity problems efficiently.
problem Graph-structured sparsity optimization in complex models.
method Stochastic gradient-based approach for non-convex graph-structured sparsity.
result Linear convergence up to a constant error.
New algorithm robustly estimates sparse models in high dimensions with corrupted data.
problem Estimating latent variable models with arbitrarily corrupted samples in high dimensional space.
method Trimmed (Gradient) Expectation Maximization with trimming gradients and hard thresholding steps.
result The algorithm converges to near optimal statistical rate geometrically under certain conditions.
In this paper, we generalize Huber's criterion to multichannel sparse recovery problem of complex-valued measurements where the objective is to find good recovery of jointly sparse unknown signal vectors from the given multiple measurement vectors which are different linear combinations of the same known elementary vec…
New method identifies extreme risk propagation in financial networks.
problem Understanding extreme risk in financial networks.
method Max-linear structural equation model, hard-thresholding, Hamming distance.
result Sparse DAG for extreme risk propagation estimated.
Paper tackles 1-bit compressed sensing, presenting efficient algorithm for sparse signal estimation.
problem Estimating sparse signals from binary measurements.
method Non-convex sparsity-constrained program with one-shot hard thresholding.
result Simple algorithm produces accurate signal approximation with high probability.
Variable selection in linear models plays a pivotal role in modern statistics. Hard-thresholding methods such as l0 regularization are theoretically ideal but computationally infeasible. In this paper, we propose a new approach, called the LAGS, short for "least absulute gradient selector", to this challenging yet i…
Online method learns sparse models efficiently in large scale settings.
problem Sparse model learning in large scale settings with high computational and memory costs.
method Online learning approach, mini-batch methods, hard thresholding based stochastic gradient algorithm.
result Sparsity promoted by batch methods is not preserved in online fashion.
A neural network, IHT-Net, improves DOA estimation with sparse arrays.
problem Single-snapshot DOA estimation with sparse arrays in dynamic settings.
method IHT-inspired neural network with recurrent neural network and autoencoders.
result IHT-Net achieves faster convergence and higher accuracy in DOA estimation.
Sparse reconstruction approaches using the re-weighted l1-penalty have been shown, both empirically and theoretically, to provide a significant improvement in recovering sparse signals in comparison to the l1-relaxation. However, numerical optimization of such penalties involves solving problems with l1-norms in the ob…