A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New RGraSP framework for efficient non-convex optimization.
Iterative thresholding algorithms seek to optimize a differentiable objective function over a sparsity or rank constraint by alternating between gradient steps that reduce the objective, and thresholding steps that enforce the constraint. This work examines the choice of the thresholding operator, and asks whether it i…
Hard Thresholding Pursuit (HTP) is an iterative greedy selection procedure for finding sparse solutions of underdetermined linear systems. This method has been shown to have strong theoretical guarantee and impressive numerical performance. In this paper, we generalize HTP from compressive sensing to a generic problem …
IHT improves sparse distribution learning.
AIHT improves online high-dimensional quantile regression by separating support discovery and refinement.
The use of M-estimators in generalized linear regression models in high dimensional settings requires risk minimization with hard constraints. Of the known methods, the class of projected gradient descent (also known as iterative hard thresholding (IHT)) methods is known to offer the fastest and most scalable sol…
We study --both in theory and practice-- the use of momentum motions in classic iterative hard thresholding (IHT) methods. By simply modifying plain IHT, we investigate its convergence behavior on convex optimization criteria with non-convex constraints, under standard assumptions. In diverse scenaria, we observe that …
New algorithm robustly estimates sparse models in high dimensions with corrupted data.
Variable selection in linear models plays a pivotal role in modern statistics. Hard-thresholding methods such as regularization are theoretically ideal but computationally infeasible. In this paper, we propose a new approach, called the LAGS, short for "least absulute gradient selector", to this challenging yet i…
We consider the problem of sparsity-constrained -estimation when both explanatory and response variables have heavy tails (bounded 4-th moments), or a fraction of arbitrary corruptions. We focus on the -sparse, high-dimensional regime where the number of variables and the sample size are related through $…
Developed a new thresholding method that connects soft and hard thresholding.
Iterative Hard Thresholding (IHT) is a class of projected gradient descent methods for optimizing sparsity-constrained minimization models, with the best known efficiency and scalability in practice. As far as we know, the existing IHT-style methods are designed for sparse minimization in primal form. It remains open t…
In this paper we study the performance of the Projected Gradient Descent(PGD) algorithm for -constrained least squares problems that arise in the framework of Compressed Sensing. Relying on the Restricted Isometry Property, we provide convergence guarantees for this algorithm for the entire range of $0\leq p\…
Paper develops algorithms to maximize AUC in imbalanced classification.
The most common method for DNN pruning is hard thresholding of network weights, followed by retraining to recover any lost accuracy. Recently developed smart pruning algorithms use the DNN response over the training set for a variety of cost functions to determine redundant network weights, leading to less accuracy deg…
Noise makes learning linear thresholds hard, but algorithms can still learn near-optimal thresholds.
We develop new stochastic gradient methods for efficiently solving sparse linear regression in a partial attribute observation setting, where learners are only allowed to observe a fixed number of actively chosen attributes per example at training and prediction times. It is shown that the methods achieve essentially a…
Paper analyzes BIHT for noisy 1-bit CS, improving results with up to τ-fraction of incorrect measurements.
Neural networks struggle with learning fixed parities.
This work interprets GELU and related activations via a first-order loss function.
Binary Iterative Hard Thresholding converges with optimal number of 1-bit measurements.
IntHT solves sparse quadratic regression in sub-quadratic time and space.
New algorithm learns sparse GLMs for binary outcomes efficiently.
New algorithm resists contamination in high-dimensional regression with optimal performance.
This paper is concerned with the hard thresholding operator which sets all but the largest absolute elements of a vector to zero. We establish a {\em tight} bound to quantitatively characterize the deviation of the thresholded solution from a given signal. Our theoretical result is universal in the sense that it ho…
Improved iterative hard thresholding for faster, sparser solutions.
Guarantees sparse recovery for neural networks with iterative hard thresholding.
We study the problem of robust linear regression with response variable corruptions. We consider the oblivious adversary model, where the adversary corrupts a fraction of the responses in complete ignorance of the data. We provide a nearly linear time estimator which consistently estimates the true regression vector, e…
Training neural networks is hard in fixed dimensions.
This paper resolves BIHT convergence, showing normalization is not necessary in noiseless settings but crucial for robustness.
Several learning applications require solving high-dimensional regression problems where the relevant features belong to a small number of (overlapping) groups. For very large datasets and under standard sparsity constraints, hard thresholding methods have proven to be extremely efficient, but such methods require NP h…
An algorithmically hard phase was described in a range of inference problems: even if the signal can be reconstructed with a small error from an information theoretic point of view, known algorithms fail unless the noise-to-signal ratio is sufficiently small. This hard phase is typically understood as a metastable bran…
Detects dense subhypergraphs in random hypergraphs using low-degree polynomials.
New method identifies extreme risk propagation in financial networks.
Paper tackles 1-bit compressed sensing, presenting efficient algorithm for sparse signal estimation.
ARHT algorithm improves sparsity guarantees in convex optimization.
Self-paced learning and hard example mining re-weight training instances to improve learning accuracy. This paper presents two improved alternatives based on lightweight estimates of sample uncertainty in stochastic gradient descent (SGD): the variance in predicted probability of the correct class across iterations of …
HARFE approximates sparse additive functions using random features and ridge regression.
Detection of dense cycles in graphs reveals a gap between easy detection and hard recovery.
Paper discusses new stochastic algorithms for sparse signal recovery.
Paper proposes DCT for efficient hybrid parallel training of large recommendation models.
New method trains neural networks with threshold activation functions efficiently.
In this paper, we consider the problem of compressed sensing where the goal is to recover almost all the sparse vectors using a small number of fixed linear measurements. For this problem, we propose a novel partial hard-thresholding operator that leads to a general family of iterative algorithms. While one extreme of …
We propose a nonconvex estimator for joint multivariate regression and precision matrix estimation in the high dimensional regime, under sparsity constraints. A gradient descent algorithm with hard thresholding is developed to solve the nonconvex estimator, and it attains a linear rate of convergence to the true regres…
High-dimensional sparse modeling via regularization provides a powerful tool for analyzing large-scale data sets and obtaining meaningful, interpretable models. The use of nonconvex penalty functions shows advantage in selecting important features in high dimensions, but the global optimality of such methods still dema…
Paper explores limits of high-order clustering with planted structures.
Optimal intervention in economic networks modeled as influence maximization, with hard computational problems.
In many high-dimensional estimation problems the main task consists in minimizing a cost function, which is often strongly non-convex when scanned in the space of parameters to be estimated. A standard solution to flatten the corresponding rough landscape consists in summing the losses associated to different data poin…