arXiv research

Optimal iterative thresholding algorithms improve upon hard and soft thresholding.

Paper analyzes adaptive ISTA with MAD for LASSO problem.

Paper finds a fast method for a matrix norm proximal operator.

Classifiers based on sparse representations have recently been shown to provide excellent results in many visual recognition and classification tasks. However, the high cost of computing sparse representations at test time is a major obstacle that limits the applicability of these methods in large-scale problems, or in…

Paper proposes a new sparse group k-max regularization for sparsity constraints.

DR-NMF uses unfolded ISTA for speech separation, offering interpretability and speed.

Sparse coding is a core building block in many data analysis and machine learning pipelines. Typically it is solved by relying on generic optimization techniques, such as the Iterative Soft Thresholding Algorithm and its accelerated version (ISTA, FISTA). These methods are optimal in the class of first-order methods fo…

Developed a new thresholding method that connects soft and hard thresholding.

New multi-layer algorithm improves CNN performance.

Adaptive neural network improves MIMO detection on real-world channels.

In this paper we study the performance of the Projected Gradient Descent(PGD) algorithm for $\ell_{p}$-constrained least squares problems that arise in the framework of Compressed Sensing. Relying on the Restricted Isometry Property, we provide convergence guarantees for this algorithm for the entire range of $0\leq p\…

STR reparameterizes DNN weights with soft thresholds for better sparsity and accuracy.

Paper analyzes iterates in high-dimensional linear models and proposes estimators for their generalization error.

New empirical Bayes estimator outperforms soft-thresholding for high-dimensional sparse vectors.

New method improves sampling from logconcave distributions truncated on polytopes.

This paper concerns the problem of matrix completion, which is to estimate a matrix from observations in a small subset of indices. We propose a calibrated spectrum elastic net method with a sum of the nuclear and Frobenius penalties and develop an iterative algorithm to solve the convex minimization problem. The itera…

Ada-LISTA adapts neural solvers for varying models.

Recurrent neural networks (RNNs) are powerful and effective for processing sequential data. However, RNNs are usually considered "black box" models whose internal structure and learned parameters are not interpretable. In this paper, we propose an interpretable RNN based on the sequential iterative soft-thresholding al…

New convergence analysis for Lasso l1 reweighting improves practical performance.

We consider solving the $\ell_1$-regularized least-squares ($\ell_1$-LS) problem in the context of sparse recovery, for applications such as compressed sensing. The standard proximal gradient method, also known as iterative soft-thresholding when applied to this problem, has low computational cost per iteration but a r…

The article examines different thresholding methods for improving PAM algorithm in cancer classification.

CV inference can be invalid for relatively unstable model comparisons.

Improved neural network reconstruction from sparse measurements with theoretical guarantees.

It has recently been observed that certain extremely simple feature encoding techniques are able to achieve state of the art performance on several standard image classification benchmarks including deep belief networks, convolutional nets, factored RBMs, mcRBMs, convolutional RBMs, sparse autoencoders and several othe…

Unified analysis of neural networks for sparse signal recovery.

Sparse coding is a core building block in many data analysis and machine learning pipelines. Typically it is solved by relying on generic optimization techniques, that are optimal in the class of first-order methods for non-smooth, convex functions, such as the Iterative Soft Thresholding Algorithm and its accelerated …

DeepAM optimizes deep neural networks for image super-resolution.

Sparse reconstruction approaches using the re-weighted l1-penalty have been shown, both empirically and theoretically, to provide a significant improvement in recovering sparse signals in comparison to the l1-relaxation. However, numerical optimization of such penalties involves solving problems with l1-norms in the ob…

Heavy Lasso improves robustness in high-dimensional linear regression with heavy-tailed errors.

A new method for sparse PCA using orthogonal rotations and soft-thresholding.

We present a theoretical analysis and empirical evaluations of a novel set of techniques for computational cost reduction of classifiers that are based on learned transform and soft-threshold. By modifying optimization procedures for dictionary and classifier training, as well as the resulting dictionary entries, our t…

Study various image defenses against adversarial attacks using basis functions.

Optimal control of reserve assets for stablecoins to maintain peg stability.

DNF-Net tackles tabular data challenges with neural architecture.

We study confidence intervals based on hard-thresholding, soft-thresholding, and adaptive soft-thresholding in a linear regression model where the number of regressors $k$ may depend on and diverge with sample size $n$. In addition to the case of known error variance, we define and study versions of the estimators when…

New models explain residual and dilated dense neural networks using sparse coding.

The total variation (TV) penalty, as many other analysis-sparsity problems, does not lead to separable factors or a proximal operatorwith a closed-form expression, such as soft thresholding for the $\ell\_1$ penalty. As a result, in a variational formulation of an inverse problem or statisticallearning estimation, it l…

Information spreads across social and technological networks, but often the network structures are hidden from us and we only observe the traces left by the diffusion processes, called cascades. Can we recover the hidden network structures from these observed cascades? What kind of cascades and how many cascades do we …

A new sampling method for log-concave distributions with warm starts and barriers.

New algorithm samples from log-concave distributions over polytopes efficiently.

This paper develops a novel deep recurrent neural network for sequential signal reconstruction.

Paper proposes methods to handle missing values in clinical data.

We propose a robust elastic net (REN) model for high-dimensional sparse regression and give its performance guarantees (both the statistical error bound and the optimization bound). A simple idea of trimming the inner product is applied to the elastic net model. Specifically, we robustify the covariance matrix by trimm…

Proposes a method to propagate uncertainty in neural networks for sparse coding.

New kernel connects deep learning to optimization, improving generalization.

This note studies a method for the efficient estimation of a finite number of unknown parameters from linear equations, which are perturbed by Gaussian noise. In case the unknown parameters have only few nonzero entries, the proposed estimator performs more efficiently than a traditional approach. The method consists o…

We study the distribution of hard-, soft-, and adaptive soft-thresholding estimators within a linear regression model where the number of parameters k can depend on sample size n and may diverge with n. In addition to the case of known error-variance, we define and study versions of the estimators when the error-varian…

Stochastic gradient descent (SGD) is commonly used for optimization in large-scale machine learning problems. Langford et al. (2009) introduce a sparse online learning method to induce sparsity via truncated gradient. With high-dimensional sparse data, however, the method suffers from slow convergence and high variance…