arXiv research

New adaptive learning rate improves FTRL's adaptivity to sparsity, game-dependency, and best-of-both-worlds.

Proposes a tail-adaptive shrinkage method for robust sparse estimation.

HALO learns to prune neural networks by adaptively shrinking weights.

Study on signal detection in sparse additive models with nonasymptotic minimax rates.

We propose two sparsity-aware normalized subband adaptive filter (NSAF) algorithms by using the gradient descent method to minimize a combination of the original NSAF cost function and the l1-norm penalty function on the filter coefficients. This l1-norm penalty exploits the sparsity of a system in the coefficients upd…

SpeqNets improve graph neural networks by scaling and adapting to graph sparsity.

A new filter design improves system identification accuracy.

Paper proposes efficient pruning method for neural networks.

Attention mechanisms have become ubiquitous in NLP. Recent architectures, notably the Transformer, learn powerful context-aware word representations through layered, multi-headed attention. The multiple heads learn diverse types of word relationships. However, with standard softmax attention, all attention heads are de…

We investigate filter level sparsity that emerges in convolutional neural networks (CNNs) which employ Batch Normalization and ReLU activation, and are trained with adaptive gradient descent techniques and L2 regularization or weight decay. We conduct an extensive experimental study casting our initial findings into hy…

Recent breakthrough results in compressed sensing (CS) have established that many high dimensional objects can be accurately recovered from a relatively small number of non- adaptive linear projection observations, provided that the objects possess a sparse representation in some basis. Subsequent efforts have shown th…

AGS-CL selectively updates penalties based on node importance for continual learning.

HS-MoE selects sparse experts using adaptive priors and data-adaptive gating.

GRASP simplifies Bayesian regression with grouped predictors using an adaptive NBP prior.

Study finds exact limits for sparse regression with fewer observations than usual.

While variational dropout approaches have been shown to be effective for network sparsification, they are still suboptimal in the sense that they set the dropout rate for each neuron without consideration of the input data. With such input-independent dropout, each neuron is evolved to be generic across inputs, which m…

Improved iterative hard thresholding for faster, sparser solutions.

ARHT algorithm improves sparsity guarantees in convex optimization.

We propose a new sparsity-smoothness penalty for high-dimensional generalized additive models. The combination of sparsity and smoothness is crucial for mathematical theory as well as performance for finite-sample data. We present a computationally efficient algorithm, with provable numerical convergence properties, fo…

This paper improves GNN efficiency for large-scale graph applications.

New method for multiclass classification reduces error bounds.

Sparsity-based approaches have been popular in many applications in image processing and imaging. Compressed sensing exploits the sparsity of images in a transform domain or dictionary to improve image recovery from undersampled measurements. In the context of inverse problems in dynamic imaging, recent research has de…

New algorithm tackles high-dimensional contextual bandits without sparsity.

Paper analyzes adaptive Lasso for high-dimensional diffusion processes, improving support recovery and bias.

New method reduces regret for sparse adversarial SSP problems.

Proposes ARSK for robust and sparse clustering.

We consider the setting of linear regression in high dimension. We focus on the problem of constructing adaptive and honest confidence sets for the sparse parameter θ, i.e. we want to construct a confidence set for theta that contains theta with high probability, and that is as small as possible. The l_2 diameter of a …

Bayesian model infers factor dimensionality and sparse loading matrix adaptively.

Theoretical framework for neural network compression using sparsity norms.

2L-FUSE enhances feature sparsity through kernel learning.

A new method reduces bias in adaptive Lasso estimates.

Automated PDE discovery from multiple noisy experiments.

Compressive sensing (CS) exploits sparsity to recover sparse or compressible signals from dimensionality reducing, non-adaptive sensing mechanisms. Sparsity is also used to enhance interpretability in machine learning and statistics applications: While the ambient dimension is vast in modern data analysis problems, the…

Novel regularization for Vision Transformers improves model generalization and sparsity.

Medical image reconstruction is typically an ill-posed inverse problem. In order to address such ill-posed problems, the prior distribution of the sought after object property is usually incorporated by means of some sparsity-promoting regularization. Recently, prior distributions for images estimated using generative …

PrAda-GAN improves synthetic data generation under differential privacy.

In this paper, we focus on online representation learning in non-stationary environments which may require continuous adaptation of model architecture. We propose a novel online dictionary-learning (sparse-coding) framework which incorporates the addition and deletion of hidden units (dictionary elements), and is inspi…

We present a probabilistic framework for both (i) determining the initial settings of kernel adaptive filters (KAFs) and (ii) constructing fully-adaptive KAFs whereby in addition to weights and dictionaries, kernel parameters are learnt sequentially. This is achieved by formulating the estimator as a probabilistic mode…

The paper studies the asymptotic behavior of adversarial training under $\ell_\infty$-perturbation.

BiSHop tackles tabular data challenges with sparse Hopfield layers.

MT-HAL learns features and task associations for multiple tasks with a shared sparse structure.

The paper decouples shrinkage and selection in Bayesian Quantile Regression.

Spectral variability is one of the major issue when conducting hyperspectral unmixing. Within a given image composed of some elementary materials (herein referred to as endmember classes), the spectral signature characterizing these classes may spatially vary due to intrinsic component fluctuations or external factors …

The paper analyzes sparse high-dimensional linear regression with random design and unknown error variance, providing adaptiveness and concentration rates.

The paper improves on existing algorithms for minimizing different types of regret in online learning.

Paper improves robustness and sparsity in adversarially trained DNNs.

Recently, a number of mostly $\ell_1$-norm regularized least squares type deterministic algorithms have been proposed to address the problem of \emph{sparse} adaptive signal estimation and system identification. From a Bayesian perspective, this task is equivalent to maximum a posteriori probability estimation under a …

In high dimensional settings, sparse structures are crucial for efficiency, either in term of memory, computation or performance. In some contexts, it is natural to handle more refined structures than pure sparsity, such as for instance group sparsity. Sparse-Group Lasso has recently been introduced in the context of l…