New interactive greedy algorithm for group sparsity in high dimensions.
problem Benefits of group sparsity for greedy-type methods in high-dimensional data analysis.
method Interactive Greedy Approach
result Proposed algorithm attains desired benefits of group sparsity under high dimensional settings.
New algorithm reduces hyperparameter search space using group sparsity.
problem Efficient hyperparameter selection in machine learning.
method Modifies Harmonica algorithm with group-sparse recovery and HyperBand.
result Improves over existing methods like Successive Halving and Random Search.
Gradient descent implicitly favors group sparsity in neural networks.
problem Understanding implicit regularization in neural networks for structured sparsity.
method Novel neural reparameterization for diagonally grouped linear networks.
result Gradient descent without explicit regularization biases towards group sparsity.
This paper develops a theory for group Lasso using a concept called strong group sparsity. Our result shows that group Lasso is superior to standard Lasso for strongly group-sparse signals. This provides a convincing theoretical justification for using group sparse regularization when the underlying group structure is …
Automated PDE discovery from multiple noisy experiments.
problem Inherent variability in experiments makes single experiment inference unreliable.
method Randomised adaptive group Lasso sparsity estimator in deep learning framework.
result More generalizable PDEs found from multiple datasets.
SNAM improves NAM's accuracy and feature selection via group sparsity.
problem Improving interpretability and accuracy in deep learning models.
method Employing group sparsity regularization in neural additive models (SNAM).
result SNAM provably converges to zero training loss and achieves exact support recovery.
New method selects variables in groups with few nonzeros, improving support recovery.
problem Structured variable selection with sparse patterns across groups.
method Composite norm and proximal algorithm for exclusive group sparsity.
result Asymptotic consistency in signed support recovery under conventional assumptions.
New method finds sparse groups of input variables for neural networks.
problem Finding optimal groups of input variables for neural networks.
method Developed a new loss function and optimization algorithm for multi-layer non-linear neural networks to achieve group sparsity.
result Achieved group sparsity in three real-world datasets, improving model performance and excluding a significant number of variables.
AGS-CL selectively updates penalties based on node importance for continual learning.
problem Catastrophic forgetting in continual learning.
method Adaptive Group Sparsity (AGS) with proximal gradient descent.
result Significantly outperforms baselines on various continual learning benchmarks.
Develops a new attack model to better capture structural information in adversarial examples.
problem Lp norm-based adversarial attacks fail to capture structural information in input images.
method Structured Adversarial Attack (StrAttack) using ADMM framework to achieve strong group sparsity.
result StrAttack achieves strong group sparsity in adversarial perturbations with similar Lp norm distortion.
Optimal PCA formulation for sparse loadings.
problem Sparse loadings in PCA with optimal variance.
method New variance definition, group-ℓ1 regularization, analytical solution. result GSMV formulation produces better and more robust results.
New theorem for generalized group sparsity improves consistency and convergence rates.
problem Improving statistical inference in high-dimensional data with element-wise and group-wise sparsity.
method Developed a generalized version of Sparse-Group Lasso and proved a universal theorem for consistency and convergence rates.
result Obtained results on consistency and convergence rates for different forms of double sparsity regularization.
Transformers can learn optimal variable selection in group-sparse classification.
problem Understanding how transformers leverage attention to select relevant variables in group-sparse classification.
method Training a one-layer transformer using gradient descent to select variables from one group of input variables.
result A one-layer transformer can correctly leverage the attention mechanism to select variables, disregarding irrelevant ones.
Paper improves group sparse recovery bounds using ℓ1-norm minimization.
problem Recovering group sparse vectors from few measurements.
method Introduces GRNSP and GRIP, and uses convex relaxations.
result New bounds for group sparse recovery, including equal and unequal group sizes.
Recent results in Compressive Sensing have shown that, under certain conditions, the solution to an underdetermined system of linear equations with sparsity-based regularization can be accurately recovered by solving convex relaxations of the original problem. In this work, we present a novel primal-dual analysis on a …
We consider the problem of sparse variable selection in nonparametric additive models, with the prior knowledge of the structure among the covariates to encourage those variables within a group to be selected jointly. Previous works either study the group sparsity in the parametric setting (e.g., group lasso), or addre…
New method learns under latent group sparsity using network dynamics.
problem Sparse learning under latent group structure without prior group information.
method Heat-flow-based local network dynamics incorporating Laplacian geometry.
result Automatic interpolation between lasso and group lasso penalties.
Localized Lasso improves interpretable models for high-dimensional data.
problem High-dimensional regression with small sample size and interpretability.
method Sample-wise network regularization and exclusive group sparsity.
result Localized Lasso outperforms alternatives in simulated and genomic data.
New algorithm for tensor factorization with outlying slabs.
problem Factoring low-rank tensors with outlying slabs in real-world data.
method Group-sparsity promoting formulation and alternating optimization framework.
result Proposed algorithm converges and performs well on various real-world data.
Unified approach for error bounds in group sparse compressed sensing.
problem Error bounds for group sparse vectors in compressed sensing.
method Unified approach using decomposable and γ-decomposable norms.
result Bounds for various group sparse norms derived.
New method fuses audio and magnetic data to identify underlying subspaces.
problem Identifying complex trends in multi-modality data.
method Robust Group Subspace Recovery (RoGSuRe) algorithm based on group sparsity and bi-sparsity pursuit.
result Competitive performance in clustering and classification of multi-modal data.
New model captures time series dependence across and within blocks.
problem Complex multivariate time series dependence structures.
method Time series Gaussian chain graph models with directed and undirected edges.
result Consistent recovery of time series chain graph structure.
Bayesian method improves dictionary learning for complex problems.
problem Efficiently identifying relevant dictionary entries for complex inverse problems.
method Bayesian group sparsity coding and deflation steps to compress and identify relevant subdictionaries.
result Significant computational complexity reduction and improved glitch detection in LIGO experiment.
Proposes a linear model for facial action recognition without requiring large datasets.
problem Limited annotated data for facial expression and action units.
method Exploits low-rank property across frames and group sparsity to subtract neutral faces and recognize actions.
result One-shot automatic method on raw face videos performs competitively and better than previous methods.
In multi-label learning, each sample is associated with several labels. Existing works indicate that exploring correlations between labels improve the prediction performance. However, embedding the label correlations into the training process significantly increases the problem size. Moreover, the mapping of the label …
A new multi-task learning estimator improves Gaussian graphical regression model fitting.
problem High error rate in fitting Gaussian graphical regression models due to separate node-wise lasso regressions.
method Proposes a multi-task learning estimator with cross-task group sparsity and within-task element-wise sparsity penalties, solved via an efficient augmented Lagrangian algorithm.
result Error rate improvement over separate node-wise lasso estimates, demonstrated through simulations and application to gene co-expression network study.
New method learns latent group structures without clustering, using heat flow dynamics.
problem Learning with latent group sparsity in machine learning problems.
method Heat flow dynamics on network structure to incorporate group structure.
result Effective performance and provable bounds on sample complexity.
Statistical dependencies among wavelet coefficients are commonly represented by graphical models such as hidden Markov trees(HMTs). However, in linear inverse problems such as deconvolution, tomography, and compressed sensing, the presence of a sensing or observation matrix produces a linear mixing of the simple Markov…
Popular sparse estimation methods based on ℓ1-relaxation, such as the Lasso and the Dantzig selector, require the knowledge of the variance of the noise in order to properly tune the regularization parameter. This constitutes a major obstacle in applying these methods in several frameworks---such as time series, …
We propose a novel SPARsity and Clustering (SPARC) regularizer, which is a modified version of the previous octagonal shrinkage and clustering algorithm for regression (OSCAR), where, the proposed regularizer consists of a K-sparse constraint and a pair-wise ℓ∞ norm restricted on the K largest componen…
We consider the prediction of weak effects in a multiple-output regression setup, when covariates are expected to explain a small amount, less than ≈1, of the variance of the target variables. To facilitate the prediction of the weak effects, we constrain our model structure by introducing a novel Bayesian ap…
New screening rules speed up Sparse-Group Lasso solving.
problem Efficiently solving Sparse-Group Lasso in high dimensions.
method Adapted safe screening rules for Sparse-Group Lasso.
result Significant speed-ups in computing time for coordinate descent.
Paper proposes a method to recover task groups and sparsity patterns in sparse learning.
problem Learning task structure when multiple tasks share relevant features.
method Formulates a joint optimization problem to recover task groups and sparsity patterns, encouraging sparse learning and correct recovery of task groups.
result Proposed method accurately recovers task groups and sparsity patterns in task parameters.
Nonnegative matrix factorization (NMF) with group sparsity constraints is formulated as a probabilistic graphical model and, assuming some observed data have been generated by the model, a feasible variational Bayesian algorithm is derived for learning model parameters. When used in a supervised learning scenario, NMF …
High-dimensional data pose challenges in statistical learning and modeling. Sometimes the predictors can be naturally grouped where pursuing the between-group sparsity is desired. Collinearity may occur in real-world high-dimensional applications where the popular l1 technique suffers from both selection inconsisten…
We compress large neural networks for quick adaptation to specific contexts.
problem How to quickly adapt a pretrained large neural network to specific contexts.
method Propose a Bayesian hypernetwork framework to compress the network and encourage sparsity.
result Generated compressed networks are significantly smaller than baseline methods.
Efficiently learns high-dimensional SIMs with structural constraints.
problem Estimating feature weights and nonlinear function in high-dimensional SIMs.
method Proposes computationally efficient algorithms for high-dimensional SIM inference with structural constraints.
result Superior predictive performance compared to generalized linear models and neural networks.
Paper proposes a new sparse group k-max regularization for sparsity constraints.
problem Linear inverse problems with sparsity constraints are NP-hard.
method Sparse group k-max regularization, iterative soft thresholding algorithm.
result Approximates l0 norm more closely and enhances group-wise and in-group sparsity.
Proposes a boosting framework for sparsity in grouped covariates.
problem Sparsity and selection bias in grouped covariates.
method Component-wise and group-wise gradient boosting with adjusted degrees of freedom.
result Reduces bias and improves predictability in variable selection.
Paper learns Granger causality for Hawkes processes using sparse group lasso.
problem Challenges in learning Granger causality for general point processes.
method Modeling impact functions with basis functions and using group sparsity for Granger causality graph recovery.
result Successfully learns Granger causality graph and triggering patterns of Hawkes processes.
Improved regret bounds for structured linear contextual bandits with Gaussian noise.
problem Optimizing bandit learning algorithms for structured contexts with Gaussian perturbations.
method Proposed simple greedy algorithms for structured linear contextual bandits with Gaussian noise.
result Unified regret analysis for structured parameters with geometric quantities as bounds.
Group-based sparsity models are proven instrumental in linear regression problems for recovering signals from much fewer measurements than standard compressive sensing. The main promise of these models is the recovery of "interpretable" signals through the identification of their constituent groups. In this paper, we e…
Efficiently solves high-dimensional regression with overlapping groups using greedy hard-thresholding.
problem High-dimensional regression problems with overlapping groups of relevant features.
method Greedy hard-thresholding combined with submodular optimization to avoid NP-hard projections.
result Strong theoretical guarantees even with poorly conditioned data and overlapping features.
We develop a highly scalable optimization method called "hierarchical group-thresholding" for solving a multi-task regression model with complex structured sparsity constraints on both input and output spaces. Despite the recent emergence of several efficient optimization algorithms for tackling complex sparsity-induci…
Improves robustness of high-dimensional regression with rank objective and group lasso regularization.
problem Heavy-tailed noise and outliers in high-dimensional regression.
method Non-smooth Wilcoxon score based rank objective, group lasso regularization, data-driven tuning rule, proximal augmented Lagrangian method.
result Robust estimator with finite-sample error bound and efficient computational method.
SR3 framework improves sparse regression solutions.
problem Sparse regression problems in various fields.
method SR3 framework solves relaxed regularized regression problems.
result SR3 provides superior solutions with faster algorithms.
Adam optimizes DNNs to induce weight sparsity.
problem Large DNN models for edge devices.
method Using Adam optimizer with ReLU activations and L2 regularization.
result Deep learning automatically induces group sparsity of weights.
Path regularization reveals convex optimization in deep ReLU networks.
problem Understanding the optimization landscape of deep neural networks.
method Introducing path regularization to make the training problem convex and sparsity-inducing.
result Path regularized parallel ReLU networks are a parsimonious convex model in high dimensions.