DFR reduces the computational cost of sparse-group lasso and adaptive sparse-group lasso.
problem Sparse-group lasso's computational expense and need for tuning.
method Dual Feature Reduction (DFR) using strong screening rules and dual norms.
result DFR drastically reduces computational cost without affecting solution optimality.
Sparse group Lasso optimizes sparse and grouped parameters in high-dimensional data.
problem Simultaneously sparse and grouped parameters in high-dimensional linear regression.
method Sparse group Lasso, debiased sparse group Lasso, statistical inference.
result Matching upper and lower bounds on sample complexity and estimation error.
This paper develops a theory for group Lasso using a concept called strong group sparsity. Our result shows that group Lasso is superior to standard Lasso for strongly group-sparse signals. This provides a convincing theoretical justification for using group sparse regularization when the underlying group structure is …
The sparse group lasso optimization problem is solved using a coordinate gradient descent algorithm. The algorithm is applicable to a broad class of convex loss functions. Convergence of the algorithm is established, and the algorithm is used to investigate the performance of the multinomial sparse group lasso classifi…
New bounds for Lasso and Group Lasso in high dimensions derived.
problem Estimation error bounds for Lasso and Group Lasso in high-dimensional settings.
method Recent advances in high-dimensional statistics to derive new L2 estimation upper bounds.
result Bounds match optimal minimax rate for Lasso and improve over existing results for Group Lasso.
In high dimensional settings, sparse structures are crucial for efficiency, either in term of memory, computation or performance. In some contexts, it is natural to handle more refined structures than pure sparsity, such as for instance group sparsity. Sparse-Group Lasso has recently been introduced in the context of l…
Classification with a sparsity constraint on the solution plays a central role in many high dimensional machine learning applications. In some cases, the features can be grouped together so that entire subsets of features can be selected or not selected. In many applications, however, this can be too restrictive. In th…
Improves Group Lasso for categorical data by reducing dimensionality and selecting models.
problem Sparse modelling of categorical data is challenging, especially for high dimensions.
method Two-step procedure: first, reduce dimensionality using Group Lasso; second, select final model using an information criterion on clustered levels.
result The method produces a sparse solution and performs better than state-of-the-art algorithms in prediction accuracy and model dimension.
New theorem for generalized group sparsity improves consistency and convergence rates.
problem Improving statistical inference in high-dimensional data with element-wise and group-wise sparsity.
method Developed a generalized version of Sparse-Group Lasso and proved a universal theorem for consistency and convergence rates.
result Obtained results on consistency and convergence rates for different forms of double sparsity regularization.
New group-sparse SVD models improve biclustering of gene expression data.
problem Identifying block patterns with similar expressions in high-dimensional gene expression data.
method Proposed GL1-SVD, GL0-SVD, OGL1-SVD, and OGL0-SVD models with group Lasso and L0-norm penalties, using alternating iterative strategies and ADMM.
result Effective in identifying biologically interpretable gene modules with gene prior group knowledge.
New guarantees for Group LASSO in sparse convex optimization.
problem Sparse convex optimization with vector-valued features.
method Group LASSO regularization and analysis of gradient norms.
result Group LASSO selects the same features as Orthogonal Matching Pursuit.
A new model identifies genetic risk factors using gene-level priors.
problem Identifying genetic risk factors from nucleotide-level genetic variants.
method Sparse Group Lasso with Group-level Graph structure (SGLGG) model.
result SGLGG effectively identifies phenotype-associated risk SNPs.
The group lasso is a penalized regression method, used in regression problems where the covariates are partitioned into groups to promote sparsity at the group level. Existing methods for finding the group lasso estimator either use gradient projection methods to update the entire coefficient vector simultaneously at e…
Paper introduces machine learning for time series data, improving nowcasting accuracy.
problem Improving accuracy in nowcasting US GDP growth using machine learning.
method Sparse-group LASSO estimator for high-dimensional time series data, considering different sampling frequencies and financial/macroeconomic data tail properties.
result Sparse-group LASSO outperforms unstructured LASSO in nowcasting US GDP growth.
In this paper we introduce a new optimization formulation for sparse regression and compressed sensing, called CLOT (Combined L-One and Two), wherein the regularizer is a convex combination of the ℓ1- and ℓ2-norms. This formulation differs from the Elastic Net (EN) formulation, in which the regularizer is a…
Standardization boosts sparse group lasso performance for categorical variables.
problem Improving performance of sparse group lasso with categorical variables.
method Column-wise scaling and re-scaling of coefficients, without orthonormalization.
result Column-wise scaling and re-scaling of coefficients achieves the same effect as orthonormalization.
We present reconstruction algorithms for smooth signals with block sparsity from their compressed measurements. We tackle the issue of varying group size via group-sparse least absolute shrinkage selection operator (LASSO) as well as via latent group LASSO regularizations. We achieve smoothness in the signal via fusion…
Study high-dimensional Granger causality tests for VIX and financial news.
problem Testing Granger causality in high-dimensional time series data.
method Regularized regressions, sparse-group LASSO, HAC estimation of variance.
result Valid time series inference for Granger causality tests in high dimensions.
A new method solves large-scale sparse group square-root Lasso problems efficiently.
problem Large-scale linearly constrained sparse group square-root Lasso problems.
method Dual semismooth Newton based augmented Lagrangian method (ALM).
result The proposed method efficiently solves the problem with numerical experiments demonstrating its effectiveness.
We introduce a recursive adaptive group lasso algorithm for real-time penalized least squares prediction that produces a time sequence of optimal sparse predictor coefficient vectors. At each time index the proposed algorithm computes an exact update of the optimal ℓ1,∞-penalized recursive least squares (R…
Proposes an algorithm for infinite-dimensional sparse learning in system identification.
problem System identification without known model structures.
method Atomic norm regularization and greedy algorithm for solving an infinite-dimensional group lasso problem.
result The proposed algorithm outperforms benchmark methods in impulse response fitting and pole location estimation.
In compressed sensing, in order to recover a sparse or nearly sparse vector from possibly noisy measurements, the most popular approach is ℓ1-norm minimization. Upper bounds for the ℓ2- norm of the error between the true and estimated vectors are given in [1] and reviewed in [2], while bounds for the $\ell_…
The paper improves machine learning for heavy-tailed panel data.
problem Improving estimates for financial and economic data with fat tails.
method Sparse-group LASSO regularization and Fuk-Nagaev concentration inequality.
result Oracle inequalities for panel data estimators.
Paper develops a new estimator for high-dimensional panel data with common shocks.
problem Cross-sectionally dependent errors driven by common shocks in high-dimensional panel data.
method Factor-augmented sparse-group LASSO estimator combining MIDAS aggregation with latent factors.
result The estimator outperforms standard LASSO for prediction and estimation in settings with cross-sectional dependence.
A new method for sparse regression using principal components.
problem Wide data with many features and few observations.
method Combines lasso (ℓ1) sparsity with quadratic penalty towards principal components. result Powerful feature selection and group-wise shrinkage.
Sparse group matrix completion reduces complexity and improves performance.
problem Matrix completion with non-informative side features.
method Group-Lasso regularization for feature selection in matrix factorization.
result Theoretical sample complexity is significantly lower than competitors.
A new estimator learns sparse linear models with context-dependent coefficients.
problem Sparse linear models lack flexibility compared to deep neural networks for handling feature groups.
method Contextual lasso estimator using a deep neural network with lasso regularization.
result Learned models can be sparser than standard lasso without sacrificing predictive power.
New method for inferring time series graph from sparse-group log-sum penalty.
problem Inferring conditional independence graph from high-dimensional stationary multivariate Gaussian time series.
method Sparse-group log-sum penalty (LSP) and alternating direction method of multipliers (ADMM) for iterative optimization.
result Local convergence of inverse PSD estimators to the true value with rate of convergence.
We study a norm for structured sparsity which leads to sparse linear predictors whose supports are unions of prede ned overlapping groups of variables. We call the obtained formulation latent group Lasso, since it is based on applying the usual group Lasso penalty on a set of latent variables. A detailed analysis of th…
In this paper, we introduce Adaptive Cluster Lasso(ACL) method for variable selection in high dimensional sparse regression models with strongly correlated variables. To handle correlated variables, the concept of clustering or grouping variables and then pursuing model fitting is widely accepted. When the dimension is…
In a recent paper, it is shown that the LASSO algorithm exhibits "near-ideal behavior," in the following sense: Suppose y=Az+η where A satisfies the restricted isometry property (RIP) with a sufficiently small constant, and ∥η∥2≤ε. Then minimizing ∥z∥1 subject to $\Vert y - Az \Ver…
Sparse modeling is a powerful framework for data analysis and processing. Traditionally, encoding in this framework is performed by solving an L1-regularized linear regression problem, commonly referred to as Lasso or Basis Pursuit. In this work we combine the sparsity-inducing property of the Lasso model at the indivi…
Exclusive Group Lasso improves feature selection in correlated biological data.
problem Correlated features hinder Lasso performance in biological classification problems.
method Proposes and solves the exclusive group Lasso, combining stability selection and random group allocation.
result Exclusive Group Lasso outperforms Lasso in comprehensive selection of informative features.
Proposes a boosting framework for sparsity in grouped covariates.
problem Sparsity and selection bias in grouped covariates.
method Component-wise and group-wise gradient boosting with adjusted degrees of freedom.
result Reduces bias and improves predictability in variable selection.
Joint sparsity offers powerful structural cues for feature selection, especially for variables that are expected to demonstrate a "grouped" behavior. Such behavior is commonly modeled via group-lasso, multitask lasso, and related methods where feature selection is effected via mixed-norms. Several mixed-norm based spar…
Bayesian Lasso Sparse model provides sparse estimates in linear and nonlinear regression.
problem Sparse learning in regression models.
method Develops a new sparse learning model using type-II maximum likelihood procedure.
result The BLS model provides sparse estimates and is more precise, especially with noisy data.
The paper improves support recovery guarantees for the group Lasso.
problem Estimating group-sparse signals from noisy measurements.
method Establishes new conditions for accurate group-level support estimation using the group Lasso.
result Allows for nearly as many recoverable nonzero groups as the total number of groups.
New algorithm efficiently learns sparse causal graphs from time series data.
problem Learning sparse causal graphs from time series data efficiently and automatically selecting the number of edges.
method Cyclical coordinate descent algorithm with two non-parametric error metrics for LASSO coefficient selection.
result State-of-the-art performance on simulated and real datasets.
OMP improves text classification accuracy with sparse models.
problem Overfitting in text classification due to high dimensionality.
method Greedy variable selection algorithm (OMP) and overlapping Group OMP.
result OMP and overlapping GOMP produce effective and very sparse models.
Sparse regularization reduces NLP model complexity without sacrificing accuracy.
problem Excessive parameter usage in neural models for NLP leads to high memory and runtime usage.
method Apply group lasso to rational RNNs to learn sparse, parameter-efficient models.
result Sparse rational RNNs can have significantly fewer parameters than non-sparse models without losing performance.
Multitask learning can be effective when features useful in one task are also useful for other tasks, and the group lasso is a standard method for selecting a common subset of features. In this paper, we are interested in a less restrictive form of multitask learning, wherein (1) the available features can be organized…
Bayesian method selects sparse models efficiently with less bias.
problem Sparse model selection and regularization in Gaussian graphical models.
method Continuous spike-and-slab framework with EM algorithm for fast explorations.
result Efficient selection of sparse models with less bias compared to other methods.
New method detects nonlinear causality in multivariate time series data.
problem Detecting nonlinear causal relationships in multidimensional time series.
method Sparse additive models (SpAMs) with B-spline bases and group-lasso optimization.
result The method can accurately estimate nonlinear causal relationships in β-mixing time series.
Proposes a new Lasso method with performance constraints.
problem No control over prediction accuracy for certain individuals.
method Adds quadratic performance constraints to Lasso-based objective functions.
result Defines a constrained sparse regression model through nonlinear optimization.
New method corrects selection bias in post-selective inference for Group LASSO.
problem Inference after Group LASSO selection is unreliable.
method Develops a consistent, post-selective Bayesian method to adjust for selection bias.
result Corrects bias in recovering effects of selected variables.
SNAM improves NAM's accuracy and feature selection via group sparsity.
problem Improving interpretability and accuracy in deep learning models.
method Employing group sparsity regularization in neural additive models (SNAM).
result SNAM provably converges to zero training loss and achieves exact support recovery.
New algorithm solves clustered lasso problem efficiently.
problem Learning group structure in regression parameters.
method Inexact semismooth Newton augmented Lagrangian algorithm with efficient Jacobian computation.
result The {\sc Ssnal} algorithm outperforms existing methods.
We consider a joint processing of n independent sparse regression problems. Each is based on a sample (yi1,xi1)...,(yim,xim) of m \iid observations from $y_{i1}=x_{i1}\tβ_i+\eps_{i1}$, yi1∈R, xi1∈Rp, i=1,...,n, and $\eps_{i1}\dist N(0,\sig^2)$, say. p is large enough so that the…