Bayesian model improves sparsity in regression coefficients.
problem Sparsity in regression coefficients.
method Bayesian generalized fused lasso modeling using NEG distribution.
result The proposed method produces more sparse solutions than the ordinary fused lasso.
The fused lasso is analyzed for high-dimensional piecewise-constant regression coefficients.
problem Estimation of high-dimensional piecewise-constant regression coefficients.
method Formulated a restricted isometry condition for the fused lasso estimator and derived estimation bounds.
result The estimation error can be dominated by either the lasso or the fused lasso rate, depending on the number of non-zero coefficients and piece-wise constant segments.
Develops a scalable framework for optimizing superposition-structured models.
problem Insufficient interpretability and generalization performance of simple structural models.
method Proximal Newton-type method with smoothed conic dual approach and LBFGS updating formula.
result Achieves super-linear convergence rate for optimizing superposition-structured models.
We study the problem of estimating high-dimensional regression models regularized by a structured sparsity-inducing penalty that encodes prior structural information on either the input or output variables. We consider two widely adopted types of penalties of this kind as motivating examples: (1) the general overlappin…
Bayesian method selects sparse models efficiently with less bias.
problem Sparse model selection and regularization in Gaussian graphical models.
method Continuous spike-and-slab framework with EM algorithm for fast explorations.
result Efficient selection of sparse models with less bias compared to other methods.
Method compares Gaussian Graphical Models to identify differences in brain connectivity.
problem Comparing functional connectivity between two populations.
method Debiased multi-task fused lasso to characterize parameter differences.
result Confidence intervals on edge differences in Gaussian Graphical Models.
We study the property of the Fused Lasso Signal Approximator (FLSA) for estimating a blocky signal sequence with additive noise. We transform the FLSA to an ordinary Lasso problem. By studying the property of the design matrix in the transformed Lasso problem, we find that the irrepresentable condition might not hold, …
This paper converts ADMM to proximal gradient for efficient sparse estimation.
problem Sparse estimation problems like fused lasso and convex clustering.
method General method converting ADMM to proximal gradient, assuming Lipschitz continuity of derivative.
result Significant improvement in efficiency for sparse estimation problems.
We consider efficient implementations of the generalized lasso dual path algorithm of Tibshirani and Taylor (2011). We first describe a generic approach that covers any penalty matrix D and any (full column rank) matrix X of predictor variables. We then describe fast implementations for the special cases of trend filte…
New proof shows weighted fused lasso has O(n^2) segments.
problem Proving the complexity of the weighted fused lasso.
method New proof showing the number of segments is O(n^2).
result The solution path of the weighted fused lasso has O(n^2) segments.
Develops a model for RNA-seq data clustering.
problem Challenges in clustering RNA-seq count data with variable selection.
method Sparse negative binomial mixture model with lasso or fused lasso regularization.
result Superior performance in clustering accuracy, feature selection, and biological interpretation.
Paper introduces MGLasso for multiscale graph inference in clustering and network analysis.
problem Graphical models in high-dimensional data analysis need to handle clustering and sparsity simultaneously.
method MGLasso combines clustering and graph inference through a convex relaxation of k-means and hierarchical clustering. It uses CONESTA for regularization.
result MGLasso improves network interpretability by estimating graphs at multiple scales.
SAIF efficiently selects features for sparse learning problems.
problem Efficiently selecting features in high-dimensional sparse learning problems.
method Incremental feature selection starting from a small set of features, with convergence guarantees.
result SAIF achieves superior or comparable efficiency and high scalability with a safe convergence guarantee.
DFR reduces the computational cost of sparse-group lasso and adaptive sparse-group lasso.
problem Sparse-group lasso's computational expense and need for tuning.
method Dual Feature Reduction (DFR) using strong screening rules and dual norms.
result DFR drastically reduces computational cost without affecting solution optimality.
ABM automates feature engineering and variable selection for loss-based models.
problem Improving model performance through better feature engineering and variable selection.
method ABM uses group and fused lasso regularization to automatically select cutting points and variables.
result ABM integrates feature engineering, variable selection, and model training.
A new algorithm speeds up and broadens the graph-fused lasso method.
problem Efficiently estimating parameters under graph structure assumptions.
method Decompose graph into trails, solve each trail using 1D fused lasso, alternate between primal and dual updates.
result Faster and more flexible GFL method compared to previous approaches.
We present the group fused Lasso for detection of multiple change-points shared by a set of co-occurring one-dimensional signals. Change-points are detected by approximating the original signals with a constraint on the multidimensional total variation, leading to piecewise-constant approximations. Fast algorithms are …
A wide class of regularization problems in machine learning and statistics employ a regularization term which is obtained by composing a simple convex function ωwith a linear transformation. This setting includes Group Lasso methods, the Fused Lasso and other total variation methods, multi-task learning methods and man…
Unified parallel ADMM for high-dimensional regression with combined regularizations.
problem Efficiently solving high-dimensional regression problems with combined regularization terms in parallel.
method Unified constrained optimization formulation based on consensus problem, parallel ADMM algorithms.
result Global convergence and linear convergence rate of the proposed algorithm.
The Lasso is a very well known penalized regression model, which adds an L1 penalty with parameter λ1 on the coefficients to the squared error loss function. The Fused Lasso extends this model by also putting an L1 penalty with parameter λ2 on the difference of neighboring coefficients, assuming the…
New screening rules speed up Sparse-Group Lasso solving.
problem Efficiently solving Sparse-Group Lasso in high dimensions.
method Adapted safe screening rules for Sparse-Group Lasso.
result Significant speed-ups in computing time for coordinate descent.
Sparse group Lasso optimizes sparse and grouped parameters in high-dimensional data.
problem Simultaneously sparse and grouped parameters in high-dimensional linear regression.
method Sparse group Lasso, debiased sparse group Lasso, statistical inference.
result Matching upper and lower bounds on sample complexity and estimation error.
Efficient ADMM algorithm detects structural breaks in multivariate time series.
problem Segmenting multivariate non-stationary time series with structural breaks.
method Alternating Direction Method of Multipliers (ADMM) with quadratic and group lasso penalties.
result Parallelizable global quadratic program and Kalman smoothing for efficient solution.
This paper develops a theory for group Lasso using a concept called strong group sparsity. Our result shows that group Lasso is superior to standard Lasso for strongly group-sparse signals. This provides a convincing theoretical justification for using group sparse regularization when the underlying group structure is …
The sparse group lasso optimization problem is solved using a coordinate gradient descent algorithm. The algorithm is applicable to a broad class of convex loss functions. Convergence of the algorithm is established, and the algorithm is used to investigate the performance of the multinomial sparse group lasso classifi…
New bounds for Lasso and Group Lasso in high dimensions derived.
problem Estimation error bounds for Lasso and Group Lasso in high-dimensional settings.
method Recent advances in high-dimensional statistics to derive new L2 estimation upper bounds.
result Bounds match optimal minimax rate for Lasso and improve over existing results for Group Lasso.
In this paper we analyze the asymptotic properties of l1 penalized maximum likelihood estimation of signals with piece-wise constant mean values and/or variances. The focus is on segmentation of a non-stationary time series with respect to changes in these model parameters. This change point detection and estimation pr…
Classification with a sparsity constraint on the solution plays a central role in many high dimensional machine learning applications. In some cases, the features can be grouped together so that entire subsets of features can be selected or not selected. In many applications, however, this can be too restrictive. In th…
Improves Group Lasso for categorical data by reducing dimensionality and selecting models.
problem Sparse modelling of categorical data is challenging, especially for high dimensions.
method Two-step procedure: first, reduce dimensionality using Group Lasso; second, select final model using an information criterion on clustered levels.
result The method produces a sparse solution and performs better than state-of-the-art algorithms in prediction accuracy and model dimension.
Estimates dynamic graph structure and changepoints in multivariate time series.
problem Estimating dynamic conditional dependency structure of multivariate time series.
method Group-fused graphical lasso for piecewise constant Gaussian graphical models.
result Efficient algorithm for estimating structure and changepoints.
The paper develops estimators for variance in graph structures using fused lasso.
problem Variance estimation in graph-structured problems.
method Developed linear time estimator for homoscedastic case and total variation regularization estimator for heteroscedastic case.
result Minimax rates and consistency for variance estimation in various graph structures.
New theorem for generalized group sparsity improves consistency and convergence rates.
problem Improving statistical inference in high-dimensional data with element-wise and group-wise sparsity.
method Developed a generalized version of Sparse-Group Lasso and proved a universal theorem for consistency and convergence rates.
result Obtained results on consistency and convergence rates for different forms of double sparsity regularization.
Unified approach for error bounds in group sparse compressed sensing.
problem Error bounds for group sparse vectors in compressed sensing.
method Unified approach using decomposable and γ-decomposable norms.
result Bounds for various group sparse norms derived.
New group-sparse SVD models improve biclustering of gene expression data.
problem Identifying block patterns with similar expressions in high-dimensional gene expression data.
method Proposed GL1-SVD, GL0-SVD, OGL1-SVD, and OGL0-SVD models with group Lasso and L0-norm penalties, using alternating iterative strategies and ADMM.
result Effective in identifying biologically interpretable gene modules with gene prior group knowledge.
New guarantees for Group LASSO in sparse convex optimization.
problem Sparse convex optimization with vector-valued features.
method Group LASSO regularization and analysis of gradient norms.
result Group LASSO selects the same features as Orthogonal Matching Pursuit.
A new model identifies genetic risk factors using gene-level priors.
problem Identifying genetic risk factors from nucleotide-level genetic variants.
method Sparse Group Lasso with Group-level Graph structure (SGLGG) model.
result SGLGG effectively identifies phenotype-associated risk SNPs.
The group lasso is a penalized regression method, used in regression problems where the covariates are partitioned into groups to promote sparsity at the group level. Existing methods for finding the group lasso estimator either use gradient projection methods to update the entire coefficient vector simultaneously at e…
Paper introduces machine learning for time series data, improving nowcasting accuracy.
problem Improving accuracy in nowcasting US GDP growth using machine learning.
method Sparse-group LASSO estimator for high-dimensional time series data, considering different sampling frequencies and financial/macroeconomic data tail properties.
result Sparse-group LASSO outperforms unstructured LASSO in nowcasting US GDP growth.
In this paper we introduce a new optimization formulation for sparse regression and compressed sensing, called CLOT (Combined L-One and Two), wherein the regularizer is a convex combination of the ℓ1- and ℓ2-norms. This formulation differs from the Elastic Net (EN) formulation, in which the regularizer is a…
We study regularized estimation in high-dimensional longitudinal classification problems, using the lasso and fused lasso regularizers. The constructed coefficient estimates are piecewise constant across the time dimension in the longitudinal problem, with adaptively selected change points (break points). We present an…
R package for multi-objective model selection in statistics.
problem Model selection challenges in statistics, especially for penalized models.
method Multi-objective optimization using Gaussian process-based optimization.
result Identification of hyperparameter values that represent desirable trade-offs.
Standardization boosts sparse group lasso performance for categorical variables.
problem Improving performance of sparse group lasso with categorical variables.
method Column-wise scaling and re-scaling of coefficients, without orthonormalization.
result Column-wise scaling and re-scaling of coefficients achieves the same effect as orthonormalization.
Substantial evidence indicates that major psychiatric disorders are associated with distributed neural dysconnectivity, leading to strong interest in using neuroimaging methods to accurately predict disorder status. In this work, we are specifically interested in a multivariate approach that uses features derived from …
Study high-dimensional Granger causality tests for VIX and financial news.
problem Testing Granger causality in high-dimensional time series data.
method Regularized regressions, sparse-group LASSO, HAC estimation of variance.
result Valid time series inference for Granger causality tests in high dimensions.
We present reconstruction algorithms for smooth signals with block sparsity from their compressed measurements. We tackle the issue of varying group size via group-sparse least absolute shrinkage selection operator (LASSO) as well as via latent group LASSO regularizations. We achieve smoothness in the signal via fusion…
A new method solves large-scale sparse group square-root Lasso problems efficiently.
problem Large-scale linearly constrained sparse group square-root Lasso problems.
method Dual semismooth Newton based augmented Lagrangian method (ALM).
result The proposed method efficiently solves the problem with numerical experiments demonstrating its effectiveness.
We introduce a recursive adaptive group lasso algorithm for real-time penalized least squares prediction that produces a time sequence of optimal sparse predictor coefficient vectors. At each time index the proposed algorithm computes an exact update of the optimal ℓ1,∞-penalized recursive least squares (R…
We consider the following signal recovery problem: given a measurement matrix Φ∈Rn×p and a noisy observation vector c∈Rn constructed from c=Φθ∗+ε where ε∈Rn is the noise vector whose entries follow i.i.d. centered sub-Gaussian distribution, how to recover …