Estimates VAR models with correlated data using structured norms.
problem Estimating VAR models with dependent data and structured norms.
method Structured VAR models with various norms, using Lasso-type techniques.
result Error bounds for structured VAR parameters are comparable to independent data settings.
Efficient implicit differentiation for Lasso hyperparameter optimization.
problem Difficult hyperparameter optimization for Lasso-type models.
method Implicit differentiation algorithm tailored for Lasso-type problems, avoiding matrix inversion and solving linear systems.
result Outperforms standard methods in optimizing error on held-out data or Stein Unbiased Risk Estimator (SURE).
New algorithm speeds up Lasso computation by proving faster convergence.
problem Lasso estimator's slow convergence rate due to ℓ1 penalty. method Homotopic approach using surrogate functions.
result Proves O([log(1/ε)]2) convergence rate for Lasso computation. We consider the problem of binary classification where one can, for a particular cost, choose not to classify an observation. We present a simple proof for the oracle inequality for the excess risk of structural risk minimizers using a lasso type penalty.
The vector autoregressive (VAR) model is a powerful tool in modeling complex time series and has been exploited in many fields. However, fitting high dimensional VAR model poses some unique challenges: On one hand, the dimensionality, caused by modeling a large number of time series and higher order autoregressive proc…
Proposes HVARX for interpretable large VARX models.
problem Challenges in estimating large VARX models with unmodeled exogenous variables.
method Lag-based hierarchically sparse estimator (HVARX).
result Improves forecast accuracy and interpretability compared to lasso-type approaches.
In this article, we analyze the SPICE method developed in [1], and establish its connections with other standard sparse estimation methods such as the Lasso and the LAD-Lasso. This result positions SPICE as a computationally efficient technique for the calculation of Lasso-type estimators. Conversely, this connection i…
Unified framework for shrinkage, thresholding, and regularization in normal mean estimation and linear regression.
problem Estimation of normal mean in multivariate settings with correlated observations.
method Approximate risk minimization over a functional class of shrinkage-thresholding rules.
result Unified estimator NOMAD for shrinkage, thresholding, and regularization.
Proximal algorithms work well for SQRT-Lasso despite its nonsmooth loss.
problem Tackles the optimization of SQRT-Lasso regression.
method Applies proximal algorithms without concern for nonsmooth loss.
result Proximal algorithms converge fast with high probability.
New method optimizes noise estimation alongside regression coefficients for multimodal neuroimaging data.
problem Heteroscedastic regression models with different noise levels across data sources.
method Generalized Concomitant Multi-Task Lasso for jointly estimating regression coefficients and noise covariance.
result Improved prediction and support identification with correct noise covariance estimation.
We address the issue of estimating the regression vector β in the generic s-sparse linear model y=Xβ+z, with β∈Rp, y∈Rn, $z\sim\mathcal N(0,\sg^2 I)$ and p>n when the variance $\sg^{2}$ is unknown. We study two LASSO-type methods that jointly estimate β and the variance. These estimators ar…
New solver speeds up Lasso-type problems by using screening rules and working sets.
problem Efficiently solving large-scale Lasso-type problems.
method Combining Gauss-Southwell rule with aggressive Gap Safe screening rules and working set strategy.
result Achieves state-of-the-art performance on sparse learning problems.
The study analyzes convergence rates for sparse pivotal estimators in high-dimensional regression.
problem Sparse pivotal estimation in high-dimensional regression problems.
method Theoretical analysis and comparison of non-smooth + non-smooth optimization problems, including smoothing techniques.
result Minimax sup-norm convergence rates for square-root Lasso-type estimators are derived.
Recent studies in the literature have paid much attention to the sparsity in linear classification tasks. One motivation of imposing sparsity assumption on the linear discriminant direction is to rule out the noninformative features, making hardly contribution to the classification problem. Most of those work were focu…
Shrinkage algorithms are of great importance in almost every area of statistics due to the increasing impact of big data. Especially time series analysis benefits from efficient and rapid estimation techniques such as the lasso. However, currently lasso type estimators for autoregressive time series models still focus …
New method detects nonlinear causality in multivariate time series data.
problem Detecting nonlinear causal relationships in multidimensional time series.
method Sparse additive models (SpAMs) with B-spline bases and group-lasso optimization.
result The method can accurately estimate nonlinear causal relationships in β-mixing time series.
New method handles correlated and repeated measurements using smoothed multivariate square-root Lasso.
problem Handling correlated and repeated measurements with complex noise structure.
method Proposes a concomitant estimator that uses non-averaged measurements and leverages smoothing theory for optimization.
result Demonstrates practical benefits on various datasets (toy, simulated, real neuroimaging).
The paper proposes a method to estimate complex models using machine learning.
problem Estimating the impact of welfare reform on women's welfare participation.
method Regularized orthogonal machine learning for non-linear semiparametric models.
result The proposed Lasso estimator converges at the oracle rate, preserving the single index property.
PS^2 selects assets then weights for high-dimensional investing.
problem High-dimensional mean--variance investing challenges.
method Two-step framework: Lasso screening followed by standard portfolio estimation.
result FPS^2 with defactored returns improves performance.
Paper introduces invariance-adapted latent space for contrastive learning.
problem Understanding the effectiveness of contrastive learning in data representation.
method Introduces invariance-adapted latent space and uses Lasso-type metric.
result Contrastive learning with Lasso-type metric can find an invariance-adapted latent space.
The paper introduces a new method to infer phylogenetic trees without bifurcations.
problem Inferring phylogenetic trees with zero-length branches and polytomies.
method Adaptive LASSO-type regularization estimators for phylogenetics.
result Regularization is a practical approach for phylogenetics, revealing zero-length branches.
In sparse regression modeling via regularization such as the lasso, it is important to select appropriate values of tuning parameters including regularization parameters. The choice of tuning parameters can be viewed as a model selection and evaluation problem. Mallows' Cp type criteria may be used as a tuning param…
One popular approach for nonstructural economic and financial forecasting is to include a large number of economic and financial variables, which has been shown to lead to significant improvements for forecasting, for example, by the dynamic factor models. A challenging issue is to determine which variables and (their)…
Proposes a new Lasso method for high missing rate data.
problem Handling high-dimensional data with many missing values.
method Integrates mean imputed covariance to overcome estimation bias.
result Effective even with high missing rates, improving upon CoCoLasso.
Components of biological systems interact with each other in order to carry out vital cell functions. Such information can be used to improve estimation and inference, and to obtain better insights into the underlying cellular mechanisms. Discovering regulatory interactions among genes is therefore an important problem…
The Mismatch Principle improves Lasso robustness to model uncertainties.
problem Estimation robustness under model misspecifications.
method Generalized Lasso with the Mismatch Principle.
result The Mismatch Principle provides robust error bounds for Lasso.
Regularized linear regression under the ℓ1 penalty, such as the Lasso, has been shown to be effective in variable selection and sparse modeling. The sampling distribution of an ℓ1-penalized estimator β^ is hard to determine as the estimator is defined by an optimization problem that in general can only…
We consider rules for discarding predictors in lasso regression and related problems, for computational efficiency. El Ghaoui et al (2010) propose "SAFE" rules that guarantee that a coefficient will be zero in the solution, based on the inner products of each predictor with the outcome. In this paper we propose strong …
New online method for multivariate probabilistic electricity price forecasting.
problem Multivariate probabilistic forecasting of electricity prices.
method Online multivariate distributional regression with LASSO regularization.
result Robust and interpretable joint prediction intervals for 24-hour electricity prices.
Due to advances in sensors, growing large and complex medical image data have the ability to visualize the pathological change in the cellular or even the molecular level or anatomical changes in tissues and organs. As a consequence, the medical images have the potential to enhance diagnosis of disease, prediction of c…
New model reveals evolving stock interactions over time.
problem Understanding time-varying relationships among stocks.
method LOcal Group Graphical Lasso Estimation (loggle) with ADMM algorithm.
result Loggle reveals evolving stock interactions over time.
Paper improves Lasso for S&P500 index tracking with post-selection inference.
problem Index tracking for S&P500 with many applications.
method Used Lasso for dimension reduction and post-selection inference.
result Lasso method for S&P500 index tracking shows high performance.
Suppose that we observe y∈Rf and X∈Rf×m in the following errors-in-variables model: \begin{eqnarray*} y & = & X_0 β^* + ε\\ X & = & X_0 + W \end{eqnarray*} where X0 is a f×m design matrix with independent subgaussian row vectors, ε∈Rf is a noise vector…
Unified framework for large-scale hypothesis testing with confounders.
problem Bias in large-scale hypothesis testing due to unmeasured confounders.
method Unified statistical estimation and inference framework that disentangles confounding effects and jointly estimates latent and primary effects.
result Effective Type-I error control and power in hypothesis testing.
It is well known that in a supervised classification setting when the number of features is smaller than the number of observations, Fisher's linear discriminant rule is asymptotically Bayes. However, there are numerous modern applications where classification is needed in the high-dimensional setting. Naive implementa…
New mathematical framework proves the effectiveness of reducing neural network sizes.
problem Selecting optimal neural network sizes to avoid overfitting.
method Adaptive group Lasso applied to one-hidden-layer feedforward networks.
result Adaptive group Lasso is consistent and can accurately reconstruct network sizes.
We propose the Bayesian bridge estimator for regularized regression and classification. Two key mixture representations for the Bayesian bridge model are developed: (1) a scale mixture of normals with respect to an alpha-stable random variable; and (2) a mixture of Bartlett--Fejer kernels (or triangle densities) with r…
The paper proposes a new model for predicting and analyzing economic variables.
problem Predicting and analyzing economic variables in developed regions.
method Time-varying parameter global vector autoregressive (TVP-GVAR) framework combined with machine learning models.
result The proposed model provides high precision out-of-sample predictions and novel insights into economic variable connectedness.
ET-Lasso improves Lasso feature selection in high-dimensional data.
problem Challenges in feature selection consistency of Lasso in high-dimensional data.
method ET-Lasso uses pseudo-features to separate active and inactive features by adding permuted features.
result ET-Lasso effectively selects active features under various scenarios.
Online graph learning from matrix-valued time series data.
problem Identifying dependency structure among sensors in a network.
method Extends VAR models to matrix-variate models, proposes online procedures for graph learning, and introduces Lasso-type approaches.
result Demonstrates effectiveness of online graph learning methods in both synthetic and real data.
Study compares univariate vs multivariate models for electricity price forecasting.
problem Optimal model structure for short-term electricity price forecasting.
method Comprehensive empirical study comparing univariate and multivariate modeling frameworks.
result Multivariate models do not uniformly outperform univariate models across all datasets, seasons, or hours.
Two new regularization methods improve neural network performance and complexity control.
problem Improving neural network performance and complexity control with correlated or high-dimensional features.
method Two regularization strategies: covariance-aware ridge and covariance-aware lasso.
result Improves predictive performance and complexity control over standard penalties.
Piecewise-linear regression trees improve tree-based regression with theoretical and practical benefits.
problem Improving tree-based regression models with theoretical guarantees and practical tractability.
method Regularized piecewise-linear node-splitting criterion, LASSO-type and ℓ2 regularization, variable selection procedure. result New high-probability generalization error bounds for piecewise-linear regression trees.
Unified framework infers time-varying graphs from incomplete signals.
problem Jointly inferring time-varying network topologies and imputing missing data from partial observations.
method Unified non-convex optimization framework with Proximal Alternating Direction Method of Multipliers (PADMM) algorithm.
result Superior robustness in high missing-data regimes, demonstrated through extensive numerical experiments.
The lasso and related sparsity inducing algorithms have been the target of substantial theoretical and applied research. Correspondingly, many results are known about their behavior for a fixed or optimally chosen tuning parameter specified up to unknown constants. In practice, however, this oracle tuning parameter is …
Proposes a new clustering algorithm for high-dimensional data.
problem Challenges of feature selection in high-dimensional clustering.
method An EM algorithm with lasso-type constraints on cluster pairs.
result Identifies informative features and cluster separability.
New hybrid rules improve lasso optimization efficiency.
problem Efficiently solving lasso problems with ultrahigh-dimensional data.
method Hybrid safe-strong rules (HSSR) incorporating safe screening into sequential strong rules.
result HSSR outperforms existing rules in synthetic and real data sets.
When applying the support vector machine (SVM) to high-dimensional classification problems, we often impose a sparse structure in the SVM to eliminate the influences of the irrelevant predictors. The lasso and other variable selection techniques have been successfully used in the SVM to perform automatic variable selec…