Improved online penalty selection for time series models.
problem Efficiently selecting penalty parameters for lasso in time series models.
method Enhanced autoregressive model with online penalty selection.
result Significantly improved computational performance and forecast accuracy.
Optimal subset selection for hypothesis testing with penalties.
problem Optimal subset selection of information sources for hypothesis testing with misclassification penalties.
method Proposes a misclassification penalty framework and studies two variants of subset selection problems under centralized Bayesian learning.
result Proves the submodularity of the objective and constraints of the subset selection problems and establishes performance guarantees for greedy algorithms.
Paper designs a penalty for model order selection using information criteria.
problem Selecting the correct model order from a set of candidate models.
method Designs a penalty for the generalized information criterion (GIC) to minimize underestimation.
result Optimal penalty minimizes underestimation while keeping overestimation below a specified level.
A new framework selects information sources to test hypotheses robustly, even with misclassifications.
problem Robust hypothesis testing with misclassification penalties.
method Introduces a misclassification penalty framework and an efficient greedy algorithm.
result Proposes a submodular surrogate metric for better selection.
Ensemble validation shows selectivity penalties but variety benefits.
problem Selecting classifiers for ensemble models and their error bounds.
method Forming an ensemble from a set of hypothesis classifiers, selecting randomly, with an error bound formula.
result No penalty for using a richer hypothesis set if same fraction selected.
We consider supervised learning problems where the features are embedded in a graph, such as gene expressions in a gene network. In this context, it is of much interest to automatically select a subgraph with few connected components; by exploiting prior knowledge, one can indeed improve the prediction performance or o…
New nonconvex penalty smooths at origin for deep learning.
problem Improving variable selection and bias in high-dimensional statistical learning.
method Developed a new nonconvex penalty function smooth at origin.
result Asymptotic bias of new penalty function vanishes exponentially fast.
A new method selects penalties for high-dimensional models using the MDL principle.
problem Selecting optimal penalties for high-dimensional regularization models.
method MDL-RS method that minimizes a tight upper bound of LNML in high-dimensional spaces.
result Improves generalization performance of regularized estimates, especially with redundant parameters.
New method selects features more sparsely than lasso.
problem Feature selection in high-dimensional data.
method Horseshoe regularization penalty for non-convex optimization.
result Better statistical performance and faster computation.
In a Gaussian graphical model, the conditional independence between two variables are characterized by the corresponding zero entries in the inverse covariance matrix. Maximum likelihood method using the smoothly clipped absolute deviation (SCAD) penalty (Fan and Li, 2001) and the adaptive LASSO penalty (Zou, 2006) hav…
LLM-Lasso uses LLMs to improve feature selection in Lasso regression.
problem Improving feature selection in Lasso regression with domain-specific knowledge.
method Combines LLMs with Lasso regularization to generate feature weights.
result Outperforms standard Lasso and feature selection baselines in biomedical studies.
We describe a simple, efficient, permutation based procedure for selecting the penalty parameter in the LASSO. The procedure, which is intended for applications where variable selection is the primary focus, can be applied in a variety of structural settings, including generalized linear models. We briefly discuss conn…
Efficient ADMM algorithm solves nonconvex SVMs with various penalties.
problem Solving nonconvex penalized SVMs due to nondifferentiability, nonsmoothness, and nonconvexity.
method ADMM-based algorithm for a wide range of nonconvex penalties.
result The proposed algorithm outperforms other methods on benchmark datasets.
Safe screening improves generalized CGM's feature selection stability.
problem Improving feature selection stability in generalized CGM.
method Coupling safe screening with generalized CGM.
result Safe screening matches solution support at rate O(1/(tδ2)). Paper introduces Bayesian EEF for model order selection using exponentially embedded family.
problem Model order selection in Bayesian statistics.
method Bayesian EEF method using exponentially embedded family.
result Bayesian EEF can use vague priors and reveals EEF mechanism for model selection.
A fast method estimates group-adaptive elastic net penalties using co-data.
problem Computational inefficiency in estimating group-adaptive elastic net penalties.
method Derive low-dimensional representation of Taylor approximation for marginal likelihood and its derivative for group-adaptive ridge penalties; approximate elastic net marginal likelihood by ridge; transform ridge penalties to elastic net penalties.
result Significantly decreases computation time and outperforms other methods.
Study develops a method to select penalty parameters for sparse neural networks without cross-validation.
problem Selecting optimal penalty parameters for sparse neural networks without cross-validation.
method Established theoretical foundation to bound the infinite norm of the gradient of the loss function at zero.
result Proposed method effectively selects penalty parameters for sparse neural networks.
Model selection based on classical information criteria, such as BIC, is generally computationally demanding, but its properties are well studied. On the other hand, model selection based on parameter shrinkage by ℓ1-type penalties is computationally efficient. In this paper we make an attempt to combine their st…
This paper tackles the problem of selecting among several linear estimators in non-parametric regression; this includes model selection for linear regression, the choice of a regularization parameter in kernel ridge regression, spline smoothing or locally weighted regression, and the choice of a kernel in multiple kern…
Rejoinder on slope heuristics for model selection in regression.
problem Model selection in least-squares fixed-design regression with biased models and general noise.
method Proves the slope heuristics works even with significant bias and computes expectations for Gaussian noise.
result The slope heuristics is valid even when models are biased and noise has a general dependence structure.
Proposes a model selection procedure for high-dimensional binary classification using sparse logistic regression.
problem High-dimensional binary classification with sparse logistic regression.
method Penalized maximum likelihood with complexity penalty on model size, Slope estimator for logistic regression.
result Proposed complexity penalty is rate-optimal in the minimax sense.
The paper proposes a method to select tuning parameters for high-dimensional data analysis.
problem Selecting the tuning parameter in penalized likelihood methods for high-dimensional data.
method Optimizing the generalized information criterion (GIC) with an appropriate model complexity penalty.
result The proposed model complexity penalty should diverge at the rate of some power of log p.
Proposes spred for solving L1 penalty with SGD.
problem Solving L1 penalty in optimization problems. method Reparametrization and SGD approach.
result Proves spred as an exact differentiable solver of L1. Bayesian rLASSO improves model selection and prediction.
problem Improving model selection and prediction in linear regression.
method A Bayesian formulation of the rLASSO problem using inverse Laplace priors and scale mixture distributions.
result Bayesian rLASSO outperforms classical rLASSO in estimation, prediction, and variable selection.
The article studies a combined L1 and concave regularization method for high-dimensional models.
problem Tackles variable selection and prediction in high-dimensional settings.
method Uses combined L1 and concave penalties to optimize model sparsity and prediction risk. result Global optimum of the method achieves oracle prediction risk and false sign rate bounds.
Penalized regression is an attractive framework for variable selection problems. Often, variables possess a grouping structure, and the relevant selection problem is that of selecting groups, not individual variables. The group lasso has been proposed as a way of extending the ideas of the lasso to the problem of group…
High-dimensional data pose challenges in statistical learning and modeling. Sometimes the predictors can be naturally grouped where pursuing the between-group sparsity is desired. Collinearity may occur in real-world high-dimensional applications where the popular l1 technique suffers from both selection inconsisten…
Accelerated gradient method tackles nonconvex penalties in sparse learning.
problem Optimizing nonconvex penalties in sparse statistical learning.
method Generalized Nesterov's accelerated gradient method with hyperparameter optimization.
result Convergence can be made considerably faster with optimal hyperparameters.
Variable selection is a fundamental task in statistical data analysis. Sparsity-inducing regularization methods are a popular class of methods that simultaneously perform variable selection and model estimation. The central problem is a quadratic optimization problem with an l0-norm penalty. Exactly enforcing the l0-no…
Gradient descent can optimize penalty parameters for non-smooth penalties.
problem Optimizing penalty parameters for non-smooth penalties in regression problems.
method Modified gradient descent algorithm for non-smooth penalty functions.
result Decreased generalization error with tuned penalty parameters.
The paper explores MMPR to select diverse models for scientific insight.
problem Model selection often fails to bring multiple underlying patterns to light.
method Multi-model penalized regression (MMPR) to acknowledge model uncertainty.
result Different penalty settings can promote either shrinkage or sparsity of coefficients in separate models.
Proposes MinPEN framework for estimating relationships in multivariate models.
problem Estimating relationships between multivariate outcomes in statistical learning.
method MinPEN framework using minimum function penalty for non-convex optimization.
result Theoretical and practical validation of MinPEN framework for multivariate models.
FIRE extracts interpretable rules from tree ensembles.
problem Building sparse and interpretable rule sets from tree ensembles.
method Optimization-based framework with fusion regularization and sparsity-inducing penalties.
result FIRE outperforms state-of-the-art rule ensemble algorithms.
A new family of penalty functions, adaptive to likelihood, is introduced for model selection in general regression models. It arises naturally through assuming certain types of prior distribution on the regression parameters. To study stability properties of the penalized maximum likelihood estimator, two types of asym…
A new method reduces bias in adaptive Lasso estimates.
problem Bias in adaptive Lasso estimates.
method Proximal gradient approach to learn penalty coefficients as decision variables.
result Reduces bias in estimates and encourages arbitrary sparsity structure.
Identifying homogeneous subgroups of variables can be challenging in high dimensional data analysis with highly correlated predictors. We propose a new method called Hexagonal Operator for Regression with Shrinkage and Equality Selection, HORSES for short, that simultaneously selects positively correlated variables and…
ARGEN method improves variable selection and regularization in high-dimensional sparse models.
problem Constrained variable selection and regularization in high-dimensional sparse linear models.
method ARGEN penalty method, variable selection and regularization.
result ARGEN method has variable selection and estimation consistency under certain conditions.
SP-SPCA improves sparse PCA by adaptively adjusting variable penalties, enhancing interpretability and stability.
problem Poor interpretability and variable redundancy in PCA for high-dimensional data.
method Introduces a single equilibrium parameter to adaptively adjust variable penalties in the L2 regularization framework.
result Consistently outperforms standard sparse PCA methods in identifying sparse loading patterns and preserving cumulative variance.
Proposes a gradient-based variable selection method for binary classification in RKHS.
problem Variable selection in high-dimensional data analysis.
method Gradient-based representation of large-margin classifier with group-lasso penalty.
result Selection consistency and risk bound of the estimated classifier.
A new method speeds up overlapping group lasso computations.
problem Time-consuming optimization of overlapping group lasso on large-scale problems.
method Non-overlapping statistical approximation to overlapping group lasso.
result The proposed penalty is statistically equivalent to overlapping group lasso.
Survey on minimal penalty algorithms and slope heuristics.
problem Choosing optimal multiplicative constants from data.
method Minimal penalty and slope heuristics approach.
result Slope heuristics performs almost as well as residual-based estimators.
The paper develops a classification method using penalties on feature selection for high-dimensional data.
problem High-dimensional binary classification with many irrelevant features.
method Empirical risk minimization with l0-penalization for feature selection.
result The method achieves a sparse solution close to true sparsity with high probability and converges to low misclassification risk.
Due to advances in sensors, growing large and complex medical image data have the ability to visualize the pathological change in the cellular or even the molecular level or anatomical changes in tissues and organs. As a consequence, the medical images have the potential to enhance diagnosis of disease, prediction of c…
ecpc R-package improves high-dimensional prediction with co-data.
problem High-dimensional prediction with more variables than samples.
method Adaptive ridge penalised models with co-data, including continuous co-data.
result Improved variable selection and prediction performance.
New method improves IV estimation with many weak and invalid instruments.
problem Identification in linear IV models with unknown validity.
method Non-convex penalized approaches, surrogate sparsest penalty.
result Advantages over other IV estimators in selection consistency and weak IV strength conditions.
Variable selection for high-dimensional linear models has received a lot of attention lately, mostly in the context of l1-regularization. Part of the attraction is the variable selection effect: parsimonious models are obtained, which are very suitable for interpretation. In terms of predictive power, however, these re…
We propose a penalized likelihood method to jointly estimate multiple precision matrices for use in quadratic discriminant analysis and model based clustering. A ridge penalty and a ridge fusion penalty are used to introduce shrinkage and promote similarity between precision matrix estimates. Block-wise coordinate desc…
Regularization approach for arbitrage-free HJM-type model selection in financial markets.
problem Learning the closest arbitrage-free HJM-type model to a prespecified factor-model.
method Asymptotic solution through a 1-parameter family of optimizers, with penalties to detect local martingale measures.
result A deep-learning approach to arbitrage-free affine term-structure modeling is formulated.