In this paper, we investigate a multivariate multi-response (MVMR) linear regression problem, which contains multiple linear regression models with differently distributed design matrices, and different regression and output vectors. The goal is to recover the support union of all regression vectors using -reg…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,694 papers · 148 categories
Trend · papers per month
48 results for “thresholded Lasso”
This paper considers the problem of estimating multiple related Gaussian graphical models from a -dimensional dataset consisting of different classes. Our work is based upon the formulation of this problem as group graphical lasso. This paper proposes a novel hybrid covariance thresholding algorithm that can effecti…
Graphical lasso may fail to fit models when data points are insufficient.
problem When does graphical lasso fail to select and fit a graphical model?
method Computational experiments with graphical lasso.
result Graphical lasso may fail when the number of data points is less than the maximum likelihood threshold.
Ridge regression is revisited with debiasing and thresholding, offering advantages over Lasso.
problem High-dimensional data challenges classical ridge regression's sparsity detection and bias issues.
method Debiasing and thresholding ridge regression, introducing a wild bootstrap for confidence regions and hypothesis testing, and a hybrid bootstrap for prediction intervals.
result Debiased and thresholded ridge regression can offer similar performance to thresholded Lasso and may be preferable in some settings.
We consider the sparse inverse covariance regularization problem or graphical lasso with regularization parameter . Suppose the co- variance graph formed by thresholding the entries of the sample covariance matrix at is decomposed into connected components. We show that the vertex-partition induced by the thresh…
Thresholded Lasso Banditstat.ML
Thresholded Lasso bandit minimizes regret in sparse linear bandits.
problem Sparse stochastic contextual linear bandits with large feature vectors.
method Uses Lasso framework with thresholding to estimate reward function and its sparse support.
result Non-asymptotic regret upper bounds scaling as .
This review summarizes five Lasso optimization algorithms.
problem Optimizing the Lasso objective function.
method Five representative algorithms: ISTA, FISTA, CGDA, SLA, PFA.
result Comparison of convergence rates and strengths/weaknesses.
Heavy Lasso improves robustness in high-dimensional linear regression with heavy-tailed errors.
problem Challenges of classical Lasso in handling heavy-tailed noise and outliers.
method Data-augmented soft-thresholding with Student's t-distribution loss.
result Heavy Lasso achieves comparable rates to Huber loss under theoretical bounds.
Paper analyzes adaptive ISTA with MAD for LASSO problem.
problem Finding solutions to LASSO problems without tuning .
method Adaptive ISTA with median absolute deviation (MAD) for estimating noise level.
result Local linear convergence and global convergence of the algorithm.
Optimal algorithm for high-dimensional stochastic linear bandits with sparse parameters.
problem High-dimensional stochastic linear bandits with sparse parameters.
method Three-stage arm selection algorithm using thresholded Lasso for estimation.
result Achieves exact minimax optimality in cumulative regret.
The high-dimensional linear model is considered and the focus is put on the problem of recovering the support of the sparse vector We introduce Lasso-Zero, a new -based estimator whose novelty resides in an "overfit, then threshold" paradigm and the use of noise dictionaries concate…
Lasso proves consistent model selection for high-dimensional Ising models.
problem Model selection consistency of Lasso for high-dimensional Ising models.
method Theoretical analysis of Lasso with and without post-thresholding for Ising models.
result Lasso without post-thresholding is model selection consistent in the whole paramagnetic phase with .
To estimate a sparse linear model from data with Gaussian noise, consilience from lasso and compressed sensing literatures is that thresholding estimators like lasso and the Dantzig selector have the ability in some situations to identify with high probability part of the significant covariates asymptotically, and are …
A simple thresholding technique improves graph selection in neural connectivity studies.
problem Graphical model selection for functional neural connectivity in the presence of latent variables.
method Apply a hard thresholding operator to graphical Lasso, neighborhood selection, or CLIME estimators.
result Thresholded estimators outperform existing methods in graph selection consistency and empirical results.
Improves graph recovery in Gaussian graphical modeling.
problem Calibrating regularization parameters for graph recovery.
method Thresholded adaptive validation applied to graphical lasso.
result Thresholding pipeline improves graph recovery.
A Unified Framework for Pattern Recovery in Penalized and Thresholded Estimation and its Geometrymath.ST
Unified framework for pattern recovery in penalized and thresholded estimation.
problem Pattern recovery in penalized and thresholded estimation methods.
method Defining a novel pattern notion based on subdifferentials, introducing accessibility and noiseless recovery conditions.
result Unified and extended conditions for pattern recovery in a broad class of penalized estimators.
Confidence intervals based on penalized maximum likelihood estimators such as the LASSO, adaptive LASSO, and hard-thresholding are analyzed. In the known-variance case, the finite-sample coverage properties of such intervals are determined and it is shown that symmetric intervals are the shortest. The length of the sho…
A privacy-preserving algorithm for high-dimensional bandits.
problem High-dimensional stochastic contextual linear bandits with sparse parameters under privacy constraints.
method PrivateLASSO algorithm based on sparse hard-thresholding and episodic thresholding.
result Minimax private lower bounds and utility guarantees for PrivateLASSO.
Iterative thresholding algorithms are well-suited for high-dimensional problems in sparse recovery and compressive sensing. The performance of this class of algorithms depends heavily on the tuning of certain threshold parameters. In particular, both the final reconstruction error and the convergence rate of the algori…
CV inference can be invalid for relatively unstable model comparisons.
problem The validity of cross-validation for model comparison is questioned when models are relatively unstable.
method The study proves that simple, individually stable models can generate relatively unstable comparisons, invalidating CV inference.
result The Lasso and soft-thresholding generate relatively unstable comparisons, invalidating CV inferences.
Robust Lasso-Zero handles missing covariates and sparse corruptions.
problem Sparse corruptions and missing covariates in sparse linear models.
method Extension of Lasso-Zero to handle sparse corruptions, with theoretical guarantees on sign recovery.
result Robust Lasso-Zero can handle missing values without specifying a parametric model.
Proposes a new Lasso method with performance constraints.
problem No control over prediction accuracy for certain individuals.
method Adds quadratic performance constraints to Lasso-based objective functions.
result Defines a constrained sparse regression model through nonlinear optimization.
Unified framework for shrinkage, thresholding, and regularization in normal mean estimation and linear regression.
problem Estimation of normal mean in multivariate settings with correlated observations.
method Approximate risk minimization over a functional class of shrinkage-thresholding rules.
result Unified estimator NOMAD for shrinkage, thresholding, and regularization.
Large-Scale Sparse Inverse Covariance Estimation via Thresholding and Max-Det Matrix Completionstat.ML
The sparse inverse covariance estimation problem is commonly solved using an -regularized Gaussian maximum likelihood estimator known as "graphical lasso", but its computational cost becomes prohibitive for large data sets. A recent line of results showed--under mild assumptions--that the graphical lasso esti…
A note relating ridge regression and OLS p-values to preconditioned sparse penalized regressionstat.ML
When the design matrix has orthonormal columns, "soft thresholding" the ordinary least squares (OLS) solution produces the Lasso solution [Tibshirani, 1996]. If one uses the Puffer preconditioned Lasso [Jia and Rohe, 2012], then this result generalizes from orthonormal designs to full rank designs (Theorem 1). Theorem …
Iterative thresholding algorithms seek to optimize a differentiable objective function over a sparsity or rank constraint by alternating between gradient steps that reduce the objective, and thresholding steps that enforce the constraint. This work examines the choice of the thresholding operator, and asks whether it i…
On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO, SCAD, and Thresholdingmath.ST
We study the distributions of the LASSO, SCAD, and thresholding estimators, in finite samples and in the large-sample limit. The asymptotic distributions are derived for both the case where the estimators are tuned to perform consistent model selection and for the case where the estimators are tuned to perform conserva…
We propose a new method of learning a sparse nonnegative-definite target matrix. Our primary example of the target matrix is the inverse of a population covariance or correlation matrix. The algorithm first estimates each column of the target matrix by the scaled Lasso and then adjusts the matrix estimator to be symmet…
Graphical Lasso (GL) is a popular method for learning the structure of an undirected graphical model, which is based on an regularization technique. The objective of this paper is to compare the computationally-heavy GL technique with a numerically-cheap heuristic method that is based on simply thresholding the s…
New method trains neural networks with threshold activation functions efficiently.
problem Training neural networks with threshold activation functions is challenging due to zero gradients.
method We study weight decay regularized training problems of deep neural networks with threshold activations, showing they can be formulated as convex optimization problems.
result Regularized deep threshold network training problems can be formulated as standard convex optimization problems, paralleling the LASSO method.
In this paper, we introduce Adaptive Cluster Lasso(ACL) method for variable selection in high dimensional sparse regression models with strongly correlated variables. To handle correlated variables, the concept of clustering or grouping variables and then pursuing model fitting is widely accepted. When the dimension is…
Shrinkage algorithms are of great importance in almost every area of statistics due to the increasing impact of big data. Especially time series analysis benefits from efficient and rapid estimation techniques such as the lasso. However, currently lasso type estimators for autoregressive time series models still focus …
A two-phase algorithm identifies the best arm in sparse linear bandits with fixed budget.
problem Best arm identification in sparse linear bandits with limited budget.
method Lasso and Optimal-Design (Lasso-OD) based linear best-arm identification.
result Lasso-OD achieves significant performance improvement for sparse and high-dimensional linear bandits.
The Lasso is suboptimal in sparse linear regression due to design matrix constraints.
problem The suboptimality of the Lasso estimator in sparse linear regression.
method Characterization of optimal estimation rates and analysis of the Lasso estimator.
result The Lasso is provably minimax rate-suboptimal when the minimum singular value is small.
In regression settings where explanatory variables have very low correlations and there are relatively few effects, each of large magnitude, we expect the Lasso to find the important variables with few errors, if any. This paper shows that in a regime of linear sparsity---meaning that the fraction of variables with a n…
The paper tackles reward-relevance in offline RL with sparse decision dynamics.
problem Offline reinforcement learning with sparse decision dynamics and estimation sparsity.
method Reward-filtered least-squares policy evaluation using thresholded lasso.
result The method provides theoretical guarantees with sample complexity dependent on sparse component size.
FILTER model uses fusion penalized logistic threshold regression for high-dimensional data with unknown cut points.
problem Modeling high-dimensional data with unknown cut points and binary responses.
method Fusion penalized logistic threshold regression (FILTER) model with fused lasso penalty for variable selection.
result Established non-asymptotic error bounds for coefficient estimation and model selection consistency.
ARHT algorithm improves sparsity guarantees in convex optimization.
problem Optimizing convex functions with sparsity constraints.
method Adaptively Regularized Hard Thresholding (ARHT) algorithm.
result ARHT achieves sparsity bound of γ=O(κ), matching theoretical limits.
Paper introduces MGLasso for multiscale graph inference in clustering and network analysis.
problem Graphical models in high-dimensional data analysis need to handle clustering and sparsity simultaneously.
method MGLasso combines clustering and graph inference through a convex relaxation of k-means and hierarchical clustering. It uses CONESTA for regularization.
result MGLasso improves network interpretability by estimating graphs at multiple scales.
Developed a new thresholding method that connects soft and hard thresholding.
problem Connecting soft and hard thresholding methods in data analysis.
method Scaled soft thresholding method with empirical scaling values.
result Found two sources of over-fitting in the scaled soft thresholding method.
The L1 regularization (Lasso) has proven to be a versatile tool to select relevant features and estimate the model coefficients simultaneously and has been widely used in many research areas such as genomes studies, finance, and biomedical imaging. Despite its popularity, it is very challenging to guarantee the feature…
Bayesian method discovers PDEs with variable coefficients robustly.
problem Discovering PDEs from noisy data is challenging.
method Bayesian sparse learning with tBGL-SS and Gibbs sampler.
result Method enhances robustness and model selection criteria.
Study on sparse recovery with mixed-quality data, establishing sample-size conditions.
problem Sparse recovery with heterogeneous noise from high- and low-quality sources.
method Establishes linear trade-off for sufficient conditions, analyzes LASSO algorithm.
result Linear trade-off for sufficient conditions, robustness of LASSO to data heterogeneity.
We present a methodology for probabilistic load forecasting that is based on lasso (least absolute shrinkage and selection operator) estimation. The model considered can be regarded as a bivariate time-varying threshold autoregressive(AR) process for the hourly electric load and temperature. The joint modeling approach…
We consider the problem of learning a high-dimensional multi-task regression model, under sparsity constraints induced by presence of grouping structures on the input covariates and on the output predictors. This problem is primarily motivated by expression quantitative trait locus (eQTL) mapping, of which the goal is …
In this paper, we consider the Graphical Lasso (GL), a popular optimization problem for learning the sparse representations of high-dimensional datasets, which is well-known to be computationally expensive for large-scale problems. Recently, we have shown that the sparsity pattern of the optimal solution of GL is equiv…
High-dimensional data analysis has motivated a spectrum of regularization methods for variable selection and sparse modeling, with two popular classes of convex ones and concave ones. A long debate has been on whether one class dominates the other, an important question both in theory and to practitioners. In this pape…
We study the performance of sparse regression methods and propose new techniques to distill the governing equations of dynamical systems from data. We first look at the generic methodology of learning interpretable equation forms from data, proposed by Brunton et al., followed by performance of LASSO for this purpose. …