S-VNNs improve VNNs by sparsifying covariance matrices.
problem Spurious correlations in covariance matrices degrade VNNs' performance and efficiency.
method Apply sparsification techniques on sample covariance matrix and integrate into VNN architecture.
result S-VNNs achieve improved performance, stability, and reduced computational time.
New method estimates sparse covariance matrices in logit mixtures.
problem Estimating correlations among random coefficients in logit models.
method Mixed-integer optimization (MIO) with Markov Chain Monte Carlo (MCMC) for posterior draws.
result Correctly recovers true covariance structure from synthetic data.
Proposes a method to recover sparse tensors with covariate info.
problem Sparse tensor with high missing entries and many zeros.
method Covariate-assisted Sparse Tensor Completion (COSTCO) using latent components.
result 23% accuracy improvement over baseline in advertisement dataset.
The paper improves ranking by integrating covariates and sparse intrinsic scores.
problem Ranking items with incomplete preference scores explained by covariates.
method Extends BTL model with covariate information and sparse intrinsic scores, using penalized MLE.
result Developed debiased estimator for penalized MLE with distributional properties.
A new method for estimating sparse inverse covariance matrices.
problem Recovering the connectivity and non-connectivity graph of covariates.
method Adaptive thresholding in a transformed domain of the inverse covariance matrix.
result The proposed method outperforms state-of-the-art methods in accuracy.
Efficiently solves large-scale sparse covariance estimation problems.
problem Sparse inverse covariance estimation for large datasets.
method Thresholding the sample covariance matrix and solving a maximum determinant matrix completion problem using a Newton-CG algorithm.
result The algorithm converges to an ε-accurate solution in O(nlog(1/ε)) time and O(n) memory.
Efficiently estimates sparse linear regression with heavy-tailed data and outliers.
problem Sparse estimation of linear regression coefficients with heavy-tailed covariates and noises, including outliers.
method Efficient computation of robust estimator with nearly optimal error bound.
result Nearly optimal error bound for robust sparse estimation.
New method optimizes sparse inverse covariance estimation with guaranteed optimality.
problem Sparse inverse covariance estimation with robustness over sparsity.
method Cardinality constrained likelihood problem solved using mixed-integer and convex optimization.
result Certifiably optimal solutions with high quality and sparsity guarantees.
In this paper we formally analyse the use of sparse filtering algorithms to perform covariate shift adaptation. We provide a theoretical analysis of sparse filtering by evaluating the conditions required to perform covariate shift adaptation. We prove that sparse filtering can perform adaptation only if the conditional…
Paper proposes DP-Thresholding for estimating sparse high-dimensional covariance matrices with differential privacy.
problem Estimating sparse high-dimensional covariance matrices under differential privacy constraints.
method DP-Thresholding method for achieving non-trivial error bounds.
result DP-Thresholding achieves significant error bounds compared to existing methods.
Method estimates sparse inverse covariance and partial correlation matrices efficiently.
problem Sparse high-dimensional inverse covariance and partial correlation matrix estimation.
method Two-stage estimation method using partial regression with positive semi-definiteness.
result Efficient estimation of inverse covariance and partial correlation matrices with derived non-asymptotic rates.
New method speeds up sparse Bayesian learning without covariance matrix.
problem Sparse coding problem with uncertainty quantification.
method Covariance-free expectation maximization (CoFEM) that avoids explicit covariance matrix computation.
result Up to thousands of times faster than existing methods without sacrificing accuracy.
New method tackles high-dimensional SBL without covariance matrices.
problem Sparse coding problem in high-dimensional settings.
method Parallel solution of multiple linear systems using conjugate gradient algorithm.
result Our method scales better in computation time and memory.
Robust Lasso-Zero handles missing covariates and sparse corruptions.
problem Sparse corruptions and missing covariates in sparse linear models.
method Extension of Lasso-Zero to handle sparse corruptions, with theoretical guarantees on sign recovery.
result Robust Lasso-Zero can handle missing values without specifying a parametric model.
EiGLasso speeds up sparse Kronecker-sum covariance estimation.
problem Sparse Kronecker-sum inverse covariance estimation challenges in scalability and parameter identification.
method Newton's method combined with eigendecomposition of sample and feature graphs, approximating Hessian for speed.
result Two to three orders-of-magnitude speed-up on simulated and real-world data.
New algorithm reduces sample complexity for sparse linear regression.
problem Sparse linear regression with correlated covariates and approximate dependencies.
method Polynomial-time algorithm that adapts the Lasso to tolerate approximate dependencies.
result Achieves near-optimal sample complexity for constant sparsity and ill-conditioned covariates.
Much recent work has concerned sparse approximations to speed up the Gaussian process regression from the unfavorable O(n3) scaling in computational time to O(nm2). Thus far, work has concentrated on models with one covariance function. However, in many practical situations additive models with multiple covariance func…
Paper proposes a new method for sparse covariance Cholesky factor estimation.
problem Estimating sparse covariance matrices for ordered data.
method Matrix loss penalization approach for sparse Cholesky factor estimation.
result The proposed method outperforms existing regression-based approaches in simulations and real data.
Proposes a convex method for high-dimensional sparse sliced inverse regression.
problem Difficulty in interpreting results and variability in high-dimensional settings.
method Convex formulation and linearized alternating direction methods of multiplier algorithm.
result Upper bound on the subspace distance between estimated and true subspaces.
Efficiently estimates sparse linear regression with heavy-tailed and outlier-contaminated data.
problem Estimating sparse linear regression coefficients with heavy-tailed and outlier-contaminated data.
method Efficient computation of estimators with sharp error bounds.
result Sharp error bounds for efficient estimators.
Nash integrates covariate-specific side info into sparse regression via neural networks.
problem Sparse linear regression struggles with covariates exhibiting structure or coming from heterogeneous sources.
method Neural Adaptive Shrinkage (Nash) framework that integrates side information into sparse regression via neural networks. Uses split variational empirical Bayes algorithm.
result Nash improves accuracy and adaptability over existing methods in real data experiments.
New combinatorial method for sparse PCA works beyond spiked identity model.
problem Sparse PCA under general covariance matrices.
method Combinatorial truncated power method with global convergence guarantee.
result First combinatorial sparse PCA method provably successful for general covariance matrices.
We present a robust alternative to principal component analysis (PCA) --- called elliptical component analysis (ECA) --- for analyzing high dimensional, elliptically distributed data. ECA estimates the eigenspace of the covariance matrix of the elliptical data. To cope with heavy-tailed elliptical distributions, a mult…
Fitting high-dimensional data involves a delicate tradeoff between faithful representation and the use of sparse models. Too often, sparsity assumptions on the fitted model are too restrictive to provide a faithful representation of the observed data. In this paper, we present a novel framework incorporating sparsity i…
Study improves error bounds for sparse regression with heavy-tailed covariates.
problem Estimating sparse coefficients in linear regression with heavy-tailed covariates.
method Employed an ℓ1-penalized Huber regression method. result Error bound identical to Gaussian case for L-subexponential covariates. Sparse covariance estimation in the vertical-split model achieves exponential improvement over dense estimates.
problem Minimax estimation error for distributed covariance matrix estimation in the vertical-split setting.
method Elementwise s-sparsity is shown to reduce communication and sample complexity. result Minimax lower bounds for 1-sparse cross-covariance estimation are established. HP-CONCORD optimizes sparse covariance estimation for large datasets.
problem Scalability and Gaussian assumption limitations in sparse inverse covariance estimation.
method Communication-avoiding proximal gradient method on a multi-node cluster.
result HP-CONCORD outperforms state-of-the-art methods on large-scale problems.
Lasso performs poorly with correlated covariates, but a rescaled approach fixes this.
problem Lasso's performance degrades with correlated covariates, leading to inefficiency.
method Proposes a rescaling method for Lasso to handle correlated covariates effectively.
result Rescaled Lasso provides strong provable guarantees for estimation with quadratic sample complexity.
Proposes SVI for covariate-shift generalization with sparse variable independence.
problem Covariate-shift generalization with limited data and unstable variables.
method Introduces sparsity constraint and combines reweighting and selection in an iterative way.
result Improves covariate-shift generalization performance on synthetic and real-world datasets.
We find a closed-form determinant for a specific sparse covariance matrix model.
problem Finding the determinant of a specific class of sparse positive definite matrices.
method Using Fourier transform of local factors, Normal Factor Graph Duality Theorem, and Matrix Determinant Lemma.
result We derive a closed-form expression for the determinant.
Solution to sparse PCA tuning problem using Empirical Bayes.
problem Sparse PCA multiple tuning problem (MTP).
method Empirical Bayes covariance decomposition for penalized PCA.
result Empirical Bayes approach efficiently solves MTP in sparse PCA.
Novel algorithm recovers sparse parameters in high-dimensional data with constant corruption.
problem Sparse regression with high dimensionality and constant fraction of corruptions.
method Robust Iterative Hard Thresholding, filtering algorithm for outlier removal.
result Near information-theoretically optimal error guarantee with sub-linear sample complexity.
Standard sparse pseudo-input approximations to the Gaussian process (GP) cannot handle complex functions well. Sparse spectrum alternatives attempt to answer this but are known to over-fit. We suggest the use of variational inference for the sparse spectrum approximation to avoid both issues. We model the covariance fu…
The graphical lasso (glasso) is a widely-used fast algorithm for estimating sparse inverse covariance matrices. The glasso solves an L1 penalized maximum likelihood problem and is available as an R library on CRAN. The output from the glasso, a regularized covariance matrix estimate a sparse inverse covariance matrix e…
The problem of estimating sparse eigenvectors of a symmetric matrix attracts a lot of attention in many applications, especially those with high dimensional data set. While classical eigenvectors can be obtained as the solution of a maximization problem, existing approaches formulate this problem by adding a penalty te…
Recently, there has been focus on penalized log-likelihood covariance estimation for sparse inverse covariance (precision) matrices. The penalty is responsible for inducing sparsity, and a very common choice is the convex l1 norm. However, the best estimator performance is not always achieved with this penalty. The …
New algorithm recovers sparse, generic mixing matrix from covariance of mixed data.
problem Recovering independent components from mixed Gaussian data.
method Sparse and generic mixing matrix, efficient algorithm.
result Efficiently recovers mixing matrix from covariance of mixed data.
Inference and Estimation in Missing Information (MI) scenarios are important topics in Statistical Learning Theory and Machine Learning (ML). In ML literature, attempts have been made to enhance prediction through precise feature selection methods. In sparse linear models, LASSO is well-known in extracting the desired …
A new covariance estimator reduces dimensionality and improves portfolio forecasting.
problem Estimating high-dimensional covariance matrices with weak factors.
method Sparse Approximate Factor (SAF) model with l1-regularization. result SAF estimator outperforms other methods in portfolio forecasting.
We study sparse principal components analysis in high dimensions, where p (the number of variables) can be much larger than n (the number of observations), and analyze the problem of estimating the subspace spanned by the principal eigenvectors of the population covariance matrix. We introduce two complementary not…
New method for factor analysis using nuclear and ℓ0 norms.
problem Finding a low-rank plus sparse decomposition from noisy covariance matrix.
method Formulated an optimization problem with nuclear norm, ℓ0 norm, and KL divergence. Used alternating minimization algorithm. result Algorithm effectively decomposes covariance matrices in synthetic and real datasets.
Multivariate regression model is a natural generalization of the classical univari- ate regression model for fitting multiple responses. In this paper, we propose a high- dimensional multivariate conditional regression model for constructing sparse estimates of the multivariate regression coefficient matrix that accoun…
New algorithm reduces semi-bandit regret using covariance estimates.
problem Complexity of semi-bandits due to joint distribution of outcomes.
method Develops a new sub-exponential distribution family and an algorithm using covariance estimates.
result Proves a new lower bound on expected regret and constructs an algorithm with asymptotic analysis.
Paper estimates GMMs with unknown covariances using sparse regularization.
problem Estimating GMMs with unknown diagonal covariances from samples.
method Employed Beurling-LASSO (BLASSO) for sparse estimation of component means, covariances, and weights.
result Established non-asymptotic recovery guarantees with nearly parametric convergence rates.
The covariance matrix of a p-dimensional random variable is a fundamental quantity in data analysis. Given n i.i.d. observations, it is typically estimated by the sample covariance matrix, at a computational cost of O(np2) operations. When n,p are large, this computation may be prohibitively slow. Moreover, …
New method simplifies Graphical Lasso for chordal graphs.
problem Sparse covariance estimation for large graphs.
method Closed-form solution for chordal graphs, reducing GL to matrix completion.
result Graphical Lasso and thresholding equivalence holds for chordal structures.
SPICE estimates sparse linear dynamic networks without hyperparameters.
problem Estimating topology and dynamics of sparse linear dynamic networks.
method SPICE (Sparse Iterative Covariance Estimation) method in an iterative framework.
result Directly reveals the underlying topology of the network.
This paper aims at achieving a simultaneously sparse and low-rank estimator from the semidefinite population covariance matrices. We first benefit from a convex optimization which develops l1-norm penalty to encourage the sparsity and nuclear norm to favor the low-rank property. For the proposed estimator, we then p…