Paper addresses robust sparse vector mean estimation under local differential privacy.
problem Challenges in defending poisoning attacks on multi-item users in LDP protocols.
method Randomized Projection with Clipping (RPC) to handle clipping bias and enhance robustness.
result Proposes a method that achieves comparable or better performance than existing methods under trusted environments and significantly enhances robustness under untrusted environments.
New algorithms reduce communication for sparse mean estimation in noisy distributed systems.
problem Sparse normal means estimation with limited communication in a distributed setting.
method Two distributed algorithms for estimating a sparse mean vector with sublinear communication.
result Correct support of the sparse mean can be recovered with significantly less communication than previously required.
New method for estimating sparse means in noisy data.
problem Estimating the mean of a sparse distribution in the presence of outliers.
method Difference-of-Pairs Filtering technique for list-decodable sparse mean estimation.
result First sample and computationally efficient algorithm for list-decodable sparse mean estimation.
Proposes EM for sparse horseshoe estimation.
problem Sparse estimation of sparse parameter vectors using the horseshoe prior.
method Expectation-Maximisation (EM) procedure for MAP estimates.
result Our approach performs comparable or superior to state-of-the-art methods.
We create interpretable word embeddings through sparse coding.
problem Difficult to interpret word embeddings in natural language processing.
method Transform pretrained dense word embeddings into sparse embeddings through sparse coding.
result Sparse embeddings are more interpretable and achieve good performance.
In this paper, we derive Hybrid, Bayesian and Marginalized Cramér-Rao lower bounds (HCRB, BCRB and MCRB) for the single and multiple measurement vector Sparse Bayesian Learning (SBL) problem of estimating compressible vectors and their prior distribution parameters. We assume the unknown vector to be drawn from a compr…
New method improves robust sparse association estimation.
problem Outliers in high-dimensional data.
method Splitting robust estimation into optimization phases, using augmented Lagrangian and adaptive gradient descent.
result Improved precision over existing methods.
New algorithm reduces runtime for robust sparse mean estimation.
problem Efficiently estimating mean from corrupted data with sparse constraints.
method Subquadratic time algorithm using poly(k, log d, 1/ε) samples.
result First subquadratic time algorithm for robust sparse mean estimation.
New method for estimating and testing impulse responses in high-dimensional VAR systems.
problem Statistical inference for impulse responses in sparse, high-dimensional vector autoregressions.
method Local projection equations and de-sparsified estimators combined with a non-regularized contemporaneous impact matrix.
result Valid inference procedures for structural impulse responses in high-dimensional systems.
Efficiently estimates sparse mean from heavy-tailed data.
problem Robustly estimating sparse mean from heavy-tailed distributions.
method Stability-based approach adapted for heavy-tailed data.
result Optimal sample complexity with logarithmic dependence on dimension.
New method estimates sparse mean from noisy data without knowing sparsity level.
problem Sparse mean estimation under adversarial corruptions.
method Incremental learning approach to nonconvex optimization.
result Achieves optimal statistical rate under moderate signal-to-noise ratio.
Scaled sparse linear regression jointly estimates the regression coefficients and noise level in a linear model. It chooses an equilibrium with a sparse regression method by iteratively estimating the noise level via the mean residual square and scaling the penalty in proportion to the estimated noise level. The iterat…
Robust estimators for Gaussian sparse tasks with optimal error under contamination.
problem Robust mean estimation, PCA, and linear regression in the presence of Huber contamination.
method Novel multidimensional filtering method for sparse regime.
result Optimal error guarantees within constant factors for Gaussian robust k-sparse mean estimation. A new algorithm improves GLasso for sparse precision matrix estimation.
problem Efficiently estimating sparse precision matrices in high-dimensional data.
method A new reparametrization and iterative block coordinate descent algorithm.
result Improved performance comparable to DP-GLasso with a simpler optimization target.
Efficiently estimates quantiles and maximum in unbounded datasets with differential privacy.
problem Efficiently estimating quantiles and maximum in unbounded datasets with differential privacy.
method Simple invocation of a subroutine called AboveThreshold, iteratively called in Sparse Vector Technique.
result Improved estimates on highest quantiles with robustness and accuracy.
New method estimates sparse canonical vectors efficiently.
problem Sparse canonical vectors estimation in CCA.
method Quasi-Bayesian estimation via Rayleigh quotient function.
result Achieves minimax rate with low computational cost.
IVF k-means algorithm improves performance on large sparse data sets.
problem Efficiently clustering large-scale sparse data sets with numerous classes.
method Sparse data representation and inverted-file structure for high-speed and low-memory clustering.
result IVF achieves better performance than other algorithms on real document data sets.
Using a Bayesian approach, we consider the problem of recovering sparse signals under additive sparse and dense noise. Typically, sparse noise models outliers, impulse bursts or data loss. To handle sparse noise, existing methods simultaneously estimate the sparse signal of interest and the sparse noise of no interest.…
Study improves error bounds for sparse regression with heavy-tailed covariates.
problem Estimating sparse coefficients in linear regression with heavy-tailed covariates.
method Employed an ℓ1-penalized Huber regression method. result Error bound identical to Gaussian case for L-subexponential covariates. We consider adaptive system identification problems with convex constraints and propose a family of regularized Least-Mean-Square (LMS) algorithms. We show that with a properly selected regularization parameter the regularized LMS provably dominates its conventional counterpart in terms of mean square deviations. We es…
New methods solve sparse estimation robustly, even with outliers.
problem Sparse estimation in high-dimensional data with outliers.
method Non-convex optimization formulations for robust sparse mean estimation and PCA.
result Any approximate stationary point yields near-optimal solutions.
Bayesian Lasso Sparse model provides sparse estimates in linear and nonlinear regression.
problem Sparse learning in regression models.
method Develops a new sparse learning model using type-II maximum likelihood procedure.
result The BLS model provides sparse estimates and is more precise, especially with noisy data.
Sparse group Lasso optimizes sparse and grouped parameters in high-dimensional data.
problem Simultaneously sparse and grouped parameters in high-dimensional linear regression.
method Sparse group Lasso, debiased sparse group Lasso, statistical inference.
result Matching upper and lower bounds on sample complexity and estimation error.
Estimates mean of distributed vectors with sparsification and spatial/temporal correlations.
problem Estimating mean of high-dimensional vectors distributed across nodes with low communication cost.
method Modifies decoding method to leverage spatial and temporal correlations in sparsified vectors.
result Estimators consistently outperform more sophisticated sparsification methods.
SpinSVAR estimates SVAR models with sparse input, improving accuracy and scalability.
problem Estimating SVAR models with sparse input assumptions.
method SpinSVAR models input as independent Laplacian variables, enforcing sparsity and using least absolute error regression.
result SpinSVAR outperforms state-of-the-art methods in accuracy and runtime, identifying significant structural shocks.
SLOPE outperforms LASSO in low noise scenarios but is suboptimal in large noise scenarios.
problem Sparse linear regression with high-dimensional data.
method Characterized SLOPE's estimation error under specific conditions and compared it with LASSO and bridge regression.
result SLOPE is optimal for low noise scenarios but suboptimal in large noise scenarios.
Proposes ARSK for robust and sparse clustering.
problem Outliers and high-dimensional noisy variables in K-means clustering.
method Introduces redundant error component and group sparse penalty for robustness, and weights and sparsity control penalty for noisy variables.
result Superior performance in identifying clusters without outliers and informative variables.
The paper analyzes SBL pruning criteria under weakened assumptions.
problem Sparse Bayesian learning hyperparameter divergence and pruning.
method Analyzing marginal likelihood function under weakened Gaussian assumptions.
result Conditions for finite vs infinite hyperparameters lead to F-SBL pruning.
Kernel means are frequently used to represent probability distributions in machine learning problems. In particular, the well known kernel density estimator and the kernel mean embedding both have the form of a kernel mean. Unfortunately, kernel means are faced with scalability issues. A single point evaluation of the …
Paper uses sparse learning to estimate quasi-potential and drift components in stochastic systems.
problem Estimating quasi-potential and drift components in stochastic systems.
method Sparse identification of non-linear dynamics (SINDy) combined with action minimization methods.
result Evaluation of quasi-potential landscape from a single trajectory.
Proposes novel wSVMs for sparse learning and accurate probability estimation.
problem Sparse features with redundant noise limit the performance of existing wSVMs.
method Develops ℓ1-norm and elastic net regularized wSVMs for automatic variable selection and probability estimation. result Elastic net regularized wSVMs achieve superior performance in variable selection and probability estimation.
Privacy improves robustness in statistical estimation.
problem Sparse mean estimation under privacy constraints.
method Sum-of-Squares method and exponential-time mechanisms.
result Private algorithms matching optimal tradeoffs are not known, but achieved via Sum-of-Squares.
This article considers the problem of sparse estimation of canonical vectors in linear discriminant analysis when p≫N. Several methods have been proposed in the literature that estimate one canonical vector in the two-group case. However, G−1 canonical vectors can be considered if the number of groups is G. In…
Optimal algorithm for high-dimensional stochastic linear bandits with sparse parameters.
problem High-dimensional stochastic linear bandits with sparse parameters.
method Three-stage arm selection algorithm using thresholded Lasso for estimation.
result Achieves exact minimax optimality in cumulative regret.
We study the problem of multivariate regression where the data are naturally grouped, and a regression matrix is to be estimated for each group. We propose an approach in which a dictionary of low rank parameter matrices is estimated across groups, and a sparse linear combination of the dictionary elements is estimated…
In this paper we derive the optimal linear shrinkage estimator for the high-dimensional mean vector using random matrix theory. The results are obtained under the assumption that both the dimension p and the sample size n tend to infinity in such a way that p/n→c∈(0,∞). Under weak conditions imposed on…
We propose a Bayesian expectation-maximization (EM) algorithm for reconstructing Markov-tree sparse signals via belief propagation. The measurements follow an underdetermined linear model where the regression-coefficient vector is the sum of an unknown approximately sparse signal and a zero-mean white Gaussian noise wi…
We consider high-dimensional quadratic classifiers in non-sparse settings. The target of classification rules is not Bayes error rates in the context. The classifier based on the Mahalanobis distance does not always give a preferable performance even if the populations are normal distributions having known covariance m…
The paper examines conditions for linearity in a conditional mean estimator under vector Poisson noise.
problem Conditions for linearity of the conditional mean estimator in vector Poisson noise.
method Analyzes prior distributions and their impact on the conditional mean estimator's linearity.
result The only prior distribution that induces linearity is a product gamma distribution, and non-zero dark current parameter prevents linearity.
Paper analyzes BIHT for noisy 1-bit CS, improving results with up to τ-fraction of incorrect measurements.
problem Estimating sparse vectors from noisy sign measurements in 1-bit compressed sensing.
method Binary Iterative Hard Thresholding (BIHT) algorithm, using Gaussian matrices and high-dimensional geometry analysis.
result BIHT provides estimates within ε+τ error with τ-fraction of incorrect measurements, maintaining universality of measurements.
Quantum algorithm improves sparse vector recovery from noisy measurements.
problem Accurately recover sparse vectors from noisy linear measurements.
method Formulated as a QUBO task, solved using quantum technology.
result Quantum approach outperforms classical methods in sparse coding.
We study sparse principal component analysis for high dimensional vector autoregressive time series under a doubly asymptotic framework, which allows the dimension d to scale with the series length T. We treat the transition matrix of time series as a nuisance parameter and directly apply sparse principal component…
A new model for sparse networks improves consistency of score estimators.
problem Statistical inference in large sparse networks.
method General pairwise comparison model with flexible parametrization.
result Maximum likelihood estimator is uniformly consistent under sparse conditions.
Study on estimating sparse transition matrix of partially-observed VAR with noisy and sparse data.
problem Estimating sparse transition matrix of partially-observed VAR with noisy and sparse data.
method Yule-Walker equation, Dantzig selector, minimax lower bound.
result Near-optimality of the proposed estimator with convergence rate analysis.
Regularization improves logistic regression performance in high-dimensional settings.
problem Improving logistic regression in scenarios with many parameters and observations.
method Introducing a convex regularizer to the negative log-likelihood function to encourage desired structures.
result Explicit expressions for various performance metrics of regularized logistic regression are derived.
New DP optimization methods for sparse gradients, improving on existing algorithms.
problem Differentially private optimization with sparse gradients in high-dimensional settings.
method Improved bounds for mean estimation, pure- and approximate-DP algorithms for stochastic convex optimization.
result First nearly dimension-independent rates for DP optimization with sparse gradients.
We provide a unified treatment of a broad class of noisy structure recovery problems, known as structured normal means problems. In this setting, the goal is to identify, from a finite collection of Gaussian distributions with different means, the distribution that produced some observed data. Recent work has studied s…
We study the value of information in sequential compressed sensing by characterizing the performance of sequential information guided sensing in practical scenarios when information is inaccurate. In particular, we assume the signal distribution is parameterized through Gaussian or Gaussian mixtures with estimated mean…