arXiv research

ACOWA improves distributed sparse classification with extra communication round.

Efficiently approximates Sparse PCA with significant speedups and minor error.

The Lasso performs well in ultra-sparse linear models with finite support size.

Transformers excel at sparse token selection, surpassing FCNs in both worst and average cases.

This paper develops several average-case reduction techniques to show new hardness results for three central high-dimensional statistics problems, implying a statistical-computational gap induced by robustness, a detection-recovery gap and a universality principle for these gaps. A main feature of our approach is to ma…

BatchTopK SAEs improve GPT-2 and Gemma activations with adjustable sparsity.

We devise a one-shot approach to distributed sparse regression in the high-dimensional setting. The key idea is to average "debiased" or "desparsified" lasso estimators. We show the approach converges at the same rate as the lasso as long as the dataset is not split across too many machines. We also extend the approach…

Sparse curves on surfaces grow at a specific intermediate rate.

Bayesian method identifies dynamical models with uncertainty quantification.

In this article the package High-dimensional Metrics (\texttt{hdm}) is introduced. It is a collection of statistical methods for estimation and quantification of uncertainty in high-dimensional approximately sparse models. It focuses on providing confidence intervals and significance testing for (possibly many) low-dim…

New RL method MAC improves performance in sparse reward settings.

Improved Compressed Sensing by optimizing sparse solutions with mixed integer programming.

SAMS-VAE models cellular perturbations using sparse additive mechanisms.

Transformers show strengths and weaknesses in complexity analysis.

Entropy regularization improves sparse model discovery in federated learning.

The package High-dimensional Metrics (\Rpackage{hdm}) is an evolving collection of statistical methods for estimation and quantification of uncertainty in high-dimensional approximately sparse models. It focuses on providing confidence intervals and significance testing for (possibly many) low-dimensional subcomponents…

Dual averaging-type methods are widely used in industrial machine learning applications due to their ability to promoting solution structure (e.g., sparsity) efficiently. In this paper, we propose a novel accelerated dual-averaging primal-dual algorithm for minimizing a composite convex function. We also derive a stoch…

In the past decade, sparse principal component analysis has emerged as an archetypal problem for illustrating statistical-computational tradeoffs. This trend has largely been driven by a line of research aiming to characterize the average-case complexity of sparse PCA through reductions from the planted clique (PC) con…

SLR tackles sparse linear regression problems, showing hardness for efficient algorithms.

Improved trajectory prediction for team sports using sparse outputs.

In this paper, we study the information-theoretic limits of community detection in the symmetric two-community stochastic block model, with intra-community and inter-community edge probabilities $\frac{a}{n}$ and $\frac{b}{n}$ respectively. We consider the sparse setting, in which $a$ and $b$ do not scale with $n$, and…

Improves test set performance and reduces out-of-sample disappointment for unstable models.

MAESTRO improves multimodal learning for dynamic time series with adaptive attention and robustness.

New insights link diverse statistical problems via secret leakage planted clique.

A new method reduces communication costs in distributed learning.

We design a new sparse projection method for a set of vectors that guarantees a desired average sparsity level measured leveraging the popular Hoyer measure (an affine function of the ratio of the $\ell_1$ and $\ell_2$ norms). Existing approaches either project each vector individually or require the use of a regulariz…

EASIER-net uses sparse networks to improve prediction accuracy for high-dimensional data.

Large-scale regression problems where both the number of variables, $p$, and the number of observations, $n$, may be large and in the order of millions or more, are becoming increasingly more common. Typically the data are sparse: only a fraction of a percent of the entries in the design matrix are non-zero. Neverthele…

In the synthesis model signals are represented as a sparse combinations of atoms from a dictionary. Dictionary learning describes the acquisition process of the underlying dictionary for a given set of training samples. While ideally this would be achieved by optimizing the expectation of the factors over the underlyin…

Study on signal recovery from low-rank matrix with sparse noise.

Sparse oblique decision tree improves security rules for renewable power systems.

New method for sparse kernel selection improves prediction accuracy.

We propose a voted dual averaging method for online classification problems with explicit regularization. This method employs the update rule of the regularized dual averaging (RDA) method, but only on the subsequence of training examples where a classification error is made. We derive a bound on the number of mistakes…

Unified framework HASSLE-free decomposes large model weights into sparse and low-rank components.

This paper examines fundamental error characteristics for a general class of matrix completion problems, where the matrix of interest is a product of two a priori unknown matrices, one of which is sparse, and the observations are noisy. Our main contributions come in the form of minimax lower bounds for the expected pe…

Sharp thresholds and contiguity for community detection in contextual SBM.

Kernel regression is a popular non-parametric fitting technique. It aims at learning a function which estimates the targets for test inputs as precise as possible. Generally, the function value for a test input is estimated by a weighted average of the surrounding training examples. The weights are typically computed b…

A fast method for Lasso and Logistic Lasso problems.

Paper proposes a method to improve variational inference for sparse networks.

RFM reduces feature space for linear models, improving sparse recovery.

Paper presents a randomized algorithm for SPCA with high probability approximation.

A new sparse benchmark metabench identifies key abilities from large benchmarks.

GrateTile optimizes CNN feature map storage for efficient data access.

We consider a random sparse graph with bounded average degree, in which a subset of vertices has higher connectivity than the background. In particular, the average degree inside this subset of vertices is larger than outside (but still bounded). Given a realization of such graph, we aim at identifying the hidden subse…

This paper investigates the average-case time complexity of certifying RIP matrices.

Prototype model improves model auditing and understanding.

Researchers prove it's impossible to partially recover graph alignments in certain conditions.

PEAR dynamically reconfigures agent roles to prevent persistent biases in multi-agent debates.