arXiv research

This paper analyzes generalization for linear models with spiked covariance structures.

Self-distillation optimally improves model performance in spiked covariance models.

A new QDA classifier for high-dimensional data with spiked covariance.

Improved LDA method for better classification and dimensionality reduction.

Study eigenvalues and eigenvectors in neural networks, focusing on signal propagation.

This work investigates how gradient-based learning performs with structured data, revealing issues and improvements.

Spike-and-slab priors are improved for high-dimensional Bayesian regression.

Adaptive classifier optimizes high-dimensional data with spiked covariance structure.

Paper optimizes private PCA for covariance estimation in statistics.

The paper examines how spike strengths and alignments affect overfitting in linear regression models.

Model improves covariance estimation from shared and distinct datasets.

New combinatorial method for sparse PCA works beyond spiked identity model.

We consider streaming, one-pass principal component analysis (PCA), in the high-dimensional regime, with limited memory. Here, $p$-dimensional samples are presented sequentially, and the goal is to produce the $k$-dimensional subspace that best approximates these points. Standard algorithms require $O(p^2)$ memory; mea…

We study risk of the minimum norm linear least squares estimator in when the number of parameters $d$ depends on $n$, and $\frac{d}{n} \rightarrow \infty$. We assume that data has an underlying low rank structure by restricting ourselves to spike covariance matrices, where a fixed finite number of eigenvalues grow with…

The paper improves classification accuracy by leveraging a shared signal across domains in high-dimensional classification.

RFMs transition from linear to nonlinear under specific input-label correlation.

Anisotropic data structure affects learning dynamics and generalization error in linear networks.

Attention learns PCA on Gaussian data, proving its connection to principal component analysis.

Algorithm recovers spike direction from modulo-reduced measurements in high dimensions.

CONCERT improves transfer learning by borrowing partial information from auxiliary datasets.

PLS-SVD struggles with missing data in multimodal datasets, showing a phase transition in performance.

We study the problem of detecting the presence of a single unknown spike in a rectangular data matrix, in a high-dimensional regime where the spike has fixed strength and the aspect ratio of the matrix converges to a finite limit. This setup includes Johnstone's spiked covariance model. We analyze the likelihood ratio …

Estimation of the covariance matrix of asset returns from high frequency data is complicated by asynchronous returns, market mi- crostructure noise and jumps. One technique for addressing both asynchronous returns and market microstructure is the Kalman-EM (KEM) algorithm. However the KEM approach assumes log-normal pr…

RBM learns in high dimensions via AMP and GD, reaching optimal weak recovery.

SpecGD mitigates misalignment in phase retrieval models with anisotropic inputs.

Algorithm detects and estimates correlated signals in spiked matrices.

New model shows weak teachers can help strong students learn even with imperfect labels.

Paper optimizes federated PCA for covariance estimation under privacy constraints.

New method for estimating covariance with robustness to outliers.

Paper develops methods for PCA inference with missing data and heteroskedastic noise.

Fast algorithm recovers principal eigenvector from noisy matrices.

Estimating the leading principal components of data, assuming they are sparse, is a central task in modern high-dimensional statistics. Many algorithms were developed for this sparse PCA problem, from simple diagonal thresholding to sophisticated semidefinite programming (SDP) methods. A key theoretical question is und…

We study the problem of detecting an abrupt change to the signal covariance matrix. In particular, the covariance changes from a "white" identity matrix to an unknown spiked or low-rank matrix. Two sequential change-point detection procedures are presented, based on the largest and the smallest eigenvalues of the sampl…

Study reveals limits of PLS in multi-modal learning with correlated signals.

A general framework for principal component analysis (PCA) in the presence of heteroskedastic noise is introduced. We propose an algorithm called HeteroPCA, which involves iteratively imputing the diagonal entries of the sample covariance matrix to remove estimation bias due to heteroskedasticity. This procedure is com…

Single-spike neurons can approximate as well as multi-spike neurons.

Bayesian framework selects features and lags for time series forecasting.

Much of studies on neural computation are based on network models of static neurons that produce analog output, despite the fact that information processing in the brain is predominantly carried out by dynamic neurons that produce discrete pulses called spikes. Research in spike-based computation has been impeded by th…

Neurons perform computations, and convey the results of those computations through the statistical structure of their output spike trains. Here we present a practical method, grounded in the information-theoretic analysis of prediction, for inferring a minimal representation of that structure and for characterizing its…

We study sparse principal components analysis in high dimensions, where $p$ (the number of variables) can be much larger than $n$ (the number of observations), and analyze the problem of estimating the subspace spanned by the principal eigenvectors of the population covariance matrix. We introduce two complementary not…

Proposes a flexible MGP model for dynamic, sparse correlations.

SNNs enhance high-frequency price spike forecasting in HFT environments.

SNNs can represent complex functions efficiently.

Third-generation neural networks, or Spiking Neural Networks (SNNs), aim at harnessing the energy efficiency of spike-domain processing by building on computing elements that operate on, and exchange, spikes. In this paper, the problem of training a two-layer SNN is studied for the purpose of classification, under a Ge…

New algorithm optimizes positions of CountSketch non-zero entries for better data compression.

Affine spiking neural networks learn efficiently and generalize well.

Developing electrophysiological recordings of brain neuronal activity and their analysis provide a basis for exploring the structure of brain function and nervous system investigation. The recorded signals are typically a combination of spikes and noise. High amounts of background noise and possibility of electric sign…

Extracting and detecting spike activities from the fluorescence observations is an important step in understanding how neuron systems work. The main challenge lies in that the combination of the ambient noise with dynamic baseline fluctuation, often contaminates the observations, thereby deteriorating the reliability o…