arXiv research

This paper analyzes generalization for linear models with spiked covariance structures.

Self-distillation optimally improves model performance in spiked covariance models.

The paper analyzes the risk of a least squares estimator under a spike covariance model.

A new QDA classifier for high-dimensional data with spiked covariance.

Improved LDA method for better classification and dimensionality reduction.

This work investigates how gradient-based learning performs with structured data, revealing issues and improvements.

Study eigenvalues and eigenvectors in neural networks, focusing on signal propagation.

Model improves covariance estimation from shared and distinct datasets.

The paper examines how spike strengths and alignments affect overfitting in linear regression models.

Paper optimizes private PCA for covariance estimation in statistics.

New combinatorial method for sparse PCA works beyond spiked identity model.

Spike-and-slab priors are improved for high-dimensional Bayesian regression.

We consider streaming, one-pass principal component analysis (PCA), in the high-dimensional regime, with limited memory. Here, $p$-dimensional samples are presented sequentially, and the goal is to produce the $k$-dimensional subspace that best approximates these points. Standard algorithms require $O(p^2)$ memory; mea…

Adaptive classifier optimizes high-dimensional data with spiked covariance structure.

The paper improves classification accuracy by leveraging a shared signal across domains in high-dimensional classification.

RFMs transition from linear to nonlinear under specific input-label correlation.

We study the problem of detecting the presence of a single unknown spike in a rectangular data matrix, in a high-dimensional regime where the spike has fixed strength and the aspect ratio of the matrix converges to a finite limit. This setup includes Johnstone's spiked covariance model. We analyze the likelihood ratio …

RBM learns in high dimensions via AMP and GD, reaching optimal weak recovery.

Estimation of the covariance matrix of asset returns from high frequency data is complicated by asynchronous returns, market mi- crostructure noise and jumps. One technique for addressing both asynchronous returns and market microstructure is the Kalman-EM (KEM) algorithm. However the KEM approach assumes log-normal pr…

Anisotropic data structure affects learning dynamics and generalization error in linear networks.

Algorithm detects and estimates correlated signals in spiked matrices.

Algorithm recovers spike direction from modulo-reduced measurements in high dimensions.

SpecGD mitigates misalignment in phase retrieval models with anisotropic inputs.

PLS-SVD struggles with missing data in multimodal datasets, showing a phase transition in performance.

Attention learns PCA on Gaussian data, proving its connection to principal component analysis.

CONCERT improves transfer learning by borrowing partial information from auxiliary datasets.

New model shows weak teachers can help strong students learn even with imperfect labels.

Fast algorithm recovers principal eigenvector from noisy matrices.

Paper develops methods for PCA inference with missing data and heteroskedastic noise.

Study reveals limits of PLS in multi-modal learning with correlated signals.

Estimating the leading principal components of data, assuming they are sparse, is a central task in modern high-dimensional statistics. Many algorithms were developed for this sparse PCA problem, from simple diagonal thresholding to sophisticated semidefinite programming (SDP) methods. A key theoretical question is und…

Bayesian framework selects features and lags for time series forecasting.

Paper optimizes federated PCA for covariance estimation under privacy constraints.

New method for estimating covariance with robustness to outliers.

A general framework for principal component analysis (PCA) in the presence of heteroskedastic noise is introduced. We propose an algorithm called HeteroPCA, which involves iteratively imputing the diagonal entries of the sample covariance matrix to remove estimation bias due to heteroskedasticity. This procedure is com…

Single-spike neurons can approximate as well as multi-spike neurons.

Much of studies on neural computation are based on network models of static neurons that produce analog output, despite the fact that information processing in the brain is predominantly carried out by dynamic neurons that produce discrete pulses called spikes. Research in spike-based computation has been impeded by th…

New method improves neural spike train models by minimizing divergence directly, leading to better performance.

Proposes a flexible MGP model for dynamic, sparse correlations.

SNNs enhance high-frequency price spike forecasting in HFT environments.

Third-generation neural networks, or Spiking Neural Networks (SNNs), aim at harnessing the energy efficiency of spike-domain processing by building on computing elements that operate on, and exchange, spikes. In this paper, the problem of training a two-layer SNN is studied for the purpose of classification, under a Ge…

Neurons perform computations, and convey the results of those computations through the statistical structure of their output spike trains. Here we present a practical method, grounded in the information-theoretic analysis of prediction, for inferring a minimal representation of that structure and for characterizing its…

We study the problem of detecting an abrupt change to the signal covariance matrix. In particular, the covariance changes from a "white" identity matrix to an unknown spiked or low-rank matrix. Two sequential change-point detection procedures are presented, based on the largest and the smallest eigenvalues of the sampl…

New algorithm optimizes positions of CountSketch non-zero entries for better data compression.

This work connects diffusion models to power iteration, revealing how low frequencies emerge earlier.

PliableBVS extends Bayesian lasso for modeling interactions with modifying variables.

Accurate statistical models of neural spike responses can characterize the information carried by neural populations. But the limited samples of spike counts during recording usually result in model overfitting. Besides, current models assume spike counts to be Poisson-distributed, which ignores the fact that many neur…

Extracting and detecting spike activities from the fluorescence observations is an important step in understanding how neuron systems work. The main challenge lies in that the combination of the ambient noise with dynamic baseline fluctuation, often contaminates the observations, thereby deteriorating the reliability o…