Develops path integral for spiked tensor model dynamics.
problem Dynamics of spiked tensor model with random initial conditions.
method Path integral approach applied to partial differential equations.
result Large-N saddle point equations dominated by melonic diagrams. Study reveals efficient recovery of multi-modal signals via Bayesian methods and sequential learning.
problem Recovering multiple high-dimensional signals from correlated modalities.
method Bayesian Approximate Message Passing and Sequential Curriculum Learning.
result Sequential learning strategy optimally recovers weak signals in multi-modal settings.
SGD recovers multiple signal vectors in noisy tensor PCA.
problem Estimating multiple signal vectors from noisy tensor observations.
method Online stochastic gradient descent (SGD) in high dimensions with detailed analysis of correlations.
result Sequential elimination of correlations allows recovery of all spikes from Np−2 samples. QAOA matches classical tensor power iteration in spiked tensor model recovery.
problem Statistical estimation in spiked tensor model with computational gap.
method Analysis of QAOA performance on spiked tensor model.
result QAOA weak recovery threshold matches tensor power iteration.
Paper studies tensor models using random matrix theory.
problem Analyzing asymmetric order-d spiked tensor models with Gaussian noise.
method Uses variational definition of singular vectors and values, constructs equivalent spiked symmetric block-wise random matrix from tensor contractions.
result Characterizes asymptotic singular values and alignments of singular vectors with true spike components.
Study of Langevin dynamics for tensor PCA recovery in high dimensions.
problem Recovering hidden signal vectors (spikes) from noisy Gaussian tensor observations.
method Langevin dynamics approach for nonconvex optimization.
result Sample complexity matches the single-spike case but degrades for all spikes.
Study recovers spike order in noisy tensor estimation without SNR assumptions.
problem Estimating multiple signal vectors from noisy tensor observations.
method Gradient flow optimization of a nonconvex function.
result Determines sample complexity for efficient permutation recovery.
The paper analyzes deflation for estimating a low-rank spike in large tensors with noise.
problem Estimating a low-rank symmetric spike in large tensors with additive Gaussian noise.
method Characterization of deflation performance in terms of vector alignments and weights.
result Understanding deflation mechanism in noisy conditions and designing more efficient methods.
Paper optimizes tensor deflation for non-orthogonal signals.
problem Recovering low-rank signals from noisy tensors with correlated components.
method Developed an asymptotic analysis and optimized deflation procedure using random tensor theory.
result Proposed an efficient tensor deflation algorithm that optimizes a parameter introduced in the deflation mechanism.
Study analyzes Hotelling-type tensor deflation for spiked tensors, providing insights into signal and noise.
problem Characterizing singular values and alignments in Hotelling-type tensor deflation.
method Asymptotic study of Hotelling-type tensor deflation in large dimensional regime using random tensor theory.
result Characterization of singular values and alignments at each step of the deflation procedure.
Study of asymmetric rank-one tensor models with non-Gaussian noise.
problem Analyzing maximum-likelihood estimators for asymmetric rank-one tensor models.
method Spectrally separated branch analysis, resolvent methods, cumulant expansions, Efron-Stein-type variance bounds.
result Asymptotic singular value and mode-wise alignments are robust to non-Gaussian noise.
New estimator for tensor weights with improved bias.
problem Estimating tensor weights from noisy data.
method Random matrix theory and KKT conditions.
result Asymptotically unbiased estimator for tensor rank.
Estimates rank-one spikes from heavy-tailed noise using self-avoiding walks.
problem Estimating rank-one spikes from heavy-tailed noise.
method Self-avoiding walks to count and estimate the spikes.
result Optimal estimation up to the BBP threshold for heavy-tailed noise.
Paper proposes a tensor model for clustering noisy multi-view data.
problem Clustering noisy multi-view data with non-uniform variances.
method Nested matrix-tensor model for best rank-one approximation.
result Theoretical results predict the exact accuracy of clustering.
Bayesian model predicts phenotype effects from multi-environmental factors.
problem Predict phenotype effects from multi-environmental trials.
method Bayesian tensor regression with spike-and-slab structure.
result Model outperforms previous methods in simulation and real-world data.
Study on estimating rank-one tensors in noisy data with heavy tails.
problem Estimating rank-one spiked tensors in the presence of heavy tailed errors.
method Analysis of spectral norm of random tensors with iid entries.
result Signal strength requirements for optimal estimation are similar for heavy tailed and Gaussian noise, but vanish for noise with finite fourth moment.
In this work we analyse quantitatively the interplay between the loss landscape and performance of descent algorithms in a prototypical inference problem, the spiked matrix-tensor model. We study a loss function that is the negative log-likelihood of the model. We analyse the number of local minima at a fixed distance …
Study efficient power iteration for tensor models, proving convergence under specific conditions.
problem Simultaneous alternating power iteration for fixed-order asymmetric rank-one spiked tensor models.
method Finite-iteration local theory, geometrically decaying transient, fixed-order multilinear noise event, warm-start mechanism.
result Convergence to the unique informative local fixed point under specific conditions.
We study rough high-dimensional landscapes in which an increasingly stronger preference for a given configuration emerges. Such energy landscapes arise in glass physics and inference. In particular we focus on random Gaussian functions, and on the spiked-tensor model and generalizations. We thoroughly analyze the stati…
Study quantifies performance gap between tensor and matrix-based approaches in nested matrix-tensor model.
problem Estimating a planted signal in a nested matrix-tensor model.
method Comparing tensor-based and matrix-based approaches for best rank-one approximation of tensor data.
result Derives precise algorithmic threshold for the unfolding approach and shows BBP-type transition behavior.
Gradient-based algorithms are effective for many machine learning tasks, but despite ample recent effort and some progress, it often remains unclear why they work in practice in optimising high-dimensional non-convex functions and why they find good minima instead of being trapped in spurious ones. Here we present a qu…
The paper studies phase transitions in random matrices and tensor unfolding for detecting signals.
problem Phase transitions in singular values and vectors of large random matrices.
method Analysis of singular values and vectors of long rectangular random matrices, and tensor unfolding algorithm for asymmetric rank-one spiked tensor models.
result An exact threshold for tensor unfolding to detect signals, independent of unfolding procedure.
Study spinorial Yamabe problem on product manifolds, proving spike layer solutions.
problem Existence of solutions on product manifolds with curvature effect.
method Variational methods and blow-up techniques.
result Existence of solutions depending only on one factor and exhibiting spike layers.
SPIDER uses deep neural networks for streaming tensor factorization.
problem Lack of effective approach for deep tensor factorization of streaming data.
method Bayesian neural networks with spike-and-slab prior, Taylor expansions, moment matching, and EPI framework.
result Effective incremental updates for latent factors and NN weights.
SMPI recovers tensor spikes from noisy data with improved performance.
problem Recovering tensor spikes corrupted by Gaussian noise.
method Selective Multiple Power Iterations (SMPI) with polynomial random initializations and symmetrized tensor power iterations.
result SMPI outperforms existing algorithms and approaches theoretical optimal recovery.
Single-spike neurons can approximate as well as multi-spike neurons.
problem Limitation of single-spike neurons in spiking neural networks.
method Comparison of single-spike and multi-spike neural networks.
result Single-spike and multi-spike neural networks are equivalent in approximation capabilities.
Study uses random matrix theory to improve tensor approximation accuracy.
problem Improving tensor approximation accuracy in the presence of noise.
method Random matrix theory applied to tensor unfoldings.
result Characterizes spectral behavior of tensor unfoldings and predicts reconstruction performance.
New insights into SGD and SGD-M in high dimensions.
problem Understanding and comparing SGD and SGD-M in high-dimensional settings.
method Developed high-dimensional scaling limits for SGD-M and online SGD, examining their dynamics and performance.
result SGD-M amplifies high-dimensional effects, potentially degrading performance compared to online SGD.
A new method discovers equations from data using Bayesian and kernel techniques.
problem Discovering equations from data is hard due to sparsity and noise.
method Kernel regression for function estimation and Bayesian spike-and-slab prior for uncertainty quantification.
result KBASS method outperforms state-of-the-art methods on benchmark tasks.
AMP algorithm for matrix tensor product model provides recovery conditions.
problem Generalization of standard spiked matrix models with multiple pairwise observations.
method Approximate message passing with optimal weighing and combining of estimates.
result Asymptotically exact performance description and necessary/sufficient recovery conditions.
Much of studies on neural computation are based on network models of static neurons that produce analog output, despite the fact that information processing in the brain is predominantly carried out by dynamic neurons that produce discrete pulses called spikes. Research in spike-based computation has been impeded by th…
The paper examines how spike strengths and alignments affect overfitting in linear regression models.
problem The impact of spike strengths and alignments on overfitting in linear regression models.
method Characterization of generalization error through exact expressions and analysis of spike strengths, aspect ratio, and target alignment.
result Increasing spike strength can lead to catastrophic overfitting before benign overfitting, especially in well-specified aligned problems.
New method improves neural spike train models by minimizing divergence directly, leading to better performance.
problem Poor performance and divergence issues in spike train models using maximum likelihood estimation.
method Directly minimize maximum mean discrepancy using spike train kernels and stochastic optimization.
result The proposed method generates well-behaved models with better control over feature trade-offs.
Study accelerates optimization methods in non-convex problems, but doesn't improve the algorithm's performance.
problem Understanding the behavior of momentum-based acceleration methods in non-convex, high-dimensional landscapes.
method Used dynamical mean field theory to describe the average dynamics of heavy-ball momentum and Nesterov acceleration in a non-convex model.
result Accelerated dynamics but did not improve the algorithm's performance with respect to gradient descent.
SNNs enhance high-frequency price spike forecasting in HFT environments.
problem Conventional financial models fail to capture fine temporal structure in high-frequency price spikes.
method Application of Spiking Neural Networks (SNNs) with hyperparameter tuning via Bayesian Optimization (BO).
result SNN models optimized with PSA achieve significantly higher cumulative returns in backtesting.
We consider the Principal Component Analysis problem for large tensors of arbitrary order k under a single-spike (or rank-one plus noise) model. On the one hand, we use information theory, and recent results in probability theory, to establish necessary and sufficient conditions under which the principal component ca…
Third-generation neural networks, or Spiking Neural Networks (SNNs), aim at harnessing the energy efficiency of spike-domain processing by building on computing elements that operate on, and exchange, spikes. In this paper, the problem of training a two-layer SNN is studied for the purpose of classification, under a Ge…
Neurons perform computations, and convey the results of those computations through the statistical structure of their output spike trains. Here we present a practical method, grounded in the information-theoretic analysis of prediction, for inferring a minimal representation of that structure and for characterizing its…
New algorithms improve tensor CP decomposition under mild conditions.
problem Improving tensor CP decomposition with theoretical guarantees under mild incoherence conditions.
method Composite PCA and Concurrent Orthogonalization algorithms.
result Theoretical guarantees and practical superiority over existing methods.
This paper analyzes generalization for linear models with spiked covariance structures.
problem Understanding the generalization performance of linear models with spiked covariance structures.
method Derives the generalization error for two simple models with spiked covariances using random matrix theory.
result The eigenvector and eigenvalue corresponding to the spike significantly influence the generalization error.
Accurate statistical models of neural spike responses can characterize the information carried by neural populations. But the limited samples of spike counts during recording usually result in model overfitting. Besides, current models assume spike counts to be Poisson-distributed, which ignores the fact that many neur…
Algorithm detects and estimates correlated signals in spiked matrices.
problem Detect and estimate correlated signals in spiked matrices.
method Proposes an efficient algorithm based on counting edge-decorated cycles.
result Algorithm succeeds under certain signal-to-noise ratio conditions.
Extracting and detecting spike activities from the fluorescence observations is an important step in understanding how neuron systems work. The main challenge lies in that the combination of the ambient noise with dynamic baseline fluctuation, often contaminates the observations, thereby deteriorating the reliability o…
A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or "spike") is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughou…
Method reconstructs neuron models from spike times efficiently.
problem Reconstructing neuron models from spike times in degenerate populations.
method Combining deep learning with DICs to map spike times to DIC densities and generate degenerate CBM populations.
result Fast and scalable reconstruction of degenerate populations from spike recordings.
Develops a new point process model for detecting neural spike sequences.
problem Detecting sparse sequences of neural spikes in high-dimensional spike trains.
method A point process model that represents sequence occurrences as marked events in continuous time, with learnable time warping parameters.
result Demonstrates improved detection and modeling of neural spike sequences.
This paper investigates the effect of leak in spiking neural networks.
problem Comparative analysis of leaky and non-leaky spiking neuron models.
method Experimental and frequency domain analysis of spiking neural networks.
result Leaky spiking neuron models provide improved robustness and generalization but decrease sparsity of computation.
This paper presents a supervised learning algorithm, namely, the Synaptic Efficacy Function with Meta-neuron based learning algorithm (SEF-M) for a spiking neural network with a time-varying weight model. For a given pattern, SEF-M uses the learning algorithm derived from meta-neuron based learning algorithm to determi…