A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
The study explains delayed spikes in batch-normalized models.
This paper explains how low-precision arithmetic causes loss spikes in deep learning models.
Stable GFlowNets prevent loss spikes and mode collapse in training.
The paper explains spikes in training loss as catapults, improving feature learning and generalization.
New method converts conventional ANNs to SNNs with minimal loss and efficiency.
Spiking neural networks (SNNs) offer a promising alternative to current artificial neural networks to enable low-power event-driven neuromorphic hardware. Spike-based neuromorphic applications require processing and extracting meaningful information from spatio-temporal data, represented as series of spike trains over …
Sleep-based regularization stabilizes STDP in recurrent neural networks.
In this work we analyse quantitatively the interplay between the loss landscape and performance of descent algorithms in a prototypical inference problem, the spiked matrix-tensor model. We study a loss function that is the negative log-likelihood of the model. We analyse the number of local minima at a fixed distance …
Improved VB algorithm for high-dimensional logistic regression with theoretical guarantees.
Framework for training stochastic spiking neural networks with rough signals.
Spiking Neural Networks (SNNs) are brain-inspired, event-driven machine learning algorithms that have been widely recognized in producing ultra-high-energy-efficient hardware. Among existing SNNs, unsupervised SNNs based on synaptic plasticity, especially Spike-Timing-Dependent Plasticity (STDP), are considered to have…
Deep convolutional neural networks (CNNs) have shown great potential for numerous real-world machine learning applications, but performing inference in large CNNs in real-time remains a challenge. We have previously demonstrated that traditional CNNs can be converted into deep spiking neural networks (SNNs), which exhi…
Estimates neuronal connectivity from spike times using flexible Hawkes processes.
Study shows how specialized attention circuits emerge during transformer training.
Single-spike neurons can approximate as well as multi-spike neurons.
The study examines the dynamic behavior of RMSprop and Adam algorithms.
Much of studies on neural computation are based on network models of static neurons that produce analog output, despite the fact that information processing in the brain is predominantly carried out by dynamic neurons that produce discrete pulses called spikes. Research in spike-based computation has been impeded by th…
Neurons perform computations, and convey the results of those computations through the statistical structure of their output spike trains. Here we present a practical method, grounded in the information-theoretic analysis of prediction, for inferring a minimal representation of that structure and for characterizing its…
The paper examines how spike strengths and alignments affect overfitting in linear regression models.
This paper introduces a new learning rule for probabilistic SNNs that improves log-likelihood, accuracy, and calibration.
SNNs enhance high-frequency price spike forecasting in HFT environments.
SNNs can represent complex functions efficiently.
The paper reveals low-rank structure in neural network gradients, influenced by data and model parameters.
Third-generation neural networks, or Spiking Neural Networks (SNNs), aim at harnessing the energy efficiency of spike-domain processing by building on computing elements that operate on, and exchange, spikes. In this paper, the problem of training a two-layer SNN is studied for the purpose of classification, under a Ge…
Affine spiking neural networks learn efficiently and generalize well.
Developing electrophysiological recordings of brain neuronal activity and their analysis provide a basis for exploring the structure of brain function and nervous system investigation. The recorded signals are typically a combination of spikes and noise. High amounts of background noise and possibility of electric sign…
Extracting and detecting spike activities from the fluorescence observations is an important step in understanding how neuron systems work. The main challenge lies in that the combination of the ambient noise with dynamic baseline fluctuation, often contaminates the observations, thereby deteriorating the reliability o…
New method improves neural spike train models by minimizing divergence directly, leading to better performance.
T2FSNN improves deep SNNs by reducing spikes and latency.
A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or "spike") is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughou…
New method uses surrogate gradients to train efficient spiking networks on neuromorphic hardware.
This research bridges binary and spiking neural networks for efficient on-chip AI.
The machine learning community has become increasingly interested in the energy efficiency of neural networks. The Spiking Neural Network (SNN) is a promising approach to energy-efficient computing, since its activation levels are quantized into temporally sparse, one-bit values (i.e., "spike" events), which additional…
A vast majority of computation in the brain is performed by spiking neural networks. Despite the ubiquity of such spiking, we currently lack an understanding of how biological spiking neural circuits learn and compute in-vivo, as well as how we can instantiate such capabilities in artificial spiking circuits in-silico.…
Study recovers spike order in noisy tensor estimation without SNR assumptions.
This paper analyzes generalization for linear models with spiked covariance structures.
This paper suggests a learning-theoretic perspective on how synaptic plasticity benefits global brain functioning. We introduce a model, the selectron, that (i) arises as the fast time constant limit of leaky integrate-and-fire neurons equipped with spiking timing dependent plasticity (STDP) and (ii) is amenable to the…
Algorithm detects and estimates correlated signals in spiked matrices.
Accurate statistical models of neural spike responses can characterize the information carried by neural populations. But the limited samples of spike counts during recording usually result in model overfitting. Besides, current models assume spike counts to be Poisson-distributed, which ignores the fact that many neur…
We develop a new theoretical framework, the \emph{envelope complexity}, to analyze the minimax regret with logarithmic loss functions and derive a Bayesian predictor that adaptively achieves the minimax regret over high-dimensional -balls within a factor of two. The prior is newly derived for achieving the mini…
Epilepsy is an important public health issue. An appropriate epileptiform discharge pattern detection of this neurological disease is a typical problem in biomedical engineering. In this paper, a new method is proposed for spike-and-wave discharge pattern detection based on Kendall's Tau-b coefficient. The proposed app…
Spike-and-slab priors are improved for high-dimensional Bayesian regression.
Spiking neural networks (SNNs) have garnered a great amount of interest for supervised and unsupervised learning applications. This paper deals with the problem of training multi-layer feedforward SNNs. The non-linear integrate-and-fire dynamics employed by spiking neurons make it difficult to train SNNs to generate de…
Develops path integral for spiked tensor model dynamics.
Method reconstructs neuron models from spike times efficiently.
Study of Langevin dynamics for tensor PCA recovery in high dimensions.
Spike-and-slab priors are popular Bayesian solutions for high-dimensional linear regression problems. Previous theoretical studies on spike-and-slab methods focus on specific prior formulations and use prior-dependent conditions and analyses, and thus can not be generalized directly. In this paper, we propose a class o…
Over the past decade, deep neural networks (DNNs) have demonstrated remarkable performance in a variety of applications. As we try to solve more advanced problems, increasing demands for computing and power resources has become inevitable. Spiking neural networks (SNNs) have attracted widespread interest as the third-g…