The study explains delayed spikes in batch-normalized models.
problem Delayed spikes in batch-normalized models.
method Analyzing batch-normalized linear models, deriving conditions for delayed onset and waiting time.
result Explicit conditions for delayed-onset and waiting time in whitened square-loss linear regression.
BayTiDe discovers time-delayed differential equations from noisy data.
problem Discovering time-delayed differential equations from data with large delays and noise.
method Bayesian inference with a sparsity-promoting prior.
result BayTiDe accurately identifies time-delayed differential equations with accuracy proportional to data resolution.
Much of studies on neural computation are based on network models of static neurons that produce analog output, despite the fact that information processing in the brain is predominantly carried out by dynamic neurons that produce discrete pulses called spikes. Research in spike-based computation has been impeded by th…
Configuring deep Spiking Neural Networks (SNNs) is an exciting research avenue for low power spike event based computation. However, the spike generation function is non-differentiable and therefore not directly compatible with the standard error backpropagation algorithm. In this paper, we introduce a new general back…
We introduce Delay Pruning, a simple yet powerful technique to regularize dynamic Boltzmann machines (DyBM). The recently introduced DyBM provides a particularly structured Boltzmann machine, as a generative model of a multi-dimensional time-series. This Boltzmann machine can have infinitely many layers of units but al…
Artificial Neural Network (ANN) based techniques have dominated state-of-the-art results in most problems related to computer vision, audio recognition, and natural language processing in the past few years, resulting in strong industrial adoption from all leading technology companies worldwide. One of the major obstac…
New model reveals balance crucial for robust neural coding.
problem Efficient neural coding in noisy, chaotic networks.
method Analytical model of balanced predictive coding with dissociated balance and weight disorder.
result Superclassical scaling in coding accuracy, independent of balance and weight disorder.
Recently, it has been shown that spiking neural networks (SNNs) can be trained efficiently, in a supervised manner, using backpropagation through time. Indeed, the most commonly used spiking neuron model, the leaky integrate-and-fire neuron, obeys a differential equation which can be approximated using discrete time st…
The article explores how organisms and machines learn and recognize the world using Bayesian inference and thermodynamics.
problem Understanding how organisms and machines learn and recognize the world.
method Introducing a thermodynamic view of the Bayesian brain hypothesis, using a simple generative model of spiking neural populations.
result The process of Bayesian inference can be quantified using entropy, revealing the perceptual capacity of neural activity.
We study the problem of detecting an abrupt change to the signal covariance matrix. In particular, the covariance changes from a "white" identity matrix to an unknown spiked or low-rank matrix. Two sequential change-point detection procedures are presented, based on the largest and the smallest eigenvalues of the sampl…
Single-spike neurons can approximate as well as multi-spike neurons.
problem Limitation of single-spike neurons in spiking neural networks.
method Comparison of single-spike and multi-spike neural networks.
result Single-spike and multi-spike neural networks are equivalent in approximation capabilities.
Neurons perform computations, and convey the results of those computations through the statistical structure of their output spike trains. Here we present a practical method, grounded in the information-theoretic analysis of prediction, for inferring a minimal representation of that structure and for characterizing its…
The paper examines how spike strengths and alignments affect overfitting in linear regression models.
problem The impact of spike strengths and alignments on overfitting in linear regression models.
method Characterization of generalization error through exact expressions and analysis of spike strengths, aspect ratio, and target alignment.
result Increasing spike strength can lead to catastrophic overfitting before benign overfitting, especially in well-specified aligned problems.
SNNs enhance high-frequency price spike forecasting in HFT environments.
problem Conventional financial models fail to capture fine temporal structure in high-frequency price spikes.
method Application of Spiking Neural Networks (SNNs) with hyperparameter tuning via Bayesian Optimization (BO).
result SNN models optimized with PSA achieve significantly higher cumulative returns in backtesting.
SNNs can represent complex functions efficiently.
problem Understanding the representational power of SNNs.
method Viewed as sequence-to-sequence processors, analyzed using spike train functions.
result SNNs have the universal representation property for certain functions.
Third-generation neural networks, or Spiking Neural Networks (SNNs), aim at harnessing the energy efficiency of spike-domain processing by building on computing elements that operate on, and exchange, spikes. In this paper, the problem of training a two-layer SNN is studied for the purpose of classification, under a Ge…
Affine spiking neural networks learn efficiently and generalize well.
problem Learning with spiking neural networks, especially with positive weights.
method Affine encoders and decoders, continuous parameter dependence, gradient-based training.
result Affine spiking neural networks can approximate shallow ReLU networks and generalize well.
Developing electrophysiological recordings of brain neuronal activity and their analysis provide a basis for exploring the structure of brain function and nervous system investigation. The recorded signals are typically a combination of spikes and noise. High amounts of background noise and possibility of electric sign…
Extracting and detecting spike activities from the fluorescence observations is an important step in understanding how neuron systems work. The main challenge lies in that the combination of the ambient noise with dynamic baseline fluctuation, often contaminates the observations, thereby deteriorating the reliability o…
New method improves neural spike train models by minimizing divergence directly, leading to better performance.
problem Poor performance and divergence issues in spike train models using maximum likelihood estimation.
method Directly minimize maximum mean discrepancy using spike train kernels and stochastic optimization.
result The proposed method generates well-behaved models with better control over feature trade-offs.
New method detects spike-and-wave epileptiform discharges using Kendall's Tau-b.
problem Detecting spike-and-wave epileptiform discharges in EEG signals.
method Proposes a new method based on Kendall's Tau-b coefficient.
result High Specificity and rule in (SpPIn) for spike-and-wave discharge detection.
Develops generic spike-and-slab priors for high-dimensional linear regression.
problem Bayesian high-dimensional linear regression challenges.
method Proposes a class of generic spike-and-slab priors and a unified framework for theoretical assessment.
result Achieves nearly-optimal posterior contraction rate and model selection consistency under general conditions.
T2FSNN improves deep SNNs by reducing spikes and latency.
problem Inefficiency in spiking neural networks due to lack of scalable training algorithms.
method Introduces time-to-first-spike coding with kernel-based dynamic threshold and dendrite, combined with gradient-based optimization and early firing methods.
result Reduces inference latency and spike count by 22% and less than 1% compared to burst coding.
A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or "spike") is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughou…
Improved SNNs with quantized activations outperform traditional networks.
problem Maintaining SotA accuracy in SNNs with limited bit precision.
method Interpolating between non-spiking and spiking regimes using signal processing tools.
result First hybrid SNN outperforms traditional RNNs in accuracy with reduced bit precision.
A learning rule for first-spike times in neural networks reduces energy consumption and reaction times.
problem Energy efficiency and reaction time in neuromorphic systems.
method Derivation of a learning rule for first-spike times in leaky integrate-and-fire neurons, using only input and output spike times.
result Demonstrated that the approach can implement error backpropagation in hierarchical spiking networks and is capable of harnessing neuromorphic system's speed and energy characteristics.
New method uses surrogate gradients to train efficient spiking networks on neuromorphic hardware.
problem Training high-performing spiking networks on analog neuromorphic hardware is challenging due to device mismatch and lack of efficient algorithms.
method Introduces a general in-the-loop learning framework based on surrogate gradients.
result Learning self-corrects for device mismatch, resulting in competitive spiking network performance.
This research bridges binary and spiking neural networks for efficient on-chip AI.
problem Reducing compute requirements in machine learning frameworks.
method Training Spiking Neural Networks in extreme quantization regime and utilizing standard training techniques for conversion.
result Training Spiking Neural Networks in extreme quantization regime achieves near full precision accuracies.
A vast majority of computation in the brain is performed by spiking neural networks. Despite the ubiquity of such spiking, we currently lack an understanding of how biological spiking neural circuits learn and compute in-vivo, as well as how we can instantiate such capabilities in artificial spiking circuits in-silico.…
Study recovers spike order in noisy tensor estimation without SNR assumptions.
problem Estimating multiple signal vectors from noisy tensor observations.
method Gradient flow optimization of a nonconvex function.
result Determines sample complexity for efficient permutation recovery.
Develops a stochastic approach to financial market delays.
problem Modeling delays in financial markets with multiple assets.
method Introduces a general stochastic framework for information and order execution delays.
result Delayed markets maintain fundamental asset pricing theorems and no asymptotic free lunch condition.
Paper tackles action delays in reinforcement learning, proposing a delay-aware framework.
problem Action delays degrade reinforcement learning performance in real-world systems.
method Formal definition of delay-aware MDP, transformation into standard MDP with augmented states, delay-aware model-based reinforcement learning framework.
result Proposed framework is more efficient in training and transferable between systems with various delay durations.
This paper analyzes generalization for linear models with spiked covariance structures.
problem Understanding the generalization performance of linear models with spiked covariance structures.
method Derives the generalization error for two simple models with spiked covariances using random matrix theory.
result The eigenvector and eigenvalue corresponding to the spike significantly influence the generalization error.
This paper suggests a learning-theoretic perspective on how synaptic plasticity benefits global brain functioning. We introduce a model, the selectron, that (i) arises as the fast time constant limit of leaky integrate-and-fire neurons equipped with spiking timing dependent plasticity (STDP) and (ii) is amenable to the…
Algorithm detects and estimates correlated signals in spiked matrices.
problem Detect and estimate correlated signals in spiked matrices.
method Proposes an efficient algorithm based on counting edge-decorated cycles.
result Algorithm succeeds under certain signal-to-noise ratio conditions.
Accurate statistical models of neural spike responses can characterize the information carried by neural populations. But the limited samples of spike counts during recording usually result in model overfitting. Besides, current models assume spike counts to be Poisson-distributed, which ignores the fact that many neur…
Spiking neural networks (SNNs) offer a promising alternative to current artificial neural networks to enable low-power event-driven neuromorphic hardware. Spike-based neuromorphic applications require processing and extracting meaningful information from spatio-temporal data, represented as series of spike trains over …
Spiking neural networks perform similarly to deep networks on occluded images.
problem Robust object recognition in partially occluded images.
method Developed a two-layer spiking neural network trained on natural scenes with a biologically plausible learning rule, compared to deep convolutional networks.
result Spiking neural networks achieve good accuracy and robustness on stepwise pixel erasement tasks.
Spike-and-slab priors are improved for high-dimensional Bayesian regression.
problem Prohibitive computational costs for existing samplers in high-dimensional settings.
method Proposes Scalable Spike-and-Slab (S3) for high-dimensional Bayesian regression. result Improves computational cost to max{n2pt,np} per iteration, demonstrating significant speed-ups and quality gains. New algorithm tackles delayed feedback in Lipschitz bandits with sublinear regret.
problem Delayed feedback in Lipschitz bandits.
method Design of algorithms for bounded and unbounded stochastic delays.
result Sublinear regret guarantees for both bounded and unbounded delays.
Spiking neural networks (SNNs) have garnered a great amount of interest for supervised and unsupervised learning applications. This paper deals with the problem of training multi-layer feedforward SNNs. The non-linear integrate-and-fire dynamics employed by spiking neurons make it difficult to train SNNs to generate de…
Develops path integral for spiked tensor model dynamics.
problem Dynamics of spiked tensor model with random initial conditions.
method Path integral approach applied to partial differential equations.
result Large-N saddle point equations dominated by melonic diagrams. New algorithms ensure fair selection in combinatorial semi-bandit with unrestricted delays.
problem Fair selection in stochastic combinatorial semi-bandit with delayed feedback.
method Introduced merit-based fairness constraints and new bandit algorithms for reward and fairness.
result Achieved sublinear expected reward and fairness regrets with dependence on delay distribution quantiles.
Banker-OMD improves online learning with delayed feedback.
problem Handling delayed feedback in online learning.
method Generalized Online Mirror Descent (OMD) framework.
result Achieves nearly-optimal performance in three bandit scenarios.
Method reconstructs neuron models from spike times efficiently.
problem Reconstructing neuron models from spike times in degenerate populations.
method Combining deep learning with DICs to map spike times to DIC densities and generate degenerate CBM populations.
result Fast and scalable reconstruction of degenerate populations from spike recordings.
New algorithm handles delayed feedback robustly, reducing regret without knowing delay bounds.
problem Bandits with variably delayed feedback, especially excessive delays.
method Implicit exploration scheme, adaptive skipping, drifted regret control.
result Can tolerate arbitrary excessive delays up to order T, reducing regret.
Study of Langevin dynamics for tensor PCA recovery in high dimensions.
problem Recovering hidden signal vectors (spikes) from noisy Gaussian tensor observations.
method Langevin dynamics approach for nonconvex optimization.
result Sample complexity matches the single-spike case but degrades for all spikes.
Over the past decade, deep neural networks (DNNs) have demonstrated remarkable performance in a variety of applications. As we try to solve more advanced problems, increasing demands for computing and power resources has become inevitable. Spiking neural networks (SNNs) have attracted widespread interest as the third-g…