Single-spike neurons can approximate as well as multi-spike neurons.
problem Limitation of single-spike neurons in spiking neural networks.
method Comparison of single-spike and multi-spike neural networks.
result Single-spike and multi-spike neural networks are equivalent in approximation capabilities.
Affine spiking neural networks learn efficiently and generalize well.
problem Learning with spiking neural networks, especially with positive weights.
method Affine encoders and decoders, continuous parameter dependence, gradient-based training.
result Affine spiking neural networks can approximate shallow ReLU networks and generalize well.
This research bridges binary and spiking neural networks for efficient on-chip AI.
problem Reducing compute requirements in machine learning frameworks.
method Training Spiking Neural Networks in extreme quantization regime and utilizing standard training techniques for conversion.
result Training Spiking Neural Networks in extreme quantization regime achieves near full precision accuracies.
Much of studies on neural computation are based on network models of static neurons that produce analog output, despite the fact that information processing in the brain is predominantly carried out by dynamic neurons that produce discrete pulses called spikes. Research in spike-based computation has been impeded by th…
Spiking neural networks perform similarly to deep networks on occluded images.
problem Robust object recognition in partially occluded images.
method Developed a two-layer spiking neural network trained on natural scenes with a biologically plausible learning rule, compared to deep convolutional networks.
result Spiking neural networks achieve good accuracy and robustness on stepwise pixel erasement tasks.
Improved SNNs with quantized activations outperform traditional networks.
problem Maintaining SotA accuracy in SNNs with limited bit precision.
method Interpolating between non-spiking and spiking regimes using signal processing tools.
result First hybrid SNN outperforms traditional RNNs in accuracy with reduced bit precision.
Chemical networks outperform spiking neural networks in classification tasks.
problem Learning tasks with spiking neural networks require hidden layers, which are computationally expensive.
method Used deterministic mass-action kinetics to prove chemical reaction networks without hidden layers can solve tasks previously solved by spiking neural networks.
result A chemical reaction network without hidden layers outperforms a spiking neural network with hidden layers in a handwritten digit classification task.
Third-generation neural networks, or Spiking Neural Networks (SNNs), aim at harnessing the energy efficiency of spike-domain processing by building on computing elements that operate on, and exchange, spikes. In this paper, the problem of training a two-layer SNN is studied for the purpose of classification, under a Ge…
Proposes a spiking neural network for efficient classification.
problem High computational and power costs of ANNs.
method Random Neural Network (RNN) with spiking neurons.
result Matches classification power of ANNs but with lower energy consumption.
This project proposes using reinforcement learning to train spiking neural networks.
problem Training spiking neural networks using traditional methods is challenging due to the discrete nature of spikes.
method The project investigates two approaches: 1) treating each neuron as an RL agent, 2) applying the reparameterization trick.
result The project demonstrates that reinforcement learning can be applied to train spiking neural networks.
SNNs can represent complex functions efficiently.
problem Understanding the representational power of SNNs.
method Viewed as sequence-to-sequence processors, analyzed using spike train functions.
result SNNs have the universal representation property for certain functions.
A vast majority of computation in the brain is performed by spiking neural networks. Despite the ubiquity of such spiking, we currently lack an understanding of how biological spiking neural circuits learn and compute in-vivo, as well as how we can instantiate such capabilities in artificial spiking circuits in-silico.…
SEF-M improves spiking neural network classification accuracy by 14%.
problem Improving spiking neural network classification accuracy.
method Meta-neuron based learning algorithm with time-varying weight model.
result Time-varying weight model improves classification accuracy by 14%.
Spiking neural networks enable efficient approximate Bayesian inference via permanent dropout.
problem Efficient uncertainty quantification in neural network predictions for critical tasks.
method Conversion of classical neural networks to spiking neural networks, applying permanent dropout for inference.
result Predictive distributions from spiking neural networks using permanent dropout are nearly identical to those from classical networks.
A new method improves fitting neural data with spiking network models.
problem Fitting spiking network models to neural activity does not produce realistic data.
method Augment log-likelihood with dissimilarity terms measured by summary statistics and optimized via back-propagation.
result The new method generates more realistic neural activity statistics and improves network connectivity inference.
SNNs enhance high-frequency price spike forecasting in HFT environments.
problem Conventional financial models fail to capture fine temporal structure in high-frequency price spikes.
method Application of Spiking Neural Networks (SNNs) with hyperparameter tuning via Bayesian Optimization (BO).
result SNN models optimized with PSA achieve significantly higher cumulative returns in backtesting.
T2FSNN improves deep SNNs by reducing spikes and latency.
problem Inefficiency in spiking neural networks due to lack of scalable training algorithms.
method Introduces time-to-first-spike coding with kernel-based dynamic threshold and dendrite, combined with gradient-based optimization and early firing methods.
result Reduces inference latency and spike count by 22% and less than 1% compared to burst coding.
Spiking-YOLO improves object detection with low power and fast convergence.
problem Challenging object detection tasks with spiking neural networks.
method Channel-wise normalization and signed neuron with imbalanced threshold.
result Spiking-YOLO achieves comparable results to Tiny YOLO but with significantly less energy consumption.
Recent studies have suggested that the cognitive process of the human brain is realized as probabilistic inference and can be further modeled by probabilistic graphical models like Markov random fields. Nevertheless, it remains unclear how probabilistic inference can be implemented by a network of spiking neurons in th…
We introduce causal pieces to improve spiking neural networks.
problem Improving the expressiveness and trainability of spiking neural networks.
method We decompose the input domain of SNNs into causal regions, proving that the number of these regions is a measure of SNNs' approximation capabilities.
result Parameter initialisations yielding a high number of causal pieces correlate with SNN training success.
SparseProp speeds up SNN simulations and training by four orders of magnitude.
problem Efficiently simulating and training large spiking neural networks.
method Event-based algorithm that reduces computational cost from O(N) to O(log(N)) per spike.
result Numerically exact simulations of large spiking networks and efficient training using backpropagation.
This paper introduces probabilistic SNNs for efficient neural processing.
problem Training algorithms for SNNs lag behind hardware implementations.
method Discrete-time probabilistic models and variational inference.
result Derivation of learning rules for SNNs from first principles.
This paper investigates the effect of leak in spiking neural networks.
problem Comparative analysis of leaky and non-leaky spiking neuron models.
method Experimental and frequency domain analysis of spiking neural networks.
result Leaky spiking neuron models provide improved robustness and generalization but decrease sparsity of computation.
Deep neural networks decode natural visual scenes from neural spikes.
problem Decoding visual scenes from neural spikes for brain-machine interfaces.
method Developed a novel spike-image decoder (SID) using deep neural networks.
result SID reconstructs natural visual scenes from neural spikes with high accuracy.
Edge devices learn directly from personal data using spiking networks.
problem Processing personal data on edge devices with low latency and energy efficiency.
method Spiking Neural Networks for local training on edge devices.
result Spiking networks enable efficient local training on edge devices without scalability limitations.
Sleep-based regularization stabilizes STDP in recurrent neural networks.
problem Pathological weight dynamics in recurrent SNNs.
method Periodic offline phases with stochastic decay and spontaneous activity.
result Sleep-based renormalization prevents weight saturation and preserves learned structure.
Unified framework for training SNNs using EP, faster convergence.
problem Training spiking neural networks with Expectation-Propagation.
method Message-passing framework for learning marginal distributions of SNN parameters.
result Faster convergence compared to gradient-based methods.
Study eigenvalues and eigenvectors in neural networks, focusing on signal propagation.
problem Characterize signal eigenvalues and eigenvectors in neural networks.
method Characterizes signal eigenvalues and eigenvectors for a nonlinear spiked covariance model.
result Provides precise quantitative characterizations of signal eigenvalues and eigenvectors in neural networks.
Neural networks have become the standard model for various computer vision tasks in automated driving including semantic segmentation, moving object detection, depth estimation, visual odometry, etc. The main flavors of neural networks which are used commonly are convolutional (CNN) and recurrent (RNN). In spite of rap…
Spiking neural networks (SNNs) have garnered a great amount of interest for supervised and unsupervised learning applications. This paper deals with the problem of training multi-layer feedforward SNNs. The non-linear integrate-and-fire dynamics employed by spiking neurons make it difficult to train SNNs to generate de…
A high-parallelism SNN improves feature learning efficiency and robustness.
problem Slow learning speed and limited learning capability in existing SNNs.
method Inspired by Inception modules, high-parallelism architecture, Vote-for-All decoding, adaptive repolarization mechanism.
result Superior performance and competitive accuracy compared to state-of-the-art unsupervised SNNs.
Synthesizes images from audio and visual data using spike-based autoencoders.
problem Extracting meaningful information from spatio-temporal data for image synthesis.
method Spike-based autoencoders trained to learn spatio-temporal representations of audio and visual data.
result Synthesized images from audio samples with high fidelity, achieving competitive performance.
This paper presents a constructive algorithm that achieves successful one-shot learning of hidden spike-patterns in a competitive detection task. It has previously been shown (Masquelier et al., 2008) that spike-timing-dependent plasticity (STDP) and lateral inhibition can result in neurons competitively tuned to repea…
New method converts conventional ANNs to SNNs with minimal loss and efficiency.
problem Difficulty in training SNNs directly from conventional ANNs due to discreteness.
method Proposes a novel pipeline combining threshold balance and soft-reset mechanisms for efficient conversion.
result Achieves almost no accuracy loss with only 1/10 of typical SNN simulation time.
Framework for training stochastic spiking neural networks with rough signals.
problem Training stochastic spiking neural networks with noisy spike timing and dynamics.
method Rough path theory and signature kernels for gradient computation.
result Pathwise gradients of SSNNs' trajectories and event times exist and satisfy a recursive relation.
We describe a novel spiking neural network (SNN) for automated, real-time handwritten digit classification and its implementation on a GP-GPU platform. Information processing within the network, from feature extraction to classification is implemented by mimicking the basic aspects of neuronal spike initiation and prop…
Classifiers trained using conventional empirical risk minimization or maximum likelihood methods are known to suffer dramatic performance degradations when tested over examples adversarially selected based on knowledge of the classifier's decision rule. Due to the prominence of Artificial Neural Networks (ANNs) as clas…
Spiking Neural Networks (SNNs) are distributed trainable systems whose computing elements, or neurons, are characterized by internal analog dynamics and by digital and sparse synaptic communications. The sparsity of the synaptic spiking inputs and the corresponding event-driven nature of neural processing can be levera…
New method uses surrogate gradients to train efficient spiking networks on neuromorphic hardware.
problem Training high-performing spiking networks on analog neuromorphic hardware is challenging due to device mismatch and lack of efficient algorithms.
method Introduces a general in-the-loop learning framework based on surrogate gradients.
result Learning self-corrects for device mismatch, resulting in competitive spiking network performance.
Bayesian neural network achieves nearly optimal performance in Besov space.
problem Bayesian neural networks in Besov space.
method Spike-and-slab prior and shrinkage prior for posterior convergence rate.
result The posterior convergence rate is nearly minimax and adaptive to unknown smoothness.
This thesis optimizes neuromorphic systems by slowing down their dynamics, improving performance.
problem Timescale mismatch between analog neuromorphic circuits and real-time sensory inputs.
method Proposes and tests solutions to slow down the dynamics of spiking neural networks.
result Spiking neural networks on analog neuromorphic systems can achieve significant performance boosts.
Low-complexity spiking networks learn complex tasks with minimal trainable parameters.
problem Training complex reinforcement learning tasks with minimal resources.
method Reinforcement learning on simple networks of spiking neurons with random connections.
result Small random spiking networks achieve learning efficiency similar to humans on complex tasks.
Neural networks can learn from higher-order cumulants efficiently, requiring quadratic samples.
problem Learning from higher-order cumulants in high-dimensional data.
method Spiked cumulant model, polynomial time algorithms, neural networks, random features.
result Neural networks require quadratic samples to learn from higher-order cumulants efficiently, while random features require more samples.
A hybrid training method reduces SNN training time and complexity.
problem Training deep SNNs is computationally expensive and time-consuming.
method Hybrid training technique combining initialization from converted SNNs and incremental spike-timing dependent backpropagation (STDB).
result The method converges in less than 20 epochs, reducing training complexity and time.
A new SNN model explains decision-making with learning and spiking neurons.
problem Lack of learning mechanism in existing models for decision-making.
method Proposes a Spiking Neural Network (SNN) model that incorporates a learning mechanism and uses multivariate Hawkes processes.
result Shows a coupling between DDM and Poisson counter models and derives a DDM from a Hawkes network of spiking neurons.
ScieNet improves deep learning resilience to input perturbations.
problem Deep learning's poor resilience to input perturbations in real-world scenarios.
method Hybrid architecture combining SNN for contextual info extraction and DNN for classification.
result Significant improvement in accuracy on noisy and rainy images without prior training.
Deep convolutional neural networks (CNNs) have shown great potential for numerous real-world machine learning applications, but performing inference in large CNNs in real-time remains a challenge. We have previously demonstrated that traditional CNNs can be converted into deep spiking neural networks (SNNs), which exhi…
Configuring deep Spiking Neural Networks (SNNs) is an exciting research avenue for low power spike event based computation. However, the spike generation function is non-differentiable and therefore not directly compatible with the standard error backpropagation algorithm. In this paper, we introduce a new general back…