Paper reviews neuromorphic engineering features and compares analog vs digital systems.
problem Lack of consensus and unclear features in neuromorphic engineering.
method Review of recent work, comparison of machine learning accelerator chips.
result Analog processing and reduced bit precision architectures offer best efficiencies.
A learning rule for first-spike times in neural networks reduces energy consumption and reaction times.
problem Energy efficiency and reaction time in neuromorphic systems.
method Derivation of a learning rule for first-spike times in leaky integrate-and-fire neurons, using only input and output spike times.
result Demonstrated that the approach can implement error backpropagation in hierarchical spiking networks and is capable of harnessing neuromorphic system's speed and energy characteristics.
The paper introduces reservoir computing models for complex systems.
problem Modeling complex engineering systems using nonlinear autoregression.
method Introduces reservoir computing with output feedback as stationary and ergodic infinite-order nonlinear autoregressive models.
result Demonstrates versatility of classical and quantum reservoir computers in modeling synthetic and real data.
Meta-learning helps models learn quickly from few samples.
problem Deep learning requires many samples, which are hard to get.
method Meta-learning optimizes models to adapt quickly to new tasks.
result Meta-learning can improve model efficiency and adaptability.
Gradient model for memristive systems in neurophysiology and neuromorphic circuits.
problem Understanding and modeling memristive systems.
method Introducing a gradient modeling framework based on Chua's definition of memristive elements.
result Gradient properties of memristive systems have implications for neuromorphic circuit analysis and design.
New method uses surrogate gradients to train efficient spiking networks on neuromorphic hardware.
problem Training high-performing spiking networks on analog neuromorphic hardware is challenging due to device mismatch and lack of efficient algorithms.
method Introduces a general in-the-loop learning framework based on surrogate gradients.
result Learning self-corrects for device mismatch, resulting in competitive spiking network performance.
Neuromorphic Computing is a nascent research field in which models and devices are designed to process information by emulating biological neural systems. Thanks to their superior energy efficiency, analog neuromorphic systems are highly promising for embedded, wearable, and implantable systems. However, optimizing neu…
A brain-inspired spiking Transformer reduces energy consumption and enhances interpretability.
problem Energy inefficiency and lack of interpretability in Transformer models.
method Spiking STDP Transformer using spike-timing-dependent plasticity (STDP) for self-attention.
result Achieves 94.35% and 78.08% accuracy on CIFAR-10 and CIFAR-100 datasets respectively, with 88.47% energy reduction.
SNNs optimize cross-market portfolios with neuromorphic computing, reducing computational overhead and improving returns.
problem Complex cross-market portfolio optimization with high-frequency, multi-dimensional datasets.
method Leaky Integrate-and-Fire neuron dynamics, adaptive thresholding, spike-timing-dependent plasticity, lateral inhibition, hierarchical clustering, population-based spike encoding, multiple decoding strategies.
result SNNs deliver superior risk-adjusted returns and reduced volatility compared to ANN benchmarks, with improved computational efficiency.
Improves energy efficiency of neuromorphic hardware by optimizing memory organization and encoding schemes.
problem Energy inefficiency in neuromorphic hardware, especially in digital accelerators.
method Synthesized controller and memory for different encoding schemes, introduced functional encoding for structured connectivity.
result Functional encoding offers a 58% reduction in energy for weight updates in convolutional layers.
Despite being originally inspired by the central nervous system, artificial neural networks have diverged from their biological archetypes as they have been remodeled to fit particular tasks. In this paper, we review several possibilites to reverse map these architectures to biologically more realistic spiking networks…
How spiking networks are able to perform probabilistic inference is an intriguing question, not only for understanding information processing in the brain, but also for transferring these computational principles to neuromorphic silicon circuits. A number of computationally powerful spiking network models have been pro…
ConformalHDC improves HDC's uncertainty quantification for neuromorphic learning.
problem Lack of rigorous uncertainty quantification in Hyperdimensional Computing.
method Combines conformal prediction with HDC's efficiency, proposing set-valued and point-valued formulations.
result Demonstrates improved robustness and accuracy in decoding neural stimulus information.
Neuromorphic column performs online unsupervised clustering.
problem Real-time clustering of streaming data.
method Localized, spike timing-dependent plasticity (STDP) neural column.
result Prototype column performs similarly to k-means clustering.
Low precision networks in the reinforcement learning (RL) setting are relatively unexplored because of the limitations of binary activations for function approximation. Here, in the discrete action ATARI domain, we demonstrate, for the first time, that low precision policy distillation from a high precision network pro…
High-speed model accurately simulates neuromorphic devices.
problem Accurately modeling stochastic synapses in large-scale neuromorphic systems.
method Generative vector autoregressive model based on resistive memory cell data.
result Fast, high-throughput model reproduces synaptic parameters and correlations.
VOWEL trains WTA-SNNs for multi-valued events, overcoming resource limitations.
problem Training WTA-SNNs for multi-valued events is challenging due to non-differentiability and recurrent behavior.
method Develops a variational online local training rule (VOWEL) for WTA-SNNs using local pre- and post-synaptic information and a common reward signal.
result VOWEL outperforms conventional binary SNNs in real-world neuromorphic datasets with multi-valued events.
New framework boosts neural network performance and resilience.
problem Susceptibility of compact neural network implementations to system disturbances.
method Realistic crossbar simulations and Mosaics framework to re-use synaptic connections.
result Compact neural networks are noise-immune and perform well under disturbances.
Python scripts analyze MRAM-based neuromorphic devices' process variation impacts on machine learning accuracy.
problem Impact of process variation on MRAM-based neuromorphic devices' performance in machine learning applications.
method Developed transportable Python scripts to analyze output variation under changes in device dimensions.
result Revealed impacts and limits for processing variation of device fabrication on energy vs. accuracy tradeoffs.
Sleep-based regularization stabilizes STDP in recurrent neural networks.
problem Pathological weight dynamics in recurrent SNNs.
method Periodic offline phases with stochastic decay and spontaneous activity.
result Sleep-based renormalization prevents weight saturation and preserves learned structure.
Using Intel's Loihi neuromorphic research chip and ABR's Nengo Deep Learning toolkit, we analyze the inference speed, dynamic power consumption, and energy cost per inference of a two-layer neural network keyword spotter trained to recognize a single phrase. We perform comparative analyses of this keyword spotter runni…
Enhances SNNs for spatio-temporal feature extraction.
problem Insufficient temporal dependencies in existing SNN synaptic structures.
method Integrates temporal convolution and attention mechanisms into synaptic connections.
result Improves SNN performance on classification tasks.
Neuromorphic hardware platforms, such as Intel's Loihi chip, support the implementation of Spiking Neural Networks (SNNs) as an energy-efficient alternative to Artificial Neural Networks (ANNs). SNNs are networks of neurons with internal analogue dynamics that communicate by means of binary time series. In this work, a…
New algorithm learns uncertainty for edge devices.
problem Uncertainty handling for edge devices in critical applications.
method e-prop 1 algorithm with Broadcast Alignment and local information.
result Algorithm can learn uncertainty locally, improving decision-making.
Bio-inspired neuromorphic hardware is a research direction to approach brain's computational power and energy efficiency. Spiking neural networks (SNN) encode information as sparsely distributed spike trains and employ spike-timing-dependent plasticity (STDP) mechanism for learning. Existing hardware implementations of…
We study two aspects of noisy computations during inference. The first aspect is how to mitigate their side effects for naturally trained deep learning systems. One of the motivations for looking into this problem is to reduce the high power cost of conventional computing of neural networks through the use of analog ne…
An online algorithm improves cPCA for efficient, interpretable data analysis.
problem Efficiently finding informative low-dimensional representations in large datasets.
method Developed an online algorithm for a modified cPCA* method.
result The online algorithm for cPCA* shows improved performance and interpretability.
Fault-tolerant neural networks inspired by biological error correction codes.
problem Achieving reliable computation with unreliable neurons.
method Using biological error correction codes from grid cells in the mammalian cortex to develop a fault-tolerant neural network.
result Noisy biological neurons operate below a fault-tolerance threshold, suggesting a mechanism for reliable computation in the brain.
PredPCA extracts key components for better time series prediction.
problem Improving time series prediction with reduced generalization error.
method Unsupervised learning scheme using convex optimization.
result PredPCA minimizes test prediction error and identifies hidden states.
Efficient SNN on Loihi achieves high gesture recognition accuracy.
problem Real-time gesture recognition using event-based sensors.
method Design and training of SNN, conversion from DNN, pre-processing of DVS data.
result 89.64% classification accuracy with 37 Loihi cores.
A new scheme reduces global search cost by a square root factor.
problem Challenges in finding global minimum of cost functions.
method Gradient descent combined with a biased crossover of two good solutions.
result Quadratic speedup of global search efficiency.
Information in neural networks is represented as weighted connections, or synapses, between neurons. This poses a problem as the primary computational bottleneck for neural networks is the vector-matrix multiply when inputs are multiplied by the neural network weights. Conventional processing architectures are not well…
The ability to learn and adapt in real time is a central feature of biological systems. Neuromorphic architectures demonstrating such versatility can greatly enhance our ability to efficiently process information at the edge. A key challenge, however, is to understand which learning rules are best suited for specific t…
SNNs can represent complex functions efficiently.
problem Understanding the representational power of SNNs.
method Viewed as sequence-to-sequence processors, analyzed using spike train functions.
result SNNs have the universal representation property for certain functions.
The paper relaxes constraints on predictive coding models, making them more biologically plausible.
problem Neurophysiological models of predictive coding are not fully biologically plausible.
method The paper relaxes constraints on standard predictive coding algorithms by removing neurally implausible features.
result The removal of neurally implausible features does not significantly affect learning performance.
Probabilistic topic models are popular unsupervised learning methods, including probabilistic latent semantic indexing (pLSI) and latent Dirichlet allocation (LDA). By now, their training is implemented on general purpose computers (GPCs), which are flexible in programming but energy-consuming. Towards low-energy imple…
A new hierarchy quantifies agency in systems based on information processing.
problem Lack of a measurable, universal definition for agency in intelligent systems.
method Developed a bottom-up framework based on information processing hierarchy.
result Identified three orders of information processing (I, II, III) as necessary for agency.
This work tackles catastrophic forgetting in neural networks by mimicking brain's metaplasticity.
problem Catastrophic forgetting in neural networks, where new tasks erase previously learned ones.
method Interpreting binarized neural networks as metaplastic systems, adjusting their training technique.
result Training technique reduces catastrophic forgetting without needing previously presented data.
We explore the robustness of recurrent neural networks when the computations within the network are noisy. One of the motivations for looking into this problem is to reduce the high power cost of conventional computing of neural network operations through the use of analog neuromorphic circuits. Traditional GPU/CPU-cen…
Spiking neural networks (SNNs) offer a promising alternative to current artificial neural networks to enable low-power event-driven neuromorphic hardware. Spike-based neuromorphic applications require processing and extracting meaningful information from spatio-temporal data, represented as series of spike trains over …
EvAn detects anomalies in event-based camera data with reduced complexity.
problem Anomaly detection in event-based camera data.
method Dual discriminator cGAN on event data with learned representation.
result Reduction in computational complexity compared to state-of-the-art methods.
New method converts ANN gates to SNNs with AMOS neurons for improved image classification.
problem Efficiently converting ANN gates to SNNs for neuromorphic hardware.
method Introducing AMOS conversion for gates in ANNs, improving accuracy and throughput.
result Improved accuracy of SNNs for ImageNet from 74.60% to 80.97%.
In this paper, a spintronic neuromorphic reconfigurable Array (SNRA) is developed to fuse together power-efficient probabilistic and in-field programmable deterministic computing during both training and evaluation phases of restricted Boltzmann machines (RBMs). First, probabilistic spin logic devices are used to devel…
This paper introduces a new learning rule for probabilistic SNNs that improves log-likelihood, accuracy, and calibration.
problem Training and inference of deterministic SNNs are constrained by their inability to generate multiple independent outputs.
method Introduces a generalized expectation-maximization (GEM) learning rule for probabilistic SNNs.
result The GEM-SNN learning rule leads to significant improvements in log-likelihood, accuracy, and calibration.
A new model uses 'ghost units' to enable efficient backpropagation in deep neural networks.
problem How to achieve efficient backpropagation in deep neural networks with biological plausibility.
method Introduces 'ghost units' to cancel feedback, enabling efficient error backpropagation.
result Demonstrates that the model can approximate error gradients and achieve good performance on classification tasks.
CoNNTrA trains DNNs with low-power, low-memory constraints.
problem Training deep neural networks on edge computing systems with low power and memory usage.
method Coordinate gradient descent-based approach for training DNNs with constrained learning parameters.
result CoNNTrA models use 32x less memory and have comparable errors to Backpropagation models.
This review discusses challenges and solutions for AI in chemical engineering.
problem Challenges in applying classical machine learning to chemical engineering data.
method Identifying four data characteristics and discussing their applications and solutions.
result Current research extends data science and machine learning to handle chemical engineering data challenges.
Optimal engine operation during a transient driving cycle is the key to achieving greater fuel economy, engine efficiency, and reduced emissions. In order to achieve continuously optimal engine operation, engine calibration methods use a combination of static correlations obtained from dynamometer tests for steady-stat…