A neuromorphic unit models complex synapses efficiently.
problem Efficiently simulating complex synaptic response functions in neural networks.
method Digital neuromorphic architecture, Spiking Temporal Processing Unit (STPU), modeling arbitrary complex synaptic response functions.
result Demonstrates flexibility and efficiency of STPU for instantiating neural algorithms.
The apparent stochasticity of in-vivo neural circuits has long been hypothesized to represent a signature of ongoing stochastic inference in the brain. More recently, a theoretical framework for neural sampling has been proposed, which explains how sample-based inference can be performed by networks of spiking neurons.…
Balanced excitation and inhibition enhance neuronal selectivity and robustness.
problem Ensuring robust neuronal responses in noisy environments.
method Investigated the conditions for balanced excitation and inhibition to enhance robustness of single neurons and network attractor states.
result Balanced excitation and inhibition are crucial for high-capacity, noise-resistant neuronal selectivity.
Synaptic pruning reduces CNNs by 96% on CIFAR-10.
problem Memory and computation constraints in CNNs for mobile devices.
method Synaptic Pruning: data-driven method to prune connections based on Synaptic Strength.
result Significant size reduction and computation saving with up to 96% pruning on CIFAR-10.
Neural network memorizes external stimuli through synaptic strength changes.
problem Memory and classification in neural networks.
method One-to-one mapping between stimulus and synaptic strength under synaptic plasticity constraints.
result Neural network can memorize external stimuli through synaptic changes.
QSD enhances deep network performance through biologically plausible dropout.
problem Overfitting in deep networks.
method Quantal Synaptic Dilution (QSD) model based on neuronal synapses.
result QSD outperforms standard dropout in various deep network architectures.
Brain uses synaptic failure to sample from posterior distributions.
problem Bayesian inference in the brain's probabilistic computations.
method Adapting synaptic failure to sample posterior predictive distributions.
result Synaptic failure enables sampling of complete posterior predictive distributions.
High-speed model accurately simulates neuromorphic devices.
problem Accurately modeling stochastic synapses in large-scale neuromorphic systems.
method Generative vector autoregressive model based on resistive memory cell data.
result Fast, high-throughput model reproduces synaptic parameters and correlations.
Study shows CNNs can perform well with less data using biological synaptic distributions.
problem Training deep neural networks with limited data.
method Synthesizing CNNs using log-normal or correlated center-surround synaptic strength distributions.
result CNNs with biological synaptic strength distributions can perform well with fewer data samples.
The study reveals how synaptic correlations promote dimension reduction in neural networks.
problem Understanding how synaptic correlations affect neural correlations and dimension reduction in deep neural networks.
method A simplified model of dimension reduction considering pairwise correlations among synapses, using mathematical self-consistency for both binary and continuous synapses.
result Weakly-correlated synapses encourage dimension reduction compared to orthogonal synapses, and they also slow down the decorrelation process.
Synaptic cluster-driven evolution improves deep neural networks by reducing synapses and clusters.
problem Efficiently synthesizing deep neural networks with fewer synapses and clusters.
method Synaptic cluster-driven genetic encoding scheme.
result Significantly smaller number of synapses and clusters in offspring networks.
Locally learned synaptic failure enables complete Bayesian inference.
problem Bayesian inference in neural networks.
method Biologically constrained neural network with synaptic failure and lateral inhibition.
result Synaptic failure allows sampling from both epistemic and aleatoric distributions.
New synaptic model derived from reinforcement learning for spiking neurons.
problem Learning optimal actions in complex systems.
method Derives a synaptic update rule from reinforcement learning algorithms.
result Synaptic strengths lead to locally optimal reward values.
Learning and memory in the brain are implemented by complex, time-varying changes in neural circuitry. The computational rules according to which synaptic weights change over time are the subject of much research, and are not precisely understood. Until recently, limitations in experimental methods have made it challen…
CNNs accurately model retinal responses to natural scenes.
problem Understanding neural computations in retinal responses to natural stimuli.
method Deep convolutional neural networks (CNNs) were used to model retinal responses to natural scenes.
result CNNs are more accurate than linear models in predicting retinal responses to natural scenes.
Natural gradient learning improves synaptic plasticity in spiking neurons.
problem Parametrization dependence leads to inconsistencies in classical synaptic plasticity theories.
method Proposes natural gradient descent in Riemannian geometry for spiking neurons.
result Derives a synaptic learning rule that explains biological phenomena.
There has been significant recent interest towards achieving highly efficient deep neural network architectures. A promising paradigm for achieving this is the concept of evolutionary deep intelligence, which attempts to mimic biological evolution processes to synthesize highly-efficient deep neural networks over succe…
Enhances SNNs for spatio-temporal feature extraction.
problem Insufficient temporal dependencies in existing SNN synaptic structures.
method Integrates temporal convolution and attention mechanisms into synaptic connections.
result Improves SNN performance on classification tasks.
Learning in neural networks poses peculiar challenges when using discretized rather then continuous synaptic states. The choice of discrete synapses is motivated by biological reasoning and experiments, and possibly by hardware implementation considerations as well. In this paper we extend a previous large deviations a…
This paper introduces a hierarchical associative memory model with multiple layers.
problem Limitations of traditional associative memory models with only one hidden layer.
method Develops a fully recurrent model with arbitrary layers, including locally connected ones, and a corresponding energy function.
result The model can dynamically assemble memories using weights from lower layers and higher layers' rules.
New model reveals balance crucial for robust neural coding.
problem Efficient neural coding in noisy, chaotic networks.
method Analytical model of balanced predictive coding with dissociated balance and weight disorder.
result Superclassical scaling in coding accuracy, independent of balance and weight disorder.
Proposes a method to prevent neural networks from forgetting learned tasks.
problem Catastrophic forgetting in neural networks.
method Attention-based selective plasticity of synapses inspired by the cholinergic neuromodulatory system.
result Competitive performance on benchmark tasks compared to state-of-the-art methods.
Bayesian inference for DNN synaptic connectivity improves regularization without manual tuning.
problem Improving deep neural network performance through effective regularization.
method Bayesian inference with Beta priors over binary synaptic variables, using Black-Box Variational inference.
result Effective, heuristics-free network regularization scheme.
ANPyC combats forgetting by pruning and consolidating neural parameters.
problem Catastrophic forgetting in neural networks, especially with long-term tasks.
method Adversarial Neural Pruning and Synaptic Consolidation (ANPyC) to balance task-relevant and irrelevant parameters.
result ANPyC prevents forgetting while enabling efficient learning of multiple tasks.
A new neural model evolves to learn at the synaptic level.
problem Lack of biologically realistic neural models in deep learning.
method Evolve individual neuron and synaptic models using ENUs.
result Evolved neural network learns complex tasks like a T-maze.
A neuron is a basic physiological and computational unit of the brain. While much is known about the physiological properties of a neuron, its computational role is poorly understood. Here we propose to view a neuron as a signal processing device that represents the incoming streaming data matrix as a sparse vector of …
New algorithm uses random matrices for neural network training without synaptic weight symmetries.
problem Training neural networks efficiently and without synaptic weight symmetries.
method Contrastive Hebbian learning with random feedback weights.
result Random contrastive Hebbian learning achieves better computational models for learning.
Dynamics and function of neuronal networks are determined by their synaptic connectivity. Current experimental methods to analyze synaptic network structure on the cellular level, however, cover only small fractions of functional neuronal circuits, typically without a simultaneous record of neuronal spiking activity. H…
This paper suggests a learning-theoretic perspective on how synaptic plasticity benefits global brain functioning. We introduce a model, the selectron, that (i) arises as the fast time constant limit of leaky integrate-and-fire neurons equipped with spiking timing dependent plasticity (STDP) and (ii) is amenable to the…
Biological neural network mimics CCA for multi-channel data.
problem Implementing CCA in a biologically plausible neural network.
method Derive an online CCA algorithm with local synaptic updates for multi-compartmental neurons.
result The derived neural network architecture and synaptic updates resemble cortical pyramidal neuron behavior.
Proposes a new method to improve deep neural network performance.
problem Improving generalization in deep neural networks on unseen data.
method Post-synaptic potential regularization (PSP).
result PSP achieves comparable performance to sophisticated learning strategies and improves generalization over ℓ2 regularization. Improved KAN model explains brain dynamics through edge learning and synaptic strength.
problem Explaining brain dynamics and frequencies in different brain regions.
method ELKAN (Edge Learning KNN) model with edge learning and trimming, inspired by brain science.
result ELKAN model outperforms KAN in explaining brain frequencies and dynamics.
A model explains credit assignment in deep learning networks.
problem Understanding how deep learning assigns credit to parameters.
method Mean-field learning model with ensembles of sub-networks.
result Synaptic connections can be categorized into three types.
Elite ONNs learn better with synaptic plasticity, improving performance over CNNs.
problem Limited heterogeneity in ONNs due to fixed operator sets.
method Synaptic plasticity-based search for optimal operator sets.
result Elite ONNs achieve superior learning performance compared to conventional methods.
Bayesian method reconstructs neural network memories from connectivity.
problem Reconstructing memories from neural network connectivity.
method Bayesian inference using statistical physics principles.
result Algorithm successfully reconstructs stored patterns from synaptic connectivity.
New framework models neural systems with random architecture on manifolds.
problem Complex, uncertain systems with non-Gaussian outputs.
method Latent random field on compact manifold generates neural architecture and weights.
result Synthetic neural systems can produce stochastic outputs for deterministic inputs.
New algorithm finds important synapses without training data.
problem Finding important synapses in neural networks without data.
method Iterative Synaptic Flow Pruning (SynFlow) based on conservation law.
result Algorithm consistently outperforms existing pruning methods.
Analyzes unsupervised neural networks using statistical mechanics and Monte Carlo simulations.
problem Understanding computational capabilities of unsupervised neural networks.
method Statistical mechanics approach and Monte Carlo simulations.
result Obtained a phase diagram summarizing network performance.
Olshausen and Field (OF) proposed that neural computations in the primary visual cortex (V1) can be partially modeled by sparse dictionary learning. By minimizing the regularized representation error they derived an online algorithm, which learns Gabor-filter receptive fields from a natural image ensemble in agreement …
Deep CNN model predicts neuronal cell health from images.
problem Predicting the biological activity of chemical compounds on neuronal cells.
method Deep convolutional neural network (CNN) with residual connections.
result Achieved 99.6% accuracy in distinguishing treated from untreated cells.
Paper models Pavlov's classical conditioning using stochastic processes and Langevin equations.
problem Lack of modeling for Pavlov's classical conditioning.
method Modeling neural and synaptic dynamics via Langevin equations.
result Pavlov's mechanism spontaneously leads to synaptic weights similar to Hebb's.
Identifies learning rules from neural network observables.
problem Determine the underlying plasticity rules governing learning in biological systems.
method Simulated idealized neuroscience experiments with artificial neural networks to generate a dataset of learning trajectories. Used linear and non-linear classifiers to identify learning rules from aggregate statistics of weights, activations, and activity changes.
result Different classes of learning rules can be separated solely on the basis of aggregate statistics of the weights, activations, or instantaneous layer-wise activity changes.
New method improves deep learning performance without weight symmetry.
problem Challenges in scaling non-symmetric learning methods to deep convolutional networks.
method Introduced techniques to mitigate scalability issues, including a modified feedback alignment method.
result Demonstrated competitive performance with backpropagation using a weaker form of weight symmetry.
New framework boosts neural network performance and resilience.
problem Susceptibility of compact neural network implementations to system disturbances.
method Realistic crossbar simulations and Mosaics framework to re-use synaptic connections.
result Compact neural networks are noise-immune and perform well under disturbances.
Intelligent synapses help neural networks learn continuously.
problem Continual learning in changing data environments.
method Intelligent synapses that accumulate and reuse task-relevant information.
result Significant reduction in forgetting with improved computational efficiency.
Unified theory for training neural networks with binary synapses.
problem Discrete nature of synapses and complex interactions in neural networks.
method Variational mean-field theory decomposing learning into maximization and expectation steps.
result Unified framework for unsupervised learning in neural networks.
Neural networks with learned biases can approximate any function.
problem Whether neural networks with only learned biases can approximate any continuous function.
method Theoretical and numerical analysis of random weights and learned biases in neural networks.
result Feedforward and recurrent neural networks with random weights can approximate any continuous function and dynamical systems.
Stochastic binary synapses in neural networks lead to robust solutions.
problem Training low-precision neural networks with stochastic weights.
method Gradient descent on a probability distribution of binary synapses.
result Binary solutions are robust and generalize well, while typical solutions are isolated.