Updated VESICLE-CNN for faster synapse detection.
problem Slow patch-based synapse detection.
method Fully convolutional approach using dilated convolutions.
result 600x speedup at test time with no loss in accuracy.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,181 papers · 148 categories
Trend · papers per month
48 results for “synapse detection”
Proposes an alternative method to train RBMs with binary synapses using Bayesian learning rule.
problem Training RBMs with binary synapses is challenging due to discrete nature of synapses.
method Proposes an alternative optimization method using the Bayesian learning rule, updating natural parameters instead of expectation parameters.
result No additional clipping is needed as natural parameters take values in the entire real domain.
The study reveals how synaptic correlations promote dimension reduction in neural networks.
problem Understanding how synaptic correlations affect neural correlations and dimension reduction in deep neural networks.
method A simplified model of dimension reduction considering pairwise correlations among synapses, using mathematical self-consistency for both binary and continuous synapses.
result Weakly-correlated synapses encourage dimension reduction compared to orthogonal synapses, and they also slow down the decorrelation process.
Variational mean-field theory for training restricted Boltzmann machines with binary synapsesstat.ML
Unified theory for training neural networks with binary synapses.
problem Discrete nature of synapses and complex interactions in neural networks.
method Variational mean-field theory decomposing learning into maximization and expectation steps.
result Unified framework for unsupervised learning in neural networks.
Synapse arbitrates TSFMs to improve time series forecasting performance.
problem TSFMs vary in performance across different forecasting tasks, domains, and horizons.
method Synapse dynamically assigns and adjusts predictive weights based on TSFM performance.
result Synapse consistently outperforms other ensembling techniques and individual TSFMs.
Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapsescond-mat.dis-nn
We show that discrete synaptic weights can be efficiently used for learning in large scale neural systems, and lead to unanticipated computational performance. We focus on the representative case of learning random patterns with binary synapses in single layer networks. The standard statistical analysis shows that this…
New training algorithm enhances SNNs for temporal signal processing.
problem Lack of robust training algorithms for large-scale SNNs.
method Formulated SNN as IIR filters, proposed training algorithm for optimal synapse filter kernels and weights.
result Model and training algorithm outperform state-of-the-art approaches in accuracy.
Learning may need only a few bits of synaptic precisioncond-mat.dis-nn
Learning in neural networks poses peculiar challenges when using discretized rather then continuous synaptic states. The choice of discrete synapses is motivated by biological reasoning and experiments, and possibly by hardware implementation considerations as well. In this paper we extend a previous large deviations a…
While deep learning has led to remarkable advances across diverse applications, it struggles in domains where the data distribution changes over the course of learning. In stark contrast, biological neural networks continually adapt to changing domains, possibly by leveraging complex molecular machinery to solve many t…
Paper proposes Memory Aware Synapses for selective knowledge preservation in lifelong learning.
problem Selective preservation of knowledge in artificial learning systems.
method Unsupervised, online computation of parameter importance based on sensitivity to changes.
result State-of-the-art performance and ability to adapt parameter importance based on unlabeled data.
A neuromemristive HTM architecture boosts robustness and performance.
problem Improving robustness and performance of HTM algorithms.
method Developed a neuromemristive crossbar architecture for HTM, incorporating memristors and neurogenesis.
result Enhanced robustness and performance of HTM through neuromemristive architecture.
Evolution in Groups: A deeper look at synaptic cluster driven evolution of deep neural networkscs.NE
A promising paradigm for achieving highly efficient deep neural networks is the idea of evolutionary deep intelligence, which mimics biological evolution processes to progressively synthesize more efficient networks. A crucial design factor in evolutionary deep intelligence is the genetic encoding scheme used to simula…
Deep learning has become a powerful and popular tool for a variety of machine learning tasks. However, it is challenging to understand the mechanism of deep learning from a theoretical perspective. In this work, we propose a random active path model to study collective properties of deep neural networks with binary syn…
Stochastic binary synapses in neural networks lead to robust solutions.
problem Training low-precision neural networks with stochastic weights.
method Gradient descent on a probability distribution of binary synapses.
result Binary solutions are robust and generalize well, while typical solutions are isolated.
Unsupervised learning for evolving data streams with STAM architecture.
problem Learning from non-stationary, unlabeled data streams over time.
method Self-Taught Associative Memory (STAM) architecture with online clustering, novelty detection, and feature storage.
result STAM architecture improves clustering and classification tasks compared to existing continual learning models.
Algorithm detects hidden spike-patterns in neural networks with one-shot learning.
problem Detecting hidden spike-patterns in high activity neural networks.
method Constructive algorithm using spike-timing-dependent plasticity (STDP) and lateral inhibition.
result Successful one-shot detection of new spike-patterns after long intervals.
A central problem in neuroscience is reconstructing neuronal circuits on the synapse level. Due to a wide range of scales in brain architecture such reconstruction requires imaging that is both high-resolution and high-throughput. Existing electron microscopy (EM) techniques possess required resolution in the lateral p…
QSD enhances deep network performance through biologically plausible dropout.
problem Overfitting in deep networks.
method Quantal Synaptic Dilution (QSD) model based on neuronal synapses.
result QSD outperforms standard dropout in various deep network architectures.
Stochastic DNNs with memristive synapses perform well despite limited device variability.
problem Performance of DNNs with memristive synapses under limited dynamic range and variability.
method Stochastic training of DNNs with memristive synapses, focusing on variability optimization.
result Stochastic memristive DNNs suffer less than 3% loss in accuracy compared to floating point software baseline.
Information in neural networks is represented as weighted connections, or synapses, between neurons. This poses a problem as the primary computational bottleneck for neural networks is the vector-matrix multiply when inputs are multiplied by the neural network weights. Conventional processing architectures are not well…
High-speed model accurately simulates neuromorphic devices.
problem Accurately modeling stochastic synapses in large-scale neuromorphic systems.
method Generative vector autoregressive model based on resistive memory cell data.
result Fast, high-throughput model reproduces synaptic parameters and correlations.
New method for continual learning without task boundaries.
problem Traditional continual learning is task-based and impractical for real-world applications.
method Developed an online continual learning system using Memory Aware Synapses.
result Valid approach demonstrated in self-supervised learning and robot collision avoidance.
A new neural model evolves to learn at the synaptic level.
problem Lack of biologically realistic neural models in deep learning.
method Evolve individual neuron and synaptic models using ENUs.
result Evolved neural network learns complex tasks like a T-maze.
We study the computational capacity of a model neuron, the Tempotron, which classifies sequences of spikes by linear-threshold operations. We use statistical mechanics and extreme value theory to derive the capacity of the system in random classification tasks. In contrast to its static analog, the Perceptron, the Temp…
New algorithm finds important synapses without training data.
problem Finding important synapses in neural networks without data.
method Iterative Synaptic Flow Pruning (SynFlow) based on conservation law.
result Algorithm consistently outperforms existing pruning methods.
Neural network memorizes external stimuli through synaptic strength changes.
problem Memory and classification in neural networks.
method One-to-one mapping between stimulus and synaptic strength under synaptic plasticity constraints.
result Neural network can memorize external stimuli through synaptic changes.
New loss function connects learning rate and momentum.
problem Finding optimal learning rate and momentum empirically.
method Proposes a new information-theoretical loss function.
result Loss, learning rate, and momentum are closely connected.
EP learns like BPTT but with local weight updates.
problem Existing EP lacks a local time learning rule.
method C-EP updates weights simultaneously with neuron dynamics.
result C-EP follows BPTT gradients and performs well.
We introduce SIM-CE, an advanced, user-friendly modeling and simulation environment in Simulink for performing multi-scale behavioral analysis of the nervous system of Caenorhabditis elegans (C. elegans). SIM-CE contains an implementation of the mathematical models of C. elegans's neurons and synapses, in Simulink, whi…
Improved KAN model explains brain dynamics through edge learning and synaptic strength.
problem Explaining brain dynamics and frequencies in different brain regions.
method ELKAN (Edge Learning KNN) model with edge learning and trimming, inspired by brain science.
result ELKAN model outperforms KAN in explaining brain frequencies and dynamics.
Until recently, research on artificial neural networks was largely restricted to systems with only two types of variable: Neural activities that represent the current or recent input and weights that learn to capture regularities among inputs, outputs and payoffs. There is no good reason for this restriction. Synapses …
Neural connectomics has begun producing massive amounts of data, necessitating new analysis methods to discover the biological and computational structure. It has long been assumed that discovering neuron types and their relation to microcircuitry is crucial to understanding neural function. Here we developed a nonpara…
Paper proves multiplicative weight updates can train neural networks without learning rate tuning.
problem Vanishing and exploding gradients in gradient descent for compositional functions.
method Proves descent lemma for compositional functions using multiplicative weight updates and derives Madam optimizer.
result Madam optimizer trains state-of-the-art neural networks without learning rate tuning.
Equivalence between algorithmic instability and transition to replica symmetry breaking in perceptron learning systemscond-mat.dis-nn
Binary perceptron's instability linked to replica symmetry breaking.
problem Understanding the relationship between algorithmic instability and replica symmetry breaking in binary perceptron learning.
method Established the connection between algorithmic instability and replica symmetry breaking by comparing the instability condition around the fixed point to the instability for breaking the replica symmetric solution of the free energy function.
result The instability condition around the algorithmic fixed point is identical to the instability for breaking the replica symmetric saddle point solution of the free energy function.
Proposes a method to prevent neural networks from forgetting learned tasks.
problem Catastrophic forgetting in neural networks.
method Attention-based selective plasticity of synapses inspired by the cholinergic neuromodulatory system.
result Competitive performance on benchmark tasks compared to state-of-the-art methods.
The highly variable dynamics of neocortical circuits observed in vivo have been hypothesized to represent a signature of ongoing stochastic inference but stand in apparent contrast to the deterministic response of neurons measured in vitro. Based on a propagation of the membrane autocorrelation across spike bursts, we …
We apply neuromodulation to deep neural networks to improve learning.
problem Static learning parameters in neural networks limit adaptability.
method Evolved neuromodulatory dynamics modify learning parameters over training.
result Evolution found dynamic, location-specific learning strategies.
Natural gradient learning improves synaptic plasticity in spiking neurons.
problem Parametrization dependence leads to inconsistencies in classical synaptic plasticity theories.
method Proposes natural gradient descent in Riemannian geometry for spiking neurons.
result Derives a synaptic learning rule that explains biological phenomena.
Theory reconstructs network connectivity from event timings.
problem Reconstructing network connectivity from incomplete continuous-time data.
method Linearizes event space mapping to reveal direct influences.
result Reveals synapse presence and inhibitory/activating nature.
There has been significant recent interest towards achieving highly efficient deep neural network architectures. A promising paradigm for achieving this is the concept of evolutionary deep intelligence, which attempts to mimic biological evolution processes to synthesize highly-efficient deep neural networks over succe…
New dropout technique reduces training time by 20-77%.
problem Efficient training of deep neural networks consumes excessive time and energy.
method Approximate Random Dropout replaces random neuron/synapse dropout with regular patterns to reduce computation and data access.
result Reduces training time by 20-77% with minimal accuracy loss.
Log-Normal Multiplicative Dynamics improves low-precision training of neural networks.
problem Training large neural networks with low precision is unstable.
method Derive a Bayesian learning rule with log-normal posterior distributions and multiplicative updates.
result LMD achieves stable and accurate training for Vision Transformer and GPT-2.
Unsupervised learning by hidden units with biological plausibility.
problem Training neural networks without labeled data.
method Global inhibition in hidden layer to learn feature detectors.
result Learned feature detectors enable supervised training of higher layers.
Balanced Excitation and Inhibition are Required for High-Capacity, Noise-Robust Neuronal Selectivityq-bio.NC
Neurons and networks in the cerebral cortex must operate reliably despite multiple sources of noise. To evaluate the impact of both input and output noise, we determine the robustness of single-neuron stimulus selective responses, as well as the robustness of attractor states of networks of neurons performing memory ta…
Pavlov Learning Machinescond-mat.dis-nn
Paper models Pavlov's classical conditioning using stochastic processes and Langevin equations.
problem Lack of modeling for Pavlov's classical conditioning.
method Modeling neural and synaptic dynamics via Langevin equations.
result Pavlov's mechanism spontaneously leads to synaptic weights similar to Hebb's.
Taking inspiration from biological evolution, we explore the idea of "Can deep neural networks evolve naturally over successive generations into highly efficient deep neural networks?" by introducing the notion of synthesizing new highly efficient, yet powerful deep neural networks over successive generations via an ev…
The study learns neural update rules by remembering past experiences.
problem Developing efficient online learning rules for neural networks.
method Representing neurons with vectors, using meta-neural networks for updates, and training for remembering past experiences.
result The approach reveals insights into learning rules and could be used for complex tasks like episodic memory.
New synaptic model derived from reinforcement learning for spiking neurons.
problem Learning optimal actions in complex systems.
method Derives a synaptic update rule from reinforcement learning algorithms.
result Synaptic strengths lead to locally optimal reward values.