Mesoscopic model infers neural population dynamics from spike trains.
problem Challenges in fitting mechanistic spiking networks to empirical population data.
method Fit mesoscopic model to aggregate population activity, using likelihood of single-neuron and connectivity parameters.
result Extracts posterior correlations between model parameters and defines subsets of parameters able to reproduce data.
Single ReLU neuron's gradient dynamics reveal support vectors as key to generalization.
problem Understanding the generalization capability of ReLU networks.
method Examined gradient flow dynamics and support vectors in single ReLU neuron training.
result Support vectors play a crucial role in the generalization of ReLU networks.
Researchers develop methods to learn neuron dynamics from colored noise.
problem Learning nonlocal stochastic neuron dynamics from colored noise.
method Proposed two methods for closing Fokker-Planck equations: nonlocal large-eddy-diffusivity closure and data-driven sparse regression.
result Mutual information and total correlation between stimulus and neuron states calculated for FHN neuron.
AKOrN uses synchronized neurons to improve AI tasks.
problem Improving AI performance through better neural representations.
method AKOrN introduces synchronized neurons to replace threshold units.
result AKOrN improves performance across various AI tasks.
Capsules learn from expert neurons using dynamic routing.
problem Training complex multidimensional neural networks.
method Formulated capsule networks as a product of expert neurons with dynamic routing.
result Capsule networks can generate realistic images.
Study on neuron dynamics for XOR classification with zero-margin.
problem Understanding neural network training dynamics in zero-margin classification problems.
method Analysis of Gaussian XOR problem, focusing on neuron block dynamics and generalization without margin assumptions.
result Neurons cluster into four directions and block-level signals evolve coherently, essential for reliable prediction in the Gaussian setting.
A neural network solves dictionary learning problems efficiently.
problem Solving dictionary learning problems efficiently.
method Combining top-down feedback and contrastive learning with spiking neurons.
result True gradients for learning are provably computable by individual neurons.
Gradient descent dynamics in neural networks show quenching and activation phases.
problem Understanding training dynamics in neural networks.
method Numerical and phenomenological study of gradient descent algorithm for two-layer neural networks.
result Gradient descent dynamics exhibit quenching and activation phases in under-parametrized networks.
Neurons in cortical circuits exhibit coordinated spiking activity, and can produce correlated synchronous spikes during behavior and cognition. We recently developed a method for estimating the dynamics of correlated ensemble activity by combining a model of simultaneous neuronal interactions (e.g., a spin-glass model)…
Develops LSTM for predicting neuronal dynamics over long time-horizons.
problem Understanding and controlling complex brain behaviors.
method Long Short-Term Memory (LSTM) neural network architecture for multi-time step predictions.
result LSTM improves short time-horizon prediction accuracy and multi-time step predictions of neuronal dynamics.
Perfect spike detection method derived and benchmarked.
problem Missing spikes in neuronal network simulations.
method Time-inverted dynamics to test threshold crossing.
result Method can be faster than alternative perfect tests.
PINNs solve neuronal parameter and state estimation problems with limited data.
problem Estimating parameters and hidden state variables from noisy partial data in multiscale neuronal models.
method Physics-informed neural networks (PINNs) for joint state and parameter estimation.
result PINNs deliver robust and accurate parameter inference and state reconstruction, even with limited data.
In the wake of recent advances in experimental methods in neuroscience, the ability to record in-vivo neuronal activity from awake animals has become feasible. The availability of such rich and detailed physiological measurements calls for the development of advanced data analysis tools, as commonly used techniques do …
The seemingly stochastic transient dynamics of neocortical circuits observed in vivo have been hypothesized to represent a signature of ongoing stochastic inference. In vitro neurons, on the other hand, exhibit a highly deterministic response to various types of stimulation. We show that an ensemble of deterministic le…
New training algorithm enhances SNNs for temporal signal processing.
problem Lack of robust training algorithms for large-scale SNNs.
method Formulated SNN as IIR filters, proposed training algorithm for optimal synapse filter kernels and weights.
result Model and training algorithm outperform state-of-the-art approaches in accuracy.
High-conductance neurons sample from target distributions in stochastic inference.
problem Understanding stochastic inference in neocortical circuits.
method Analytical derivation of neural activation function, simulation of spiking networks, Bayesian inference.
result Ensemble of spiking neurons can sample from a target distribution.
Neurons predict future scalar inputs by learning top modes of lag vectors.
problem Predicting future scalar inputs with physiological delays.
method Normal Mode Decomposition to extract independently evolving modes.
result Temporal filters of neurons correspond to left eigenvectors of a generalized eigenvalue problem.
Modeling hidden neurons in SNNs using mesoscopic approximations.
problem Underconstrained problem of modeling unobserved neurons in SNNs.
method Coarse-graining and mean-field approximations to derive neuLVM.
result neuLVM can efficiently model large SNNs and recover connectivity parameters.
T-SVM improves learning in spiking neurons by maximizing dynamical margin.
problem Finding robust solutions in spiking neuronal networks with temporal correlations.
method Introduces Temporal Support Vector Machine (T-SVM) to maximize dynamical margin.
result T-SVM enables learning of tasks requiring nonlinear spatial integration.
Complex neural networks simplify to a mean field model as the number of neurons increases.
problem Understanding the behavior of multilayer neural networks with many neurons.
method Developed a mean field limit formalism for multilayer neural networks under stochastic gradient descent.
result The behavior of multilayer neural networks simplifies to a mean field model as the number of neurons grows large.
New ABC method for Bayesian inference of neural models.
problem Challenging to identify which mechanistic models quantitatively reproduce neural data.
method Likelihood-free inference using neural networks and Bayesian mixture-density networks.
result Efficiently estimates posterior distributions and recovers ground-truth parameters.
Gradient descent slows significantly in over-parameterized single neuron learning.
problem Learning a single neuron with over-parameterization and square loss.
method Analysis of gradient descent dynamics, proving convergence rates and lower bounds.
result Over-parameterization can exponentially slow down the convergence rate of gradient descent.
Develops CLDS models to model neural activity with nonlinear dynamics.
problem Complex, nonlinear dynamics in neural population activity.
method Conditionally Linear Dynamical System (CLDS) models using Gaussian Process (GP) priors.
result CLDS models can perform well even in data-limited conditions.
New model explains complex, nonlinear systems with simpler units that switch based on observations.
problem Complex, nonlinear systems with switching dynamics.
method Recurrent switching linear dynamical systems (recSLDS) model.
result Models the switching behavior of simpler units based on observations or latent states.
SpaRCe optimizes reservoir computing by learning neuron thresholds to improve performance and prevent forgetting.
problem Improving performance and preventing forgetting in reservoir computing networks.
method Integrates neuron-specific learnable thresholds to optimize sparsity without altering dynamics, learning read-out weights and thresholds via gradient rule.
result Threshold learning improves performance and alleviates catastrophic forgetting.
New method optimizes objective function for vector field dynamics.
problem Neuroscientists' doubts about backpropagation algorithm.
method Two-phase learning procedure for fixed point recurrent networks.
result Algorithm optimizes objective function without computing true gradient.
Solves internal covariate shift and dying neurons with linked neurons.
problem Internal covariate shift and dying neurons in deep learning.
method Defining linked neurons with two constraints: shared operating point and non-zero gradient.
result Linked neurons effectively solve internal covariate shift and improve training efficiency.
Model neural plasticity as binary optimization to dynamically activate or deactivate network units.
problem Dynamic learning and adaptability of neural networks.
method Model neural plasticity as an L0-norm regularized binary optimization problem, where units can be activated or deactivated based on a cost-benefit tradeoff. result Demonstrates that a single parameter k can modulate learning dynamics, unifying network sparsification and expansion. AGF explains feature learning in neural networks through alternating steps.
problem Understanding what features neural networks learn and how they learn them.
method AGF is an algorithmic framework that approximates the dynamics of feature learning in two-layer networks.
result AGF provides a unified framework to understand feature learning in neural networks, matching experimental results across various architectures.
Improved SNNs for speech classification with PyTorch.
problem Training convolutional spiking neural networks efficiently.
method Supervised training using backpropagation through time, surrogate gradient, and novel connections.
result Achieved nearly state-of-the-art accuracy on speech classification task.
Neural mesh models brain-like neural network dynamics with energy conservation.
problem Traditional neural networks lack the complexity of brain-like interactions and energy dynamics.
method Developed a neural network architecture with persistent state, adjacency constraints, and energy conservation.
result Increased accuracy of neural mesh without increasing parameters by extending runtime.
New learning algorithm mimics biological neural networks.
problem Biologically implausible backpropagation for directed neural networks.
method Introduces new neuronal dynamics and learning rule for arbitrary architectures, sparsity-inducing pruning method, and dynamical-systems characterization.
result Prunes irrelevant connections and improves learning efficiency.
A new neural network uses stigmergy to incorporate time.
problem Designing neural networks that can handle temporal data.
method Employing computational stigmergy to dynamically adjust neuron activation levels.
result Solved the XOR problem with a single stigmergic neuron.
Trains recurrent networks to perform complex tasks with fewer neurons and better robustness.
problem Training recurrent networks to handle temporally complex tasks efficiently.
method Introduces a target network to provide suitable dynamics for the task, modifying the full connectivity matrix.
result Trained networks perform tasks with fewer neurons and greater noise robustness.
DSG activates only a small amount of neurons for efficient deep learning.
problem Efficient deployment of deep neural networks on embedded devices.
method Dynamic and sparse graph structure with dimension-reduction search and double-mask selection.
result Significant memory saving (1.7-4.5x) and operation reduction (2.3-4.4x) with little accuracy loss.
EP learns like BPTT but with local weight updates.
problem Existing EP lacks a local time learning rule.
method C-EP updates weights simultaneously with neuron dynamics.
result C-EP follows BPTT gradients and performs well.
Paper models Pavlov's classical conditioning using stochastic processes and Langevin equations.
problem Lack of modeling for Pavlov's classical conditioning.
method Modeling neural and synaptic dynamics via Langevin equations.
result Pavlov's mechanism spontaneously leads to synaptic weights similar to Hebb's.
Deep learning has recently led to great successes in tasks such as image recognition (e.g Krizhevsky et al., 2012). However, deep networks are still outmatched by the power and versatility of the brain, perhaps in part due to the richer neuronal computations available to cortical circuits. The challenge is to identify …
A new theory explains large associative memory with biological plausibility.
problem Large associative memory in neurobiology and machine learning.
method Microscopic theory with hidden neurons and two-body interactions.
result Valid model of large associative memory with biological plausibility.
Neural circuit model re-purposed for robotic control tasks.
problem Learning simple robotic control tasks.
method Re-purposing a biological neural circuit model to control robotic tasks using a search-based optimization algorithm.
result Neuronal Circuit Policies (NCPs) perform on par and in some cases surpass contemporary deep learning models with fewer parameters and interpretable dynamics.
This paper explores adaptive neural activation in RNNs for better learning.
problem Fixed neural activation functions limit the performance and adaptability of RNNs.
method Developed a novel parametric family of nonlinear activation functions inspired by biological neurons.
result Adaptive neural activation improves learning speed and performance in RNNs.
Biological neurons learn tensor decompositions of higher-order correlations using nonlinear Hebbian plasticity.
problem Learning higher-order correlations in biological neurons.
method Introduce and study generalized nonlinear Hebbian learning rules.
result Neurons can learn tensor eigenvectors of higher-order input correlation tensors.
New birth-death dynamics accelerates convergence in neural networks.
problem Accelerating convergence in neural networks with large parameters.
method Proposed a non-local mass transport dynamics as a stochastic neuronal birth-death process.
result Proved that the birth-death dynamics accelerates the rate of convergence in the mean-field limit.
The paper examines how deep linear neural networks behave as they become infinitely wide.
problem Understanding the behavior of deep linear neural networks as they approach infinite width.
method Analyzes the infinite-width limit of deep linear neural networks, proving convergence to deterministic models and providing precise laws for random weights.
result The training dynamics of deep linear neural networks converge to those of a deterministic model, and the weights' behavior is precisely described.
Method infers dynamics from incomplete time series data.
problem Challenges in inferring stochastic dynamics from time series with missing data.
method Expectation Maximization (EM) algorithm that iterates between E-step and M-step.
result The EM algorithm effectively recovers missing data points and infers underlying network models from real neuronal activities.
Experiments that study neural encoding of stimuli at the level of individual neurons typically choose a small set of features present in the world --- contrast and luminance for vision, pitch and intensity for sound --- and assemble a stimulus set that systematically varies along these dimensions. Subsequent analysis o…
Neural population activity often exhibits rich variability and temporal structure. This variability is thought to arise from single-neuron stochasticity, neural dynamics on short time-scales, as well as from modulations of neural firing properties on long time-scales, often referred to as "non-stationarity". To better …
Bayesian method detects neuron spike activities from noisy fluorescence data.
problem Detecting neuron spike activities from noisy fluorescence data.
method Random finite set (RFS) based Bayesian approach.
result Gains 12% extra detection accuracy compared to MLSpike method.