Oja's rule improves neural network training without engineered tricks.
problem Training deep neural networks with biological constraints.
method Incorporating Oja's plasticity rule into error-driven training.
result Stable, efficient learning in feedforward and recurrent architectures.
A neural network learns to control a two-link arm with non-linear dynamics.
problem Training spiking neural networks to control complex, non-linear systems.
method Feedback-based Online Local Learning Of Weights (FOLLOW) to train a network of spiking neurons with hidden layers.
result The network learns an inverse model of the arm's dynamics and uses it to generate a motor command for control.
New method improves deep learning performance without weight symmetry.
problem Challenges in scaling non-symmetric learning methods to deep convolutional networks.
method Introduced techniques to mitigate scalability issues, including a modified feedback alignment method.
result Demonstrated competitive performance with backpropagation using a weaker form of weight symmetry.
Proposes VPF for efficient training of DBMs without Gibbs sampling or feedback phases.
problem Efficient training of deep neural networks with biological plausibility.
method Variational Probability Flow (VPF) for binary Deep Boltzmann Machines (DBMs).
result VPF learns features quickly and generates high-likelihood samples.
Differentiable plasticity enables efficient lifelong learning in neural networks.
problem Building agents that can learn from experience quickly and efficiently.
method Optimized plastic connections in recurrent neural networks using gradient descent.
result Plastic neural networks can learn and reconstruct novel images and solve meta-learning tasks.
New STDP rule for spiking neurons solves discrete action reinforcement learning tasks.
problem Applying standard STDP to discrete action reinforcement learning tasks.
method Feedback-modulated TD-STDP learning rule for spiking neuron networks.
result Feedback modulation improves credit assignment in reinforcement learning.
Proposes a new neural network approach to credit assignment.
problem Credit assignment problem in deep neural networks.
method Contrastive similarity matching objective function.
result Deep networks learn to match similarity between layers.
Introduces generalized almost plastic structures on manifolds.
problem Integrability of plastic structures on manifolds.
method Constructs a family of generalized almost plastic structures on pseudo-Riemannian manifolds with specific tensor fields and compatibility conditions.
result Characterizes the integrability of these structures with respect to a given affine connection.
Model for material elasticity and plasticity using networks.
problem Understanding the elasticity and plasticity of materials.
method Developed a mathematical model based on networks, defining tension tensor for periodic graphs.
result The model explains elasticity and plasticity through local moves on graphs.
Natural gradient learning improves synaptic plasticity in spiking neurons.
problem Parametrization dependence leads to inconsistencies in classical synaptic plasticity theories.
method Proposes natural gradient descent in Riemannian geometry for spiking neurons.
result Derives a synaptic learning rule that explains biological phenomena.
AANets balance stability and plasticity in CIL.
problem Stability-plasticity dilemma in class-incremental learning.
method Adaptive Aggregation Networks (AANets) with stable and plastic residual blocks.
result AANets improve performance on CIL benchmarks.
Flashback Learning balances model stability and plasticity in continual learning.
problem Balancing model stability and plasticity in continual learning.
method Flashback Learning (FL) uses a bidirectional regularization approach to balance stability and plasticity.
result FL improves model accuracy by up to 4.91% in Class-Incremental and 3.51% in Task-Incremental settings.
Physics-informed deep learning approximates strain gradient plasticity solutions.
problem Stiffness and computational challenges in solving strain gradient plasticity models.
method Physics-informed deep learning (PIDL) with modified loss functions and optimization schemes.
result PIDL methods address stiffness and computational challenges in strain gradient plasticity.
Biological neurons learn tensor decompositions of higher-order correlations using nonlinear Hebbian plasticity.
problem Learning higher-order correlations in biological neurons.
method Introduce and study generalized nonlinear Hebbian learning rules.
result Neurons can learn tensor eigenvectors of higher-order input correlation tensors.
A model retains learned knowledge for longer by adding a plastic component to neural networks.
problem Catastrophic forgetting in neural networks when learning new tasks.
method Differentiable Hebbian Consolidation model with a DHP Softmax layer.
result Reduces forgetting in benchmarks like Permuted MNIST and Vision Datasets Mixture.
Unified approach for neural networks with multi-compartmental neurons and non-Hebbian plasticity.
problem Limited computational power of existing neural network models for multi-compartmental neurons and non-Hebbian plasticity.
method Unified extension of similarity matching approach to derive neural networks with multi-compartmental neurons and local, non-Hebbian learning rules.
result Unified approach facilitates understanding of multi-compartmental neuronal structures and non-Hebbian plasticity.
Vision transformers benefit from non-smooth components in adaptation.
problem Understanding the role of non-smoothness in vision transformer adaptation.
method Theoretical analysis and extensive experiments on large-scale vision transformers.
result High plasticity of attention modules and feedforward layers leads to better finetuning performance.
Metric anomalies arising from a distribution of point defects (intrinsic interstitials, vacancies, point stacking faults), thermal deformation, biological growth, etc. are well known sources of material inhomogeneity and internal stress. By emphasizing the geometric nature of such anomalies we seek their representation…
Learning and memory in the brain are implemented by complex, time-varying changes in neural circuitry. The computational rules according to which synaptic weights change over time are the subject of much research, and are not precisely understood. Until recently, limitations in experimental methods have made it challen…
Proposes a plastic neural memory model for better anomaly detection.
problem Static attention mechanisms limit NMNs in anomaly detection.
method Introduces dynamic connection weights for improved knowledge retrieval.
result Outperforms state-of-the-art in three medical anomaly detection tasks.
Unified framework for strain-gradient plasticity from dislocations.
problem Deriving strain-gradient plasticity from edge-dislocations.
method Γ-limit derivation in a continuum framework with smooth frame fields and dislocation circulation.
result Unified strain-gradient model with new geometric rigidity estimates.
Selective reinitialization improves adaptability of neural bandits in dynamic environments.
problem Loss of plasticity in neural bandits, leading to rigid neural network parameters.
method Selective Reinitialization (SeRe) framework that dynamically resets underutilized units.
result SeRe enhances adaptability of CNB algorithms, reducing cumulative regret in dynamic environments.
This paper presents a method to improve continual learning stability and plasticity.
problem Balancing learning stability and plasticity in deep learning.
method Batch-level Experience Replay with Review approach.
result Achieved 1st place in all three scenarios of the CVPR 2020 CLVision challenge.
This paper suggests a learning-theoretic perspective on how synaptic plasticity benefits global brain functioning. We introduce a model, the selectron, that (i) arises as the fast time constant limit of leaky integrate-and-fire neurons equipped with spiking timing dependent plasticity (STDP) and (ii) is amenable to the…
Multi-model forgetting occurs when training multiple deep networks sequentially, leading to performance degradation of previously trained models.
problem Performance degradation of previously trained models when sequentially training multiple deep networks with shared parameters.
method Introduce a weight plasticity loss that regularizes the learning of shared parameters based on their importance for previous models.
result Weight plasticity loss effectively preserves the performance of previously trained models during sequential training and neural architecture search.
Elite ONNs learn better with synaptic plasticity, improving performance over CNNs.
problem Limited heterogeneity in ONNs due to fixed operator sets.
method Synaptic plasticity-based search for optimal operator sets.
result Elite ONNs achieve superior learning performance compared to conventional methods.
Model neural plasticity as binary optimization to dynamically activate or deactivate network units.
problem Dynamic learning and adaptability of neural networks.
method Model neural plasticity as an L0-norm regularized binary optimization problem, where units can be activated or deactivated based on a cost-benefit tradeoff. result Demonstrates that a single parameter k can modulate learning dynamics, unifying network sparsification and expansion. Proposes a method to prevent neural networks from forgetting learned tasks.
problem Catastrophic forgetting in neural networks.
method Attention-based selective plasticity of synapses inspired by the cholinergic neuromodulatory system.
result Competitive performance on benchmark tasks compared to state-of-the-art methods.
Dropout helps a stable network learn new tasks without forgetting old ones.
problem Catastrophic forgetting in neural networks when learning multiple tasks.
method Investigate the relationship between dropout and stability in neural networks, showing dropout acts as an implicit gating mechanism.
result Dropout stabilizes a network's learning, allowing it to learn new tasks without forgetting old ones.
Neural network memorizes external stimuli through synaptic strength changes.
problem Memory and classification in neural networks.
method One-to-one mapping between stimulus and synaptic strength under synaptic plasticity constraints.
result Neural network can memorize external stimuli through synaptic changes.
A neuromemristive HTM architecture boosts robustness and performance.
problem Improving robustness and performance of HTM algorithms.
method Developed a neuromemristive crossbar architecture for HTM, incorporating memristors and neurogenesis.
result Enhanced robustness and performance of HTM through neuromemristive architecture.
Sleep-based regularization stabilizes STDP in recurrent neural networks.
problem Pathological weight dynamics in recurrent SNNs.
method Periodic offline phases with stochastic decay and spontaneous activity.
result Sleep-based renormalization prevents weight saturation and preserves learned structure.
Analyze and predict complex 3D shape deformations using LSTM autoencoders and oriented bounding boxes.
problem Detecting and predicting patterns in sequences of deforming 3D shapes.
method Use LSTM autoencoders to create low-dimensional representations of 3D shapes, incorporating oriented bounding boxes for structural components.
result The method detects patterns in plastic deformation and predicts future states of 3D shapes with improved accuracy.
EILearn learns incrementally using previous classifier knowledge.
problem Incremental learning with previous data.
method Retains and uses previous classifier knowledge, monitors performance, eliminates poorly performing classifiers.
result Outperforms existing incremental learning approaches.
We propose a particularly structured Boltzmann machine, which we refer to as a dynamic Boltzmann machine (DyBM), as a stochastic model of a multi-dimensional time-series. The DyBM can have infinitely many layers of units but allows exact and efficient inference and learning when its parameters have a proposed structure…
A new learning framework mimics biological STDP for neural networks.
problem To create a neural network that learns like biological systems.
method Developed MSTDP framework using Spike-timing dependent plasticity rules.
result Framework can learn and generate patterns without additional supervision.
A high-parallelism SNN improves feature learning efficiency and robustness.
problem Slow learning speed and limited learning capability in existing SNNs.
method Inspired by Inception modules, high-parallelism architecture, Vote-for-All decoding, adaptive repolarization mechanism.
result Superior performance and competitive accuracy compared to state-of-the-art unsupervised SNNs.
A dynamic Boltzmann machine (DyBM) has been proposed as a model of a spiking neural network, and its learning rule of maximizing the log-likelihood of given time-series has been shown to exhibit key properties of spike-timing dependent plasticity (STDP), which had been postulated and experimentally confirmed in the fie…
Improved neural ODEs learn adaptable flows.
problem Neural ODEs struggle with expressive power and adaptability.
method Introduce N-CODE modules with dynamic parameters controlled by a trainable map.
result N-CODE modules enhance expressivity of neural ODEs.
New synaptic model derived from reinforcement learning for spiking neurons.
problem Learning optimal actions in complex systems.
method Derives a synaptic update rule from reinforcement learning algorithms.
result Synaptic strengths lead to locally optimal reward values.
Study examines how training regime affects neural networks' forgetting.
problem Catastrophic forgetting in neural networks when learning multiple tasks sequentially.
method Analyzes the impact of different training regimes (learning rate, batch size, regularization) on forgetting.
result Training regimes that widen tasks' local minima help prevent catastrophic forgetting.
Spiking neural networks (SNNs) could play a key role in unsupervised machine learning applications, by virtue of strengths related to learning from the fine temporal structure of event-based signals. However, some spike-timing-related strengths of SNNs are hindered by the sensitivity of spike-timing-dependent plasticit…
New insights into continual learning with task similarity.
problem Challenges in learning similar tasks without interference.
method Linear teacher-student model with latent structure.
result High input feature similarity with low readout similarity is catastrophic.
The edges of torn plastic sheets and growing leaves often display hierarchical buckling patterns. We show that this complex morphology (i) emerges even in zero strain configurations, and (ii) is driven by a competition between the two principal curvatures, rather than between bending and stretching. We identify the key…
A new neural model evolves to learn at the synaptic level.
problem Lack of biologically realistic neural models in deep learning.
method Evolve individual neuron and synaptic models using ENUs.
result Evolved neural network learns complex tasks like a T-maze.
ABS dynamically adjusts batch size based on policy stability, improving RL performance.
problem Diminishing returns with large batch sizes in RL due to non-stationary data.
method Adaptive Batch Scaling (ABS) with Behavioral Divergence metric.
result Larger batch sizes can improve RL performance, contrary to conventional wisdom.
Review of lifelong learning challenges for neural networks.
problem Catastrophic forgetting in neural networks when learning from non-stationary data.
method Comparison of existing approaches to mitigate forgetting.
result Lifelong learning is crucial for real-world machine learning, but challenging.
Study evaluates CL methods in RNNs, highlighting differences from feedforward networks.
problem Preventing catastrophic forgetting in RNNs processing sequential data.
method Comprehensive evaluation of CL methods, including elastic weight consolidation and hypernetworks.
result Weight-importance methods perform similarly regardless of sequence length but require more stability for high working memory demands.