Study evaluates CL methods in RNNs, highlighting differences from feedforward networks.
problem Preventing catastrophic forgetting in RNNs processing sequential data.
method Comprehensive evaluation of CL methods, including elastic weight consolidation and hypernetworks.
result Weight-importance methods perform similarly regardless of sequence length but require more stability for high working memory demands.
Neuromorphic column performs online unsupervised clustering.
problem Real-time clustering of streaming data.
method Localized, spike timing-dependent plasticity (STDP) neural column.
result Prototype column performs similarly to k-means clustering.
How can we build agents that keep learning from experience, quickly and efficiently, after their initial training? Here we take inspiration from the main mechanism of learning in biological brains: synaptic plasticity, carefully tuned by evolution to produce efficient lifelong learning. We show that plasticity, just li…
Introduces generalized almost plastic structures on manifolds.
problem Integrability of plastic structures on manifolds.
method Constructs a family of generalized almost plastic structures on pseudo-Riemannian manifolds with specific tensor fields and compatibility conditions.
result Characterizes the integrability of these structures with respect to a given affine connection.
Model for material elasticity and plasticity using networks.
problem Understanding the elasticity and plasticity of materials.
method Developed a mathematical model based on networks, defining tension tensor for periodic graphs.
result The model explains elasticity and plasticity through local moves on graphs.
Natural gradient learning improves synaptic plasticity in spiking neurons.
problem Parametrization dependence leads to inconsistencies in classical synaptic plasticity theories.
method Proposes natural gradient descent in Riemannian geometry for spiking neurons.
result Derives a synaptic learning rule that explains biological phenomena.
AANets balance stability and plasticity in CIL.
problem Stability-plasticity dilemma in class-incremental learning.
method Adaptive Aggregation Networks (AANets) with stable and plastic residual blocks.
result AANets improve performance on CIL benchmarks.
Flashback Learning balances model stability and plasticity in continual learning.
problem Balancing model stability and plasticity in continual learning.
method Flashback Learning (FL) uses a bidirectional regularization approach to balance stability and plasticity.
result FL improves model accuracy by up to 4.91% in Class-Incremental and 3.51% in Task-Incremental settings.
Physics-informed deep learning approximates strain gradient plasticity solutions.
problem Stiffness and computational challenges in solving strain gradient plasticity models.
method Physics-informed deep learning (PIDL) with modified loss functions and optimization schemes.
result PIDL methods address stiffness and computational challenges in strain gradient plasticity.
Biological neurons learn tensor decompositions of higher-order correlations using nonlinear Hebbian plasticity.
problem Learning higher-order correlations in biological neurons.
method Introduce and study generalized nonlinear Hebbian learning rules.
result Neurons can learn tensor eigenvectors of higher-order input correlation tensors.
A model retains learned knowledge for longer by adding a plastic component to neural networks.
problem Catastrophic forgetting in neural networks when learning new tasks.
method Differentiable Hebbian Consolidation model with a DHP Softmax layer.
result Reduces forgetting in benchmarks like Permuted MNIST and Vision Datasets Mixture.
Vision transformers benefit from non-smooth components in adaptation.
problem Understanding the role of non-smoothness in vision transformer adaptation.
method Theoretical analysis and extensive experiments on large-scale vision transformers.
result High plasticity of attention modules and feedforward layers leads to better finetuning performance.
Unified approach for neural networks with multi-compartmental neurons and non-Hebbian plasticity.
problem Limited computational power of existing neural network models for multi-compartmental neurons and non-Hebbian plasticity.
method Unified extension of similarity matching approach to derive neural networks with multi-compartmental neurons and local, non-Hebbian learning rules.
result Unified approach facilitates understanding of multi-compartmental neuronal structures and non-Hebbian plasticity.
Oja's rule improves neural network training without engineered tricks.
problem Training deep neural networks with biological constraints.
method Incorporating Oja's plasticity rule into error-driven training.
result Stable, efficient learning in feedforward and recurrent architectures.
Metric anomalies arising from a distribution of point defects (intrinsic interstitials, vacancies, point stacking faults), thermal deformation, biological growth, etc. are well known sources of material inhomogeneity and internal stress. By emphasizing the geometric nature of such anomalies we seek their representation…
Learning and memory in the brain are implemented by complex, time-varying changes in neural circuitry. The computational rules according to which synaptic weights change over time are the subject of much research, and are not precisely understood. Until recently, limitations in experimental methods have made it challen…
Paper proposes a new AI design philosophy for distributed learning.
problem Designing AI systems that mimic human cognitive capabilities.
method Democratized learning (Dem-AI) with self-organized hierarchical agents.
result Self-organized learning systems can perform complex tasks more efficiently.
Unified framework for strain-gradient plasticity from dislocations.
problem Deriving strain-gradient plasticity from edge-dislocations.
method Γ-limit derivation in a continuum framework with smooth frame fields and dislocation circulation.
result Unified strain-gradient model with new geometric rigidity estimates.
Selective reinitialization improves adaptability of neural bandits in dynamic environments.
problem Loss of plasticity in neural bandits, leading to rigid neural network parameters.
method Selective Reinitialization (SeRe) framework that dynamically resets underutilized units.
result SeRe enhances adaptability of CNB algorithms, reducing cumulative regret in dynamic environments.
This paper presents a method to improve continual learning stability and plasticity.
problem Balancing learning stability and plasticity in deep learning.
method Batch-level Experience Replay with Review approach.
result Achieved 1st place in all three scenarios of the CVPR 2020 CLVision challenge.
This paper suggests a learning-theoretic perspective on how synaptic plasticity benefits global brain functioning. We introduce a model, the selectron, that (i) arises as the fast time constant limit of leaky integrate-and-fire neurons equipped with spiking timing dependent plasticity (STDP) and (ii) is amenable to the…
Elite ONNs learn better with synaptic plasticity, improving performance over CNNs.
problem Limited heterogeneity in ONNs due to fixed operator sets.
method Synaptic plasticity-based search for optimal operator sets.
result Elite ONNs achieve superior learning performance compared to conventional methods.
Dropout helps a stable network learn new tasks without forgetting old ones.
problem Catastrophic forgetting in neural networks when learning multiple tasks.
method Investigate the relationship between dropout and stability in neural networks, showing dropout acts as an implicit gating mechanism.
result Dropout stabilizes a network's learning, allowing it to learn new tasks without forgetting old ones.
The paper proposes a new framework to generate synthetic data with human-like imperfections to prevent model collapse.
problem Model collapse due to statistical optimization of synthetic data.
method Introduces Prompt-driven Cognitive Computing Framework (PMCSF) with Cognitive State Decoder (CSD) and Cognitive Text Encoder (CTE).
result The framework generates text with cognitive imperfections, reducing maximum drawdown and delivering defensive alpha.
Study shows cognitive load impacts financial market efficiency, especially for less sophisticated investors.
problem Cognitive load's effect on financial market information processing.
method Developed a theoretical framework and tested it with exogenous disclosure complexity variation.
result Cognitive load significantly impairs price discovery, particularly for less sophisticated investors.
A cognitive model of human learning provides information about skills a learner must acquire to perform accurately in a task domain. Cognitive models of learning are not only of scientific interest, but are also valuable in adaptive online tutoring systems. A more accurate model yields more effective tutoring through b…
Pension benefits in rural China lead to cognitive decline among the elderly.
problem Cognitive decline in late adulthood among rural Chinese elderly.
method Examined the effects of a new pension scheme on cognitive performance.
result Pension benefits negatively impact cognitive functioning, particularly delayed recall.
We identify a phenomenon, which we refer to as multi-model forgetting, that occurs when sequentially training multiple deep networks with partially-shared parameters; the performance of previously-trained models degrades as one optimizes a subsequent one, due to the overwriting of shared parameters. To overcome this, w…
The ubiquity of smartphone usage in many people's lives make it a rich source of information about a person's mental and cognitive state. In this work we analyze 12 weeks of phone usage data from 113 older adults, 31 with diagnosed cognitive impairment and 82 without. We develop structured models of users' smartphone i…
Cognitive neuroscience is enjoying rapid increase in extensive public brain-imaging datasets. It opens the door to large-scale statistical models. Finding a unified perspective for all available data calls for scalable and automated solutions to an old challenge: how to aggregate heterogeneous information on brain func…
In the domain of machine learning, Neural Memory Networks (NMNs) have recently achieved impressive results in a variety of application areas including visual question answering, trajectory prediction, object tracking, and language modelling. However, we observe that the attention based knowledge retrieval mechanisms us…
In the eyes of a rationalist like Descartes or Spinoza, human reasoning is flawless, marching toward uncovering ultimate truth. A few centuries later, however, culminating in the work of Kahneman and Tversky, human reasoning was portrayed as anything but flawless, filled with numerous misjudgments, biases, and cognitiv…
Sleep-based regularization stabilizes STDP in recurrent neural networks.
problem Pathological weight dynamics in recurrent SNNs.
method Periodic offline phases with stochastic decay and spontaneous activity.
result Sleep-based renormalization prevents weight saturation and preserves learned structure.
Analyze and predict complex 3D shape deformations using LSTM autoencoders and oriented bounding boxes.
problem Detecting and predicting patterns in sequences of deforming 3D shapes.
method Use LSTM autoencoders to create low-dimensional representations of 3D shapes, incorporating oriented bounding boxes for structural components.
result The method detects patterns in plastic deformation and predicts future states of 3D shapes with improved accuracy.
New method identifies latent variables in cognitive models using neural networks.
problem Inference of latent variables in complex cognitive models is limited.
method Recurrent neural networks and simulation-based inference for latent variable sequences.
result Extends neural Bayes estimation to broader classes of cognitive models.
Catastrophic forgetting/interference is a critical problem for lifelong learning machines, which impedes the agents from maintaining their previously learned knowledge while learning new tasks. Neural networks, in particular, suffer plenty from the catastrophic forgetting phenomenon. Recently there has been several eff…
While the interpretability of machine learning models is often equated with their mere syntactic comprehensibility, we think that interpretability goes beyond that, and that human interpretability should also be investigated from the point of view of cognitive science. The goal of this paper is to discuss to what exten…
Study links cognitive effort to thermodynamic principles, optimizing decision-making.
problem Understanding the energy cost of decision-making in living systems.
method Applied Landauer's principle to analyze belief dynamics in a two-armed bandit game.
result Cognitive energy cost correlates with optimal decision-making outcomes.
New filters improve radar target inference in complex scenarios.
problem Improving radar target inference in highly non-linear system models.
method Developed inverse cubature Kalman filter (I-CKF), inverse quadrature Kalman filter (I-QKF), and inverse cubature-quadrature Kalman filter (I-CQKF) for non-linear systems.
result Numerical experiments show improved estimation accuracy compared to existing methods.
Accurate diagnosis of Alzheimer's Disease (AD) entails clinical evaluation of multiple cognition metrics and biomarkers. Metrics such as the Alzheimer's Disease Assessment Scale - Cognitive test (ADAS-cog) comprise multiple subscores that quantify different aspects of a patient's cognitive state such as learning, memor…
For effective treatment of Alzheimer disease (AD), it is important to identify subjects who are most likely to exhibit rapid cognitive decline. Herein, we developed a novel framework based on a deep convolutional neural network which can predict future cognitive decline in mild cognitive impairment (MCI) patients using…
Study improves robustness of Bayesian inference for cognitive models.
problem Outliers and contaminants affect parameter estimation in cognitive models.
method Robustness of parameter estimation using amortized Bayesian inference (ABI) with neural networks.
result Introducing contaminants from a Cauchy distribution increases robustness.
Develops an inverse particle filter for cognitive systems.
problem Tracking cognitive adversaries in counter-adversarial applications.
method Global filtering approach using Monte Carlo methods and differentiable I-PF.
result Demonstrates convergence to optimal inverse filter and improved estimation performance.
New methods make Bayesian inference feasible for complex cognitive models.
problem Bayesian inference is hard for complex, unknown models.
method Amortized neural network estimators for simulation-based inference.
result Effort in training networks amortizes over multiple uses.
Resting-state functional MRI (rs-fMRI) in functional neuroimaging techniques have improved in brain disorders, dysfunction studies via mapping the topology of the brain connections, i.e. connectopic mapping. Since, there are the slight differences between healthy and unhealthy brain regions and functions, investigation…
QCML uses quantum geometry to represent data.
problem Data representation and the curse of dimensionality.
method QCML encodes data as Hermitian matrices in Hilbert space.
result Data geometry reveals intrinsic dimension and topological properties.
We propose a particularly structured Boltzmann machine, which we refer to as a dynamic Boltzmann machine (DyBM), as a stochastic model of a multi-dimensional time-series. The DyBM can have infinitely many layers of units but allows exact and efficient inference and learning when its parameters have a proposed structure…
We propose a neural superstatistics method to estimate dynamic cognitive models from time series data.
problem Memoryless cognitive models ignore parameter fluctuations, leading to inaccurate predictions.
method Developed a simulation-based deep learning method for Bayesian inference of superstatistical models.
result Deep learning method efficiently recovers time-varying and time-invariant parameters.