A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Combines Hebbian and DQN for better POMDP problem solving.
Unified approach for neural networks with multi-compartmental neurons and non-Hebbian plasticity.
Hebbian learning derived from maximum entropy principles.
In this work we propose Hebbian-descent as a biologically plausible learning rule for hetero-associative as well as auto-associative learning in single layer artificial neural networks. It can be used as a replacement for gradient descent as well as Hebbian learning, in particular in online learning, as it inherits the…
Neural network models of early sensory processing typically reduce the dimensionality of streaming input data. Such networks learn the principal subspace, in the sense of principal component analysis (PCA), by adjusting synaptic weights according to activity-dependent learning rules. When derived from a principled cost…
Enhanced BCPNN learns hidden representations without labels.
Proposes a new neural network approach to credit assignment.
A model retains learned knowledge for longer by adding a plastic component to neural networks.
Paper introduces supervised and unsupervised TAM models for binary neurons.
Biological neurons learn tensor decompositions of higher-order correlations using nonlinear Hebbian plasticity.
Neural networks are commonly trained to make predictions through learning algorithms. Contrastive Hebbian learning, which is a powerful rule inspired by gradient backpropagation, is based on Hebb's rule and the contrastive divergence algorithm. It operates in two phases, the forward (or free) phase, where the data are …
New learning rules from information bottleneck improve deep learning without precise labels.
Recently, a novel family of biologically plausible online algorithms for reducing the dimensionality of streaming data has been derived from the similarity matching principle. In these algorithms, the number of output dimensions can be determined adaptively by thresholding the singular values of the input data matrix. …
Despite our extensive knowledge of biophysical properties of neurons, there is no commonly accepted algorithmic theory of neuronal function. Here we explore the hypothesis that single-layer neuronal networks perform online symmetric nonnegative matrix factorization (SNMF) of the similarity matrix of the streamed data. …
Enhances sequence memory capacity in neural networks.
The study reveals how synaptic correlations promote dimension reduction in neural networks.
A new learning framework mimics biological STDP for neural networks.
Olshausen and Field (OF) proposed that neural computations in the primary visual cortex (V1) can be partially modeled by sparse dictionary learning. By minimizing the regularized representation error they derived an online algorithm, which learns Gabor-filter receptive fields from a natural image ensemble in agreement …
We show that deep networks can be trained using Hebbian updates yielding similar performance to ordinary back-propagation on challenging image datasets. To overcome the unrealistic symmetry in connections between layers, implicit in back-propagation, the feedback weights are separate from the feedforward weights. The f…
Sleep-based regularization stabilizes STDP in recurrent neural networks.
We unify recent neural approaches to one-shot learning with older ideas of associative memory in a model for metalearning. Our model learns jointly to represent data and to bind class labels to representations in a single shot. It builds representations via slow weights, learned across tasks through SGD, while fast wei…
The study revisits Hopfield's associative memory model and calculates its capacity for two specific pattern basins.
Bio-inspired neural networks use predictive coding for efficient weight updates.
New neural network can learn multiple patterns at once.
Recently a daily routine for associative neural networks has been proposed: the network Hebbian-learns during the awake state (thus behaving as a standard Hopfield model), then, during its sleep state, optimizing information storage, it consolidates pure patterns and removes spurious ones: this forces the synaptic matr…
New derivation shows how a three-factor learning rule is derived from Oja's rule.
The paper relaxes constraints on predictive coding models, making them more biologically plausible.
BDH model learns like the brain, rivaling Transformer performance.
How can we build agents that keep learning from experience, quickly and efficiently, after their initial training? Here we take inspiration from the main mechanism of learning in biological brains: synaptic plasticity, carefully tuned by evolution to produce efficient lifelong learning. We show that plasticity, just li…
Catastrophic forgetting/interference is a critical problem for lifelong learning machines, which impedes the agents from maintaining their previously learned knowledge while learning new tasks. Neural networks, in particular, suffer plenty from the catastrophic forgetting phenomenon. Recently there has been several eff…
Semi-supervised learning algorithms typically construct a weighted graph of data points to represent a manifold. However, an explicit graph representation is problematic for neural networks operating in the online setting. Here, we propose a feed-forward neural network capable of semi-supervised learning on manifolds w…
Develops a new method for solving generalized eigenvalue problems efficiently.
VSML unifies meta learning concepts and enables simple backpropagation.
Thanks to the availability of large scale digital datasets and massive amounts of computational power, deep learning algorithms can learn representations of data by exploiting multiple levels of abstraction. These machine learning methods have greatly improved the state-of-the-art in many challenging cognitive tasks, s…
Analyzes unsupervised neural networks using statistical mechanics and Monte Carlo simulations.
Oja's rule improves neural network training without engineered tricks.
We propose a new cognitive framework for option price modelling, using quantum neural computation formalism. Briefly, when we apply a classical nonlinear neural-network learning to a linear quantum Schrödinger equation, as a result we get a nonlinear Schrödinger equation (NLS), performing as a quantum stochastic filter…
Biological neural network mimics CCA for multi-channel data.
Study on neural networks with non-normal interactions reveals unique spectral properties.
Artifical Neural Networks are a particular class of learning systems modeled after biological neural functions with an interesting penchant for Hebbian learning, that is "neurons that wire together, fire together". However, unlike their natural counterparts, artificial neural networks have a close and stringent couplin…
Finding the reduced-dimensional structure is critical to understanding complex networks. Existing approaches such as spectral clustering are applicable only when the full network is explicitly observed. In this paper, we focus on the online factorization and partition of implicit large-scale networks based on observati…
A neural network learns from examples and optimizes by dreaming.
Local Hebbian learning is believed to be inferior in performance to end-to-end training using a backpropagation algorithm. We question this popular belief by designing a local algorithm that can learn convolutional filters at scale on large image datasets. These filters combined with patch normalization and very steep …
Neural networks can detect weak patterns hidden in noise.
We propose a modification of the cost function of the Hopfield model whose salient features shine in its Taylor expansion and result in more than pairwise interactions with alternate signs, suggesting a unified framework for handling both with deep learning and network pruning. In our analysis, we heavily rely on the H…
Dense neural networks learn efficiently with large datasets and noise.
New algorithm shows neural networks can learn without full backpropagation.
A central problem to understanding intelligence is the concept of generalisation. This allows previously learnt structure to be exploited to solve tasks in novel situations differing in their particularities. We take inspiration from neuroscience, specifically the hippocampal-entorhinal system known to be important for…