A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
This work bridges theory and practice in spiking reservoirs, identifying robust parameter ranges.
The seemingly stochastic transient dynamics of neocortical circuits observed in vivo have been hypothesized to represent a signature of ongoing stochastic inference. In vitro neurons, on the other hand, exhibit a highly deterministic response to various types of stimulation. We show that an ensemble of deterministic le…
This paper suggests a learning-theoretic perspective on how synaptic plasticity benefits global brain functioning. We introduce a model, the selectron, that (i) arises as the fast time constant limit of leaky integrate-and-fire neurons equipped with spiking timing dependent plasticity (STDP) and (ii) is amenable to the…
How spiking networks are able to perform probabilistic inference is an intriguing question, not only for understanding information processing in the brain, but also for transferring these computational principles to neuromorphic silicon circuits. A number of computationally powerful spiking network models have been pro…
The highly variable dynamics of neocortical circuits observed in vivo have been hypothesized to represent a signature of ongoing stochastic inference but stand in apparent contrast to the deterministic response of neurons measured in vitro. Based on a propagation of the membrane autocorrelation across spike bursts, we …
For a biological agent operating under environmental pressure, energy consumption and reaction times are of critical importance. Similarly, engineered systems are optimized for short time-to-solution and low energy-to-solution characteristics. At the level of neuronal implementation, this implies achieving the desired …
Random SNNs are stable and simple, with low-frequency Fourier spectra.
Single-spike neurons can approximate as well as multi-spike neurons.
Restricted Boltzmann Machine (RBM) is a bipartite graphical model that is used as the building block in energy-based deep generative models. Due to numerical stability and quantifiability of the likelihood, RBM is commonly used with Bernoulli units. Here, we consider an alternative member of exponential family RBM with…
Machine learning classifies topological phases in leaky photonic lattices.
Researchers develop methods to learn neuron dynamics from colored noise.
New insights into how linear classifiers and leaky ReLU networks can overfit without harming generalization.
Recently, it has been shown that spiking neural networks (SNNs) can be trained efficiently, in a supervised manner, using backpropagation through time. Indeed, the most commonly used spiking neuron model, the leaky integrate-and-fire neuron, obeys a differential equation which can be approximated using discrete time st…
We introduce tools to capture the dynamics of three different pathways, in which the synchronization of human decision-making could lead to turbulent periods and contagion phenomena in financial markets. The first pathway is caused when stock market indices, seen as a set of coupled integrate-and-fire oscillators, sync…
Paper finds conditions for benign overfitting in neural networks.
This paper introduces a new learning rule for probabilistic SNNs that improves log-likelihood, accuracy, and calibration.
Study on benign overfitting in leaky ReLUs with moderate input dimensions.
Leaky ReLU activations improve the calibration of Bayesian neural networks.
New methods for uncertainty in neural networks with leaky ReLU activations.
In this paper we investigate the performance of different types of rectified activation functions in convolutional neural network: standard rectified linear unit (ReLU), leaky rectified linear unit (Leaky ReLU), parametric rectified linear unit (PReLU) and a new randomized leaky rectified linear units (RReLU). We evalu…
Making an informed, correct and quick decision can be life-saving. It's crucial for animals during an escape behaviour or for autonomous cars during driving. The decision can be complex and may involve an assessment of the amount of threats present and the nature of each threat. Thus, we should expect early sensory pro…
Training deep neural networks with the error backpropagation algorithm is considered implausible from a biological perspective. Numerous recent publications suggest elaborate models for biologically plausible variants of deep learning, typically defining success as reaching around 98% test accuracy on the MNIST data se…
Study on feature learning in Leaky ResNets, explaining bottleneck structure.
Gradient descent biases towards stable rank networks for nearly-orthogonal data.
DCT-SNN uses DCT to reduce inference latency in SNNs.
DIET-SNN optimizes SNNs for faster, lower-energy image classification.
This paper investigates the effect of leak in spiking neural networks.
We present two Bayesian procedures to infer the interactions and external currents in an assembly of stochastic integrate-and-fire neurons from the recording of their spiking activity. The first procedure is based on the exact calculation of the most likely time courses of the neuron membrane potentials conditioned by …
This study investigates how gradient-based methods bias neural networks trained on high-dimensional data.
Learning and memory are intertwined in our brain and their relationship is at the core of several recent neural network models. In particular, the Attention-Gated MEmory Tagging model (AuGMEnT) is a reinforcement learning network with an emphasis on biological plausibility of memory dynamics and learning. We find that …
This study uses neural networks to solve interpolation problems with sparse, infinitely wide layers.
Early training of deep neural networks leads to small, directionally converging weights.
Sparsity inducing regularization is an important part for learning over-complete visual representations. Despite the popularity of regularization, in this paper, we investigate the usage of non-convex regularizations in this problem. Our contribution consists of three parts. First, we propose the leaky capped …
We study how the phenomenon of contagion can take place in the network of the world's stock exchanges due to the behavioral trait "blindeness to small changes". On large scale individual, the delay in the collective response may significantly change the dynamics of the overall system. We explicitely insert a term descr…
New method stabilizes deep neural networks by setting Lyapunov exponent to zero.
In order to classify linearly non-separable data, neurons are typically organized into multi-layer neural networks that are equipped with at least one hidden layer. Inspired by some recent discoveries in neuroscience, we propose a new model of artificial neuron along with a novel activation function enabling the learni…
Deep nets solve MDPs without high dimensions.
LQF linearizes deep models for better interpretability.
Solves financial and non-financial problems using heat potentials.
The paper aims to mathematically define and learn abstractions from data.
We empirically investigate the (negative) expected accuracy as an alternative loss function to cross entropy (negative log likelihood) for classification tasks. Coupled with softmax activation, it has small derivatives over most of its domain, and is therefore hard to optimize. A modified, leaky version is evaluated on…
An artificial neuron is modelled as a weighted summation followed by an activation function which determines its output. A wide variety of activation functions such as rectified linear units (ReLU), leaky-ReLU, Swish, MISH, etc. have been explored in the literature. In this short paper, we explore what happens when the…
Visual attention serves as a means of feature selection mechanism in the perceptual system. Motivated by Broadbent's leaky filter model of selective attention, we evaluate how such mechanism could be implemented and affect the learning process of deep reinforcement learning. We visualize and analyze the feature maps of…
We consider a deep ReLU / Leaky ReLU student network trained from the output of a fixed teacher network of the same depth, with Stochastic Gradient Descent (SGD). The student network is \emph{over-realized}: at each layer , the number of student nodes is more than that () of teacher. Under mild conditions…
New initialization schemes preserve fractional moments of weights in deep networks, improving training and test performance.
In the recent literature the important role of depth in deep learning has been emphasized. In this paper we argue that sufficient width of a feedforward network is equally important by answering the simple question under which conditions the decision regions of a neural network are connected. It turns out that for a cl…
We study the sample complexity of learning one-hidden-layer convolutional neural networks (CNNs) with non-overlapping filters. We propose a novel algorithm called approximate gradient descent for training CNNs, and show that, with high probability, the proposed algorithm with random initialization grants a linear conve…
SGD-trained neural networks generalize well even with adversarial label noise.