arXiv research

SpaRCe optimizes reservoir computing by learning neuron thresholds to improve performance and prevent forgetting.

Spiking neuronal networks are usually simulated with three main simulation schemes: the classical time-driven and event-driven schemes, and the more recent hybrid scheme. All three schemes evolve the state of a neuron through a series of checkpoints: equally spaced in the first scheme and determined neuron-wise by spik…

The mushroom body is the key network for the representation of learned olfactory stimuli in Drosophila and insects. The sparse activity of Kenyon cells, the principal neurons in the mushroom body, plays a key role in the learned classification of different odours. In the specific case of the fruit fly, the sparseness o…

A simple thresholding technique improves graph selection in neural connectivity studies.

Fault-tolerant neural networks inspired by biological error correction codes.

We improve deep threshold networks' memorization capacity exponentially.

New training method for ReLU networks achieves optimal weight size for memorization.

The paper tackles uniform sampling from neural networks efficiently.

AKOrN uses synchronized neurons to improve AI tasks.

New method trains neural networks with threshold activation functions efficiently.

Local convergence theory for mildly over-parameterized neural nets.

Estimates individualized treatment effects using shared RBF-net neurons.

Training neural networks is hard in fixed dimensions.

It has been shown recently that deep convolutional generative adversarial networks (GANs) can learn to generate music in the form of piano-rolls, which represent music by binary-valued time-pitch matrices. However, existing models can only generate real-valued piano-rolls and require further post-processing, such as ha…

New method converts conventional ANNs to SNNs with minimal loss and efficiency.

For using neural networks in safety critical domains, it is important to know if a decision made by a neural network is supported by prior similarities in training. We propose runtime neuron activation pattern monitoring - after the standard training process, one creates a monitor by feeding the training data to the ne…

DIET-SNN optimizes SNNs for faster, lower-energy image classification.

Improved SNNs for speech classification with PyTorch.

Paper introduces supervised and unsupervised TAM models for binary neurons.

Developing electrophysiological recordings of brain neuronal activity and their analysis provide a basis for exploring the structure of brain function and nervous system investigation. The recorded signals are typically a combination of spikes and noise. High amounts of background noise and possibility of electric sign…

Method reconstructs neuron models from spike times efficiently.

T-SVM improves learning in spiking neurons by maximizing dynamical margin.

Improved learning bounds for corrupted data using thresholded gradient descent.

We study the computational capacity of a model neuron, the Tempotron, which classifies sequences of spikes by linear-threshold operations. We use statistical mechanics and extreme value theory to derive the capacity of the system in random classification tasks. In contrast to its static analog, the Perceptron, the Temp…

We revisit fuzzy neural network with a cornerstone notion of generalized hamming distance, which provides a novel and theoretically justified framework to re-interpret many useful neural network techniques in terms of fuzzy logic. In particular, we conjecture and empirically illustrate that, the celebrated batch normal…

Understanding the inductive bias of neural networks is critical to explaining their ability to generalise. Here, for one of the simplest neural networks -- a single-layer perceptron with n input neurons, one output neuron, and no threshold bias term -- we prove that upon random initialisation of weights, the a priori p…

For neural networks (NNs) with rectified linear unit (ReLU) or binary activation functions, we show that their training can be accomplished in a reduced parameter space. Specifically, the weights in each neuron can be trained on the unit sphere, as opposed to the entire space, and the threshold can be trained in a boun…

Study neural networks learning from noisy examples via reverberation.

Neuron Shapley identifies key neurons in deep networks, improving model accuracy and fairness.

Two-layer ReLU networks often converge to simpler solutions, improving generalization.

Describes explaining neurons in deep representations using compositional logical concepts.

Paper proposes DCT for efficient hybrid parallel training of large recommendation models.

SeReNe prunes neurons with low sensitivity to reduce network size.

Under-parameterized networks can either copy or average teacher weights, leading to universal optimal solutions.

Over the past decade, deep neural networks (DNNs) have demonstrated remarkable performance in a variety of applications. As we try to solve more advanced problems, increasing demands for computing and power resources has become inevitable. Spiking neural networks (SNNs) have attracted widespread interest as the third-g…

This research investigates selectively pruning hyper and hypo neurons to improve neural network generalization.

Modeling hidden neurons in SNNs using mesoscopic approximations.

Topological methods improve neuron analysis and tracer injection summary.

Inspired by complexity and diversity of biological neurons, our group proposed quadratic neurons by replacing the inner product in current artificial neurons with a quadratic operation on input data, thereby enhancing the capability of an individual neuron. Along this direction, we are motivated to evaluate the power o…

We propose a new generic type of stochastic neurons, called $q$-neurons, that considers activation functions based on Jackson's $q$-derivatives with stochastic parameters $q$. Our generalization of neural network architectures with $q$-neurons is shown to be both scalable and very easy to implement. We demonstrate expe…

Approximate Bayesian Computation (ABC) provides methods for Bayesian inference in simulation-based stochastic models which do not permit tractable likelihoods. We present a new ABC method which uses probabilistic neural emulator networks to learn synthetic likelihoods on simulated data -- both local emulators which app…

It will be shown that according to theorems of K. Menger, every neuron grid if identified with a curve is able to preserve the adopted qualitative structure of a data space. Furthermore, if this identification is made, the neuron grid structure can always be mapped to a subset of a universal neuron grid which is constr…

Neural networks learn to mimic brain neurons with two-input activation functions, improving performance and robustness.

CHANI learns classification tasks with local transformations inspired by biology.

Researchers develop methods to learn neuron dynamics from colored noise.

Despite our extensive knowledge of biophysical properties of neurons, there is no commonly accepted algorithmic theory of neuronal function. Here we explore the hypothesis that single-layer neuronal networks perform online symmetric nonnegative matrix factorization (SNMF) of the similarity matrix of the streamed data. …

Single neuron with ADA learns XOR and outperforms other functions.

Rectified Linear Units (ReLU) have become the main model for the neural units in current deep learning systems. This choice has been originally suggested as a way to compensate for the so called vanishing gradient problem which can undercut stochastic gradient descent (SGD) learning in networks composed of multiple lay…