BinaryGAN uses binary neurons for GAN training.
problem Training GANs with binary outputs.
method End-to-end backpropagation with sigmoid-adjusted straight-through estimators.
result BinaryGAN can generate binary-valued predictions.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “Binary Neurons”
Single neurons can perform as well as dense networks in binary and multi-class recognition tasks.
problem Designing efficient neural networks for recognition tasks.
method Investigated the use of single or multiple neurons in neural networks for binary and multi-class recognition tasks.
result Sparse networks can be as efficient as dense networks in both binary and multi-class tasks.
Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generationcs.LG
It has been shown recently that deep convolutional generative adversarial networks (GANs) can learn to generate music in the form of piano-rolls, which represent music by binary-valued time-pitch matrices. However, existing models can only generate real-valued piano-rolls and require further post-processing, such as ha…
Model detects patterns in noisy binary data, explaining neuron activity in terms of cell assemblies.
problem Detecting structure in noisy or approximate repeats of patterns in sparse binary data.
method Probabilistic binary latent variable model based on Noisy-OR model, inferring sparse activity in latent variables.
result Model successfully extracts and explains latent structure in spiking neural data.
Continuous Dropoutcs.CV
Continuous dropout mimics brain neuron firing rates to prevent feature detector co-adaptation.
problem Preventing overfitting in deep neural networks.
method Extending binary dropout to continuous dropout, inspired by brain neuron firing rates.
result Continuous dropout improves test performance by preventing feature detector co-adaptation.
Model neural plasticity as binary optimization to dynamically activate or deactivate network units.
problem Dynamic learning and adaptability of neural networks.
method Model neural plasticity as an -norm regularized binary optimization problem, where units can be activated or deactivated based on a cost-benefit tradeoff.
result Demonstrates that a single parameter can modulate learning dynamics, unifying network sparsification and expansion.
BSF algorithm reduces neural network size and selects features efficiently.
problem Neural network size and feature selection optimization.
method Binary Stochastic Filtering (BSF) layer that penalizes information, stochastically passes or drops features.
result Multifold decrease in neural network size and optimal feature selection.
One of the main difficulties in analyzing neural networks is the non-convexity of the loss function which may have many bad local minima. In this paper, we study the landscape of neural networks for binary classification tasks. Under mild assumptions, we prove that after adding one special neuron with a skip connection…
Deep CNN model predicts neuronal cell health from images.
problem Predicting the biological activity of chemical compounds on neuronal cells.
method Deep convolutional neural network (CNN) with residual connections.
result Achieved 99.6% accuracy in distinguishing treated from untreated cells.
Paper introduces supervised and unsupervised TAM models for binary neurons.
problem Learning and retrieval of structured triplets of patterns in neural networks.
method Extends Hebbian paradigm to supervised and unsupervised protocols, using glassy statistical mechanical techniques.
result Obtained self-consistency equations for critical dataset sizes and retrieval performance.
A spiking neural network model for probabilistic inference of binary Markov random fields.
problem Implementing probabilistic inference in spiking neural networks.
method Designing a spiking recurrent neural network and proving its equivalence to mean-field inference of binary Markov random fields.
result The spiking neural network model can implement inference of arbitrary binary Markov random fields.
New training method for ReLU networks achieves optimal weight size for memorization.
problem Approximate memorization of arbitrary real labels with neural networks.
method Complex recombination training procedure for ReLU networks.
result Approximate memorization with nearly optimal weight size and neuron count.
We formulate learning of a binary autoencoder as a biconvex optimization problem which learns from the pairwise correlations between encoded and decoded bits. Among all possible algorithms that use this information, ours finds the autoencoder that reconstructs its inputs with worst-case optimal loss. The optimal decode…
The paper tackles uniform sampling from neural networks efficiently.
problem Uniform sampling from neural network labelings given a sample.
method Polynomial-time algorithm for general neural networks, random walk for single neuron.
result Uniform sampling with high probability for general neural networks, exact uniform sampling for single neuron.
This paper proves adding neurons eliminates all bad local minima in deep learning.
problem Eliminating all suboptimal local minima in deep learning models.
method Adding one special neuron per output unit eliminates all suboptimal local minima of deep neural networks.
result At every local minimum, the original neural network parameters are a global minimum.
Entropy from Machine Learningcond-mat.stat-mech
Machine learning for entropy calculation from binary signals.
problem Calculating entropy from binary configurations/signals.
method Transformed entropy calculation into supervised classification tasks using machine learning.
result Reproduced entropy and free energy of the 2D Ising model.
Binary representation is desirable for its memory efficiency, computation speed and robustness. In this paper, we propose adjustable bounded rectifiers to learn binary representations for deep neural networks. While hard constraining representations across layers to be binary makes training unreasonably difficult, we s…
Information in neural networks is represented as weighted connections, or synapses, between neurons. This poses a problem as the primary computational bottleneck for neural networks is the vector-matrix multiply when inputs are multiplied by the neural network weights. Conventional processing architectures are not well…
Binary autoencoder with sparse hidden layer preserves information and zero reconstruction error.
problem Preserving information and zero reconstruction error in binary neural networks.
method Binary autoencoder with random binary weights, sparse hidden layer, and varying neuron thresholds.
result Zero reconstruction error for any input with a large hidden layer and varying neuron thresholds.
We improve deep threshold networks' memorization capacity exponentially.
problem Memorizing datasets with randomized labels using deep neural networks.
method Using Gaussian random weights in the first layer and binary or integer weights in subsequent layers, we prove a new dependence on minimum distance.
result We show that neurons and weights are sufficient.
Modeling and estimating dynamic graphs from binary pattern sequences.
problem Extracting dominant correlation structures from time-dependent binary patterns.
method State-space model of an Ising-type network composed of multiple undirected graphs, sequential Bayes algorithm.
result The method outperforms traditional methods in uncovering overlapping graphs and estimating dynamics of weights.
Structured pruning method reduces RNN sizes and speeds up inference.
problem Large RNN models are hard to deploy on edge devices.
method Structured pruning through neuron selection, minimizing L0 norm of weight matrix.
result Nearly 20x speedup achieved without performance loss.
Unified approach to multiclass classification using Gabriel graphs.
problem Improving multiclass classification accuracy and efficiency.
method Integrates Gabriel graphs for binary and multiclass classification, proposing new activation functions and support edge neurons.
result Experimental results show superior performance compared to previous GG-based classifiers.
Alternative to convolutions using decision trees for neural networks.
problem Replacing complex convolutions with simpler decision-based layers.
method Binary decisions as indices to conditional distributions, trained using backpropagation.
result Performance similar to conventional neural networks, with runtime improvements.
Dual neural network architecture improves accuracy and interpretability.
problem Improving neural network interpretability and accuracy.
method Stacked recurrent and feedforward layers, binary activation function.
result Binary activation leads to simpler, more interpretable models with higher accuracy.
A new method sparsifies neural networks using stochastic binary optimization.
problem Sparsifying neural networks to reduce computational cost and improve efficiency.
method Stochastic binary optimization with the Augment-Reinforce-Merge (ARM) estimator.
result ARM enables efficient network sparsification with comparable accuracy to baseline methods.
Modified Metropolis algorithm ensures convergence for multivariate binary distributions with fixed-order updates.
problem Infeasibility of standard Metropolis algorithm for multivariate binary distributions with fixed-order updates.
method Proposed a modified Metropolis transition operator ensuring irreducibility and convergence.
result Ensures convergence to the limiting distribution in multivariate binary case with fixed-order updates.
The mesoscopic organization of complex systems, from financial markets to the brain, is an intermediate between the microscopic dynamics of individual units (stocks or neurons, in the mentioned cases), and the macroscopic dynamics of the system as a whole. The organization is determined by "communities" of units whose …
Significant success has been reported recently using deep neural networks for classification. Such large networks can be computationally intensive, even after training is over. Implementing these trained networks in hardware chips with a limited precision of synaptic weights may improve their speed and energy efficienc…
There is a pressing need to build an architecture that could subsume these networks under a unified framework that achieves both higher performance and less overhead. To this end, two fundamental issues are yet to be addressed. The first one is how to implement the back propagation when neuronal activations are discret…
Convexifying Sparse Interpolation with Infinitely Wide Neural Networks: An Atomic Norm Approachcs.LG
This study uses neural networks to solve interpolation problems with sparse, infinitely wide layers.
problem Exact data interpolation using sparse, infinitely wide neural networks.
method Atomic norm framework to derive convex hulls and equivalent convex formulations.
result Simple characterizations of convex hulls for different constraints on network weights and biases.
SNNs as first-to-spike policies improve energy efficiency in RL control.
problem Reducing energy consumption in RL control systems.
method Policy gradient approach with GLM for spiking neurons.
result Online trained SNNs outperform ANN conversion in energy efficiency and control performance.
It is widely conjectured that the reason that training algorithms for neural networks are successful because all local minima lead to similar performance, for example, see (LeCun et al., 2015, Choromanska et al., 2015, Dauphin et al., 2014). Performance is typically measured in terms of two metrics: training performanc…
Efficiently estimates neural connectivity using ensemble stimulation.
problem Estimating functional connectivity in large, behaving neural populations.
method Noisy group testing combined with convex optimization and Bayesian inference.
result Connectivity can be inferred with logarithmic growth in tests, even for large networks.
PoET-BiN reduces power consumption in neural networks on embedded devices.
problem Power inefficiency in neural network implementations on embedded platforms.
method Look-Up Table based implementation with a modified Decision Tree approach.
result Near state-of-the-art results with up to 6 orders of magnitude energy reduction.
Gradient flow on ReLU networks converges to a simple model with few regions.
problem Understanding the dynamics of gradient flow in shallow ReLU networks.
method Analysis of gradient flow dynamics on univariate ReLU neural networks.
result Gradient flow converges to a network with at most O(r) linear regions.
We propose a novel online learning algorithm for Restricted Boltzmann Machines (RBM), namely, the Online Generative Discriminative Restricted Boltzmann Machine (OGD-RBM), that provides the ability to build and adapt the network architecture of RBM according to the statistics of streaming data. The OGD-RBM is trained in…
DNPUs improve neural network performance with high-capacity nanoelectronic nodes.
problem Limited performance of single DNPUs in solving complex classification problems.
method Developed DNPUs as high-capacity neurons and implemented multi-DNPU networks.
result Feed-forward DNPU networks improve single DNPU performance from 77% to 94% test accuracy.
We propose Very Simple Classifier (VSC) a novel method designed to incorporate the concepts of subsampling and locality in the definition of features to be used as the input of a perceptron. The rationale is that locality theoretically guarantees a bound on the generalization error. Each feature in VSC is a max-margin …
For neural networks (NNs) with rectified linear unit (ReLU) or binary activation functions, we show that their training can be accomplished in a reduced parameter space. Specifically, the weights in each neuron can be trained on the unit sphere, as opposed to the entire space, and the threshold can be trained in a boun…
New method uses surrogate gradients to train efficient spiking networks on neuromorphic hardware.
problem Training high-performing spiking networks on analog neuromorphic hardware is challenging due to device mismatch and lack of efficient algorithms.
method Introduces a general in-the-loop learning framework based on surrogate gradients.
result Learning self-corrects for device mismatch, resulting in competitive spiking network performance.
Neuron Shapley identifies key neurons in deep networks, improving model accuracy and fairness.
problem Identifying responsible neurons in deep networks for better model performance and fairness.
method Neuron Shapley framework quantifies neuron contributions, accounting for interactions.
result Removing just 30 critical filters can destroy model accuracy, revealing network function.
Describes explaining neurons in deep representations using compositional logical concepts.
problem Interpreting neuron behavior in deep neural networks.
method Identifying compositional logical concepts that closely approximate neuron behavior.
result Compositional explanations provide insights into model performance and allow for adversarial example creation.
SeReNe prunes neurons with low sensitivity to reduce network size.
problem Large neural networks consume too many resources on resource-constrained devices.
method Exploits neural sensitivity as a regularizer to prune neurons with low sensitivity.
result Pruning neurons with low sensitivity achieves competitive compression ratios.
Under-parameterized networks can either copy or average teacher weights, leading to universal optimal solutions.
problem Approximating a teacher network with an under-parameterized student network.
method Analyzing shallow neural networks with erf activation function and unitary teacher weights, proving copy-average configurations are critical points and finding the optimal solution.
result The optimal solution for under-parameterized networks has a universal structure, whether copying or averaging teacher neurons.
This research investigates selectively pruning hyper and hypo neurons to improve neural network generalization.
problem Improving neural network generalization to unseen data.
method Investigates pruning hyper and hypo neurons selectively in fully connected layers of CNNs.
result Selective pruning of hyper and hypo neurons improves model performance on out-of-domain data.
Modeling hidden neurons in SNNs using mesoscopic approximations.
problem Underconstrained problem of modeling unobserved neurons in SNNs.
method Coarse-graining and mean-field approximations to derive neuLVM.
result neuLVM can efficiently model large SNNs and recover connectivity parameters.
Detection and skeletonization of single neurons and tracer injections using topological methodscs.CV
Topological methods improve neuron analysis and tracer injection summary.
problem Traditional methods fail to capture the tree-like structure of neurons.
method Discrete Morse (DM) Theory for neuron skeletonization and consensus tree summarization.
result Significant performance improvements over non-topological methods.