This work ranks CNN filters based on their importance.
problem Unclear role of CNN neurons in producing output.
method Two methods: Shapley value game theory and Importance switch variational inference.
result Filters with higher importance are more crucial for output.
New method estimates neuronal connectivity from partially observed data.
problem Estimating neuronal connectivity from partially observed data.
method Two-step approach: low-rank covariance completion followed by graph structure estimation.
result Graph selection consistency demonstrated for one approach.
Elite ONNs learn better with synaptic plasticity, improving performance over CNNs.
problem Limited heterogeneity in ONNs due to fixed operator sets.
method Synaptic plasticity-based search for optimal operator sets.
result Elite ONNs achieve superior learning performance compared to conventional methods.
Partial soft-matching distance improves neural representation comparison by allowing some neurons to remain unmatched.
problem Neural representations are noisy and contain outliers, making traditional matching methods unreliable.
method Extends soft-matching distance to a partial optimal transport setting, allowing some neurons to remain unmatched.
result Partial soft-matching provides robust correspondences that are more reliable under noise and outliers.
Study recovers neuron assemblies from cognitive data using tensor decomposition.
problem Recovering neuron assemblies from cognitive data.
method Linear independence analysis of rank one tensors, polynomial measurements.
result Reconstructs Venn diagram of neuron assemblies from intersection sizes.
Single neuron learns predictive uncertainty in deep learning models.
problem Uncertainty estimation in deep learning models, especially with respect to model specification and training procedure.
method Introduces a non-parametric quantile estimation method using a single neuron.
result The method achieves competitive predictive uncertainty quantification quality and coverage compared to state-of-the-art solutions.
Biological neural network mimics CCA for multi-channel data.
problem Implementing CCA in a biologically plausible neural network.
method Derive an online CCA algorithm with local synaptic updates for multi-compartmental neurons.
result The derived neural network architecture and synaptic updates resemble cortical pyramidal neuron behavior.
Active learning method for neural population dynamics using optogenetics.
problem Efficiently selecting neurons to stimulate for identifying neural population dynamics.
method Developed active learning procedure for low-rank regression to determine informative photostimulation patterns.
result Demonstrated a two-fold reduction in data required for predictive power using low-rank linear dynamical systems model.
HOPE uses Hilbert space to deconstruct deep network representations.
problem Deconstructing learned representations in deep networks is challenging.
method Introduces Hilbert Operator for Progressive Encoding (HOPE) to deconstruct network weights.
result HOPE provides an unbiased approach to network compression and fine-tuning.
Optimization geometry affects deep learning performance.
problem The impact of optimization geometry on deep learning performance.
method Analysis of pseudogradient methods for learning generalized linear models.
result Non-asymptotic bounds on generalization error characterize model performance.
Low-rank structure emerges in neural networks during learning.
problem Understanding the evolution of synaptic connectivity over learning.
method Investigated the rank of 3-tensor formed by weight matrices throughout learning.
result Inferred weights are low-tensor-rank and evolve in a fixed low-dimensional subspace.
We propose a method to infer stochastic low-rank RNNs from neural data.
problem Fitting low-rank RNNs to noisy, stochastic neural data.
method Variational sequential Monte Carlo methods for stochastic low-rank RNNs.
result Lower dimensional latent dynamics compared to state-of-the-art methods.
Neuron Shapley identifies key neurons in deep networks, improving model accuracy and fairness.
problem Identifying responsible neurons in deep networks for better model performance and fairness.
method Neuron Shapley framework quantifies neuron contributions, accounting for interactions.
result Removing just 30 critical filters can destroy model accuracy, revealing network function.
Describes explaining neurons in deep representations using compositional logical concepts.
problem Interpreting neuron behavior in deep neural networks.
method Identifying compositional logical concepts that closely approximate neuron behavior.
result Compositional explanations provide insights into model performance and allow for adversarial example creation.
SeReNe prunes neurons with low sensitivity to reduce network size.
problem Large neural networks consume too many resources on resource-constrained devices.
method Exploits neural sensitivity as a regularizer to prune neurons with low sensitivity.
result Pruning neurons with low sensitivity achieves competitive compression ratios.
Under-parameterized networks can either copy or average teacher weights, leading to universal optimal solutions.
problem Approximating a teacher network with an under-parameterized student network.
method Analyzing shallow neural networks with erf activation function and unitary teacher weights, proving copy-average configurations are critical points and finding the optimal solution.
result The optimal solution for under-parameterized networks has a universal structure, whether copying or averaging teacher neurons.
This research investigates selectively pruning hyper and hypo neurons to improve neural network generalization.
problem Improving neural network generalization to unseen data.
method Investigates pruning hyper and hypo neurons selectively in fully connected layers of CNNs.
result Selective pruning of hyper and hypo neurons improves model performance on out-of-domain data.
Modeling hidden neurons in SNNs using mesoscopic approximations.
problem Underconstrained problem of modeling unobserved neurons in SNNs.
method Coarse-graining and mean-field approximations to derive neuLVM.
result neuLVM can efficiently model large SNNs and recover connectivity parameters.
Topological methods improve neuron analysis and tracer injection summary.
problem Traditional methods fail to capture the tree-like structure of neurons.
method Discrete Morse (DM) Theory for neuron skeletonization and consensus tree summarization.
result Significant performance improvements over non-topological methods.
We propose a new generic type of stochastic neurons, called q-neurons, that considers activation functions based on Jackson's q-derivatives with stochastic parameters q. Our generalization of neural network architectures with q-neurons is shown to be both scalable and very easy to implement. We demonstrate expe…
Quadratic autoencoder improves low-dose CT image denoising.
problem Low-dose CT image denoising.
method Quadratic autoencoder architecture applied to CT denoising.
result Quadratic autoencoder achieves superior denoising performance and efficiency.
BEAN models neuronal correlations to create interpretable representations.
problem Hard interpretation of dense-layer representations in DNNs.
method Inspired by neuroscience, BEAN models neuronal correlations and dependencies.
result BEAN enables formation of interpretable neuronal clusters without sacrificing model performance.
SpaRCe optimizes reservoir computing by learning neuron thresholds to improve performance and prevent forgetting.
problem Improving performance and preventing forgetting in reservoir computing networks.
method Integrates neuron-specific learnable thresholds to optimize sparsity without altering dynamics, learning read-out weights and thresholds via gradient rule.
result Threshold learning improves performance and alleviates catastrophic forgetting.
It will be shown that according to theorems of K. Menger, every neuron grid if identified with a curve is able to preserve the adopted qualitative structure of a data space. Furthermore, if this identification is made, the neuron grid structure can always be mapped to a subset of a universal neuron grid which is constr…
This work improves DNN interpretability by reducing neuron ambiguity.
problem Lack of interpretability in DNNs, especially in healthcare applications.
method Developed a metric to evaluate neuron consistency, used adversarial examples to identify ambiguous features, and proposed adversarial training to improve consistency.
result Reduced ambiguity of neurons in DNNs, improving interpretability.
A new method to understand neural networks by sampling the 'inverse set' of a neuron.
problem Understanding the internal representation of neurons in neural networks.
method Optimization-based sampling approach to characterize the input space that excites a neuron.
result Inspection of samples reveals regularities that help understand the neuron's representation.
Neural networks learn to mimic brain neurons with two-input activation functions, improving performance and robustness.
problem Training neural networks to mimic the complex interactions of brain neurons.
method Developed a network-in-network architecture with two-input activation functions, optimized hyperparameters, and compared to conventional ReLU networks.
result Two-input activation functions can learn soft XOR functions, improving network performance and robustness.
New methods estimate brain connectivity from calcium imaging data with missing data.
problem Estimating functional neuronal connectivity from calcium imaging data with missing data.
method Two approaches for nonparanormal Graph Quilting based on the Gaussian copula graphical model.
result Our methods yield more meaningful functional connectivity estimates than existing Gaussian graph quilting methods.
CHANI learns classification tasks with local transformations inspired by biology.
problem Proving neural networks can learn classification tasks with local transformations.
method CHANI uses spiking neurons modeled by Hawkes processes with expert aggregation for local learning.
result CHANI can learn and encode multiple classes, forming assemblies of neurons.
BinaryGAN uses binary neurons for GAN training.
problem Training GANs with binary outputs.
method End-to-end backpropagation with sigmoid-adjusted straight-through estimators.
result BinaryGAN can generate binary-valued predictions.
Researchers develop methods to learn neuron dynamics from colored noise.
problem Learning nonlocal stochastic neuron dynamics from colored noise.
method Proposed two methods for closing Fokker-Planck equations: nonlocal large-eddy-diffusivity closure and data-driven sparse regression.
result Mutual information and total correlation between stimulus and neuron states calculated for FHN neuron.
LoRA fine-tuning explained with gradient dynamics for low-rank perturbations.
problem Understanding why gradient descent converges to useful low-rank perturbations in LoRA fine-tuning.
method Generalized student-teacher setting with i.i.d. samples and online gradient descent.
result Gradient descent converges to the teacher model in dkO(1) iterations under certain conditions. Single neuron with ADA learns XOR and outperforms other functions.
problem Classifying linearly non-separable data.
method Proposed a new artificial neuron with apical dendrite activation.
result ADA function achieves 100% accuracy on XOR and superior performance on benchmark datasets.
Despite our extensive knowledge of biophysical properties of neurons, there is no commonly accepted algorithmic theory of neuronal function. Here we explore the hypothesis that single-layer neuronal networks perform online symmetric nonnegative matrix factorization (SNMF) of the similarity matrix of the streamed data. …
Stable unactivated neurons reduce expressiveness in ReLU networks.
problem Reducing expressiveness in ReLU neural networks due to stably unactivated neurons.
method Investigated the probability of neurons being stably unactivated in ReLU networks with symmetric weight and bias distributions.
result Proved the probability of a neuron being stably unactivated in the second hidden layer of a ReLU network.
Novel chaotic neurons improve AI with minimal training data.
problem Limited training data for AI algorithms.
method Intrinsically chaotic neurons inspired by chaos theory.
result Classification accuracy up to 95.8% with just 2 training samples per class.
Single-spike neurons can approximate as well as multi-spike neurons.
problem Limitation of single-spike neurons in spiking neural networks.
method Comparison of single-spike and multi-spike neural networks.
result Single-spike and multi-spike neural networks are equivalent in approximation capabilities.
The challenge of assigning importance to individual neurons in a network is of interest when interpreting deep learning models. In recent work, Dhamdhere et al. proposed Total Conductance, a "natural refinement of Integrated Gradients" for attributing importance to internal neurons. Unfortunately, the authors found tha…
Optimal neuron activation functions improve neural network performance.
problem Limited expressive power of standard neuron activation functions in neural networks.
method Additive Gaussian process regression to construct individual neuron activation functions.
result Optimal neuron activation functions lead to better performance and reduced overfitting.
The artificial neural network is a popular framework in machine learning. To empower individual neurons, we recently suggested that the current type of neurons could be upgraded to 2nd order counterparts, in which the linear operation between inputs to a neuron and the associated weights is replaced with a nonlinear qu…
Novel 'strong neuron' improves deep learning efficiency and robustness.
problem Improving deep learning efficiency and robustness against adversarial attacks.
method Introducing a novel 'strong neuron' model and a constructive training algorithm.
result Achieved 10x-100x reduction in operations count and hardware requirements.
Deep CNN model predicts neuronal cell health from images.
problem Predicting the biological activity of chemical compounds on neuronal cells.
method Deep convolutional neural network (CNN) with residual connections.
result Achieved 99.6% accuracy in distinguishing treated from untreated cells.
New artificial neuron doubles weight for improved deep learning accuracy.
problem Improving deep learning models' accuracy.
method Introducing a double-weight neuron, tested on MNIST and CIFAR-10 datasets.
result Significant improvement in classification accuracy for MNIST and CIFAR-10 datasets.
We view a neural network as a distributed system of which neurons can fail independently, and we evaluate its robustness in the absence of any (recovery) learning phase. We give tight bounds on the number of neurons that can fail without harming the result of a computation. To determine our bounds, we leverage the fact…
Single ReLU neuron's gradient dynamics reveal support vectors as key to generalization.
problem Understanding the generalization capability of ReLU networks.
method Examined gradient flow dynamics and support vectors in single ReLU neuron training.
result Support vectors play a crucial role in the generalization of ReLU networks.
This paper presents a new artificial neuron model capable of learning its receptive field in the topological domain of inputs. The model provides adaptive and differentiable local connectivity (plasticity) applicable to any domain. It requires no other tool than the backpropagation algorithm to learn its parameters whi…
Novel method decorrelates neurons for better deep learning model generalization.
problem High correlations between neurons limit deep learning model generalization.
method Regularization terms from minimum spanning tree of neuron cliques, using correlation dissimilarities.
result Our regularizers outperform existing methods and minimize neuron redundancies.
A neuron is a basic physiological and computational unit of the brain. While much is known about the physiological properties of a neuron, its computational role is poorly understood. Here we propose to view a neuron as a signal processing device that represents the incoming streaming data matrix as a sparse vector of …