This work improves DNN interpretability by reducing neuron ambiguity.
problem Lack of interpretability in DNNs, especially in healthcare applications.
method Developed a metric to evaluate neuron consistency, used adversarial examples to identify ambiguous features, and proposed adversarial training to improve consistency.
result Reduced ambiguity of neurons in DNNs, improving interpretability.
We propose a new generic type of stochastic neurons, called q-neurons, that considers activation functions based on Jackson's q-derivatives with stochastic parameters q. Our generalization of neural network architectures with q-neurons is shown to be both scalable and very easy to implement. We demonstrate expe…
A method uses autoencoders to align multi-modal neuron data.
problem Inconsistent cell type definitions across different data modalities.
method Coupled training of autoencoders for cross-modal alignment.
result Representations learned by coupled autoencoders can identify single-modality sampled cell types.
The artificial neural network is a popular framework in machine learning. To empower individual neurons, we recently suggested that the current type of neurons could be upgraded to 2nd order counterparts, in which the linear operation between inputs to a neuron and the associated weights is replaced with a nonlinear qu…
Neural networks learn to mimic brain neurons with two-input activation functions, improving performance and robustness.
problem Training neural networks to mimic the complex interactions of brain neurons.
method Developed a network-in-network architecture with two-input activation functions, optimized hyperparameters, and compared to conventional ReLU networks.
result Two-input activation functions can learn soft XOR functions, improving network performance and robustness.
Spatially positioned neurons in neural networks mimic biological systems.
problem Creating neural networks that can perform multiple tasks efficiently.
method Added spatial positions and proximity penalties to artificial neurons.
result Neurons naturally cluster, each responsible for a specific task.
This research investigates selectively pruning hyper and hypo neurons to improve neural network generalization.
problem Improving neural network generalization to unseen data.
method Investigates pruning hyper and hypo neurons selectively in fully connected layers of CNNs.
result Selective pruning of hyper and hypo neurons improves model performance on out-of-domain data.
New artificial neuron doubles weight for improved deep learning accuracy.
problem Improving deep learning models' accuracy.
method Introducing a double-weight neuron, tested on MNIST and CIFAR-10 datasets.
result Significant improvement in classification accuracy for MNIST and CIFAR-10 datasets.
Method selects task-specific neurons for unsupervised transfer learning.
problem Challenges in fine-tuning large language models for specific tasks.
method Selects important neurons for a specific classification task and extends to multi-source transfer learning.
result Higher similarity between task-specific fingerprints leads to better transferability.
Paper introduces supervised and unsupervised TAM models for binary neurons.
problem Learning and retrieval of structured triplets of patterns in neural networks.
method Extends Hebbian paradigm to supervised and unsupervised protocols, using glassy statistical mechanical techniques.
result Obtained self-consistency equations for critical dataset sizes and retrieval performance.
Method infers dynamics from incomplete time series data.
problem Challenges in inferring stochastic dynamics from time series with missing data.
method Expectation Maximization (EM) algorithm that iterates between E-step and M-step.
result The EM algorithm effectively recovers missing data points and infers underlying network models from real neuronal activities.
Estimates neuronal connectivity from spike times using flexible Hawkes processes.
problem Learning latent network structure from multivariate point process data.
method Proposes a new nonstationary Hawkes process and uses sparse least squares estimation.
result Establishes non-asymptotic error bounds and selection consistency for estimated parameters.
A dynamical neural network consists of a set of interconnected neurons that interact over time continuously. It can exhibit computational properties in the sense that the dynamical system's evolution and/or limit points in the associated state space can correspond to numerical solutions to certain mathematical optimiza…
The paper introduces walks with jumps for modeling neuron activity in hyperbolic space.
problem Encoding neuron activity sequences in hyperbolic space.
method Introducing walks with jumps in hyperbolic geometry to model neuron activity.
result Endpoints of walks with jumps do not fully encode the sequence of jump times.
Neural networks are based on a simplified model of the brain. In this project, we wanted to relax the simplifying assumptions of a traditional neural network by making a model that more closely emulates the low level interactions of neurons. Like in an RNN, our model has a state that persists between time steps, so tha…
This paper compares spike sorting techniques for rat brain neuronal activity.
problem Improving the accuracy of spike sorting for neuronal activity analysis.
method Three-step spike sorting process: detection, feature extraction, and clustering. Various methods are compared.
result Kernel PCA outperforms in feature extraction, leading to better spike sorting results.
OPNP prunes parameters and neurons to improve OOD detection without training.
problem Detecting out-of-distribution samples in real-world machine learning models.
method OPNP approach that identifies and removes sensitive parameters and neurons.
result OPNP consistently outperforms existing methods on multiple OOD detection tasks.
While variational dropout approaches have been shown to be effective for network sparsification, they are still suboptimal in the sense that they set the dropout rate for each neuron without consideration of the input data. With such input-independent dropout, each neuron is evolved to be generic across inputs, which m…
Efficiently verifies neural networks by handling neuron splits, improving speed and accuracy.
problem Handling neuron split constraints in incomplete neural network verification.
method β-CROWN, which optimizes parameters β to encode neuron splits and uses them in bound propagation.
result β-CROWN significantly speeds up verification while maintaining high accuracy.
T-SVM improves learning in spiking neurons by maximizing dynamical margin.
problem Finding robust solutions in spiking neuronal networks with temporal correlations.
method Introduces Temporal Support Vector Machine (T-SVM) to maximize dynamical margin.
result T-SVM enables learning of tasks requiring nonlinear spatial integration.
Sequential learning, also called lifelong learning, studies the problem of learning tasks in a sequence with access restricted to only the data of the current task. In this paper we look at a scenario with fixed model capacity, and postulate that the learning process should not be selfish, i.e. it should account for fu…
A new geometric perceptron model improves 3D shape classification.
problem Challenges in geometric tasks involving point clouds using machine learning.
method Introduces multilayer geometric perceptron (MLGP) with geometric neurons.
result MLGP outperforms vanilla MLP in 3D shape classification and noise resistance.
New neural network design resists small ℓ∞-norm adversarial perturbations.
problem Vulnerability of neural networks to small ℓ∞-norm adversarial perturbations. method Designing ℓ∞-dist neurons and constructing ℓ∞-dist nets, proving their 1-Lipschitz property and expressive power. result Certified robustness of ℓ∞-dist nets with state-of-the-art performance on various datasets. Neuron Shapley identifies key neurons in deep networks, improving model accuracy and fairness.
problem Identifying responsible neurons in deep networks for better model performance and fairness.
method Neuron Shapley framework quantifies neuron contributions, accounting for interactions.
result Removing just 30 critical filters can destroy model accuracy, revealing network function.
Deep neural networks (DNNs) are vulnerable to maliciously generated adversarial examples. These examples are intentionally designed by making imperceptible perturbations and often mislead a DNN into making an incorrect prediction. This phenomenon means that there is significant risk in applying DNNs to safety-critical …
Describes explaining neurons in deep representations using compositional logical concepts.
problem Interpreting neuron behavior in deep neural networks.
method Identifying compositional logical concepts that closely approximate neuron behavior.
result Compositional explanations provide insights into model performance and allow for adversarial example creation.
SeReNe prunes neurons with low sensitivity to reduce network size.
problem Large neural networks consume too many resources on resource-constrained devices.
method Exploits neural sensitivity as a regularizer to prune neurons with low sensitivity.
result Pruning neurons with low sensitivity achieves competitive compression ratios.
Under-parameterized networks can either copy or average teacher weights, leading to universal optimal solutions.
problem Approximating a teacher network with an under-parameterized student network.
method Analyzing shallow neural networks with erf activation function and unitary teacher weights, proving copy-average configurations are critical points and finding the optimal solution.
result The optimal solution for under-parameterized networks has a universal structure, whether copying or averaging teacher neurons.
Modeling hidden neurons in SNNs using mesoscopic approximations.
problem Underconstrained problem of modeling unobserved neurons in SNNs.
method Coarse-graining and mean-field approximations to derive neuLVM.
result neuLVM can efficiently model large SNNs and recover connectivity parameters.
Neural networks learn task-specific features, influenced by nonlinearity.
problem Understanding the nature of task-dependent feature learning in neural networks.
method Investigation of fully-connected, wide neural networks using Bayesian framework.
result The nature of internal representations depends on neuronal nonlinearity, leading to analog, redundant, or sparse coding schemes.
Topological methods improve neuron analysis and tracer injection summary.
problem Traditional methods fail to capture the tree-like structure of neurons.
method Discrete Morse (DM) Theory for neuron skeletonization and consensus tree summarization.
result Significant performance improvements over non-topological methods.
Optimized neural network approximates high-dimensional functions with minimal parameters.
problem Achieving optimal approximation of high-dimensional continuous functions with minimal parameters.
method Developed a neural network with a specific activation function and architecture to achieve super approximation property.
result A composed network with at most 10889d + 10887 nonzero parameters achieves super approximation property, suggesting optimality in parameter growth.
Develops a new neural spike train decoding framework using topological data.
problem Decoding neural spike trains from head direction and grid cells.
method Combines simplicial complex discovery with deep learning to capture higher-order connectivity.
result Demonstrates effectiveness on head direction and trajectory prediction datasets.
Quadratic autoencoder improves low-dose CT image denoising.
problem Low-dose CT image denoising.
method Quadratic autoencoder architecture applied to CT denoising.
result Quadratic autoencoder achieves superior denoising performance and efficiency.
BEAN models neuronal correlations to create interpretable representations.
problem Hard interpretation of dense-layer representations in DNNs.
method Inspired by neuroscience, BEAN models neuronal correlations and dependencies.
result BEAN enables formation of interpretable neuronal clusters without sacrificing model performance.
SpaRCe optimizes reservoir computing by learning neuron thresholds to improve performance and prevent forgetting.
problem Improving performance and preventing forgetting in reservoir computing networks.
method Integrates neuron-specific learnable thresholds to optimize sparsity without altering dynamics, learning read-out weights and thresholds via gradient rule.
result Threshold learning improves performance and alleviates catastrophic forgetting.
New method estimates neuronal connectivity from partially observed data.
problem Estimating neuronal connectivity from partially observed data.
method Two-step approach: low-rank covariance completion followed by graph structure estimation.
result Graph selection consistency demonstrated for one approach.
It will be shown that according to theorems of K. Menger, every neuron grid if identified with a curve is able to preserve the adopted qualitative structure of a data space. Furthermore, if this identification is made, the neuron grid structure can always be mapped to a subset of a universal neuron grid which is constr…
A new method to understand neural networks by sampling the 'inverse set' of a neuron.
problem Understanding the internal representation of neurons in neural networks.
method Optimization-based sampling approach to characterize the input space that excites a neuron.
result Inspection of samples reveals regularities that help understand the neuron's representation.
ChaosNet uses chaotic neural networks for classification with high accuracy on limited data.
problem Classification with limited training data.
method ChaosNet architecture with GLS neurons and a novel learning algorithm exploiting topological transitivity.
result ChaosNet achieves high accuracy (73.89% - 98.33%) with as few as 7 training samples per class.
CHANI learns classification tasks with local transformations inspired by biology.
problem Proving neural networks can learn classification tasks with local transformations.
method CHANI uses spiking neurons modeled by Hawkes processes with expert aggregation for local learning.
result CHANI can learn and encode multiple classes, forming assemblies of neurons.
Researchers develop methods to learn neuron dynamics from colored noise.
problem Learning nonlocal stochastic neuron dynamics from colored noise.
method Proposed two methods for closing Fokker-Planck equations: nonlocal large-eddy-diffusivity closure and data-driven sparse regression.
result Mutual information and total correlation between stimulus and neuron states calculated for FHN neuron.
Single neuron with ADA learns XOR and outperforms other functions.
problem Classifying linearly non-separable data.
method Proposed a new artificial neuron with apical dendrite activation.
result ADA function achieves 100% accuracy on XOR and superior performance on benchmark datasets.
Despite our extensive knowledge of biophysical properties of neurons, there is no commonly accepted algorithmic theory of neuronal function. Here we explore the hypothesis that single-layer neuronal networks perform online symmetric nonnegative matrix factorization (SNMF) of the similarity matrix of the streamed data. …
Stable unactivated neurons reduce expressiveness in ReLU networks.
problem Reducing expressiveness in ReLU neural networks due to stably unactivated neurons.
method Investigated the probability of neurons being stably unactivated in ReLU networks with symmetric weight and bias distributions.
result Proved the probability of a neuron being stably unactivated in the second hidden layer of a ReLU network.
Novel chaotic neurons improve AI with minimal training data.
problem Limited training data for AI algorithms.
method Intrinsically chaotic neurons inspired by chaos theory.
result Classification accuracy up to 95.8% with just 2 training samples per class.
Single-spike neurons can approximate as well as multi-spike neurons.
problem Limitation of single-spike neurons in spiking neural networks.
method Comparison of single-spike and multi-spike neural networks.
result Single-spike and multi-spike neural networks are equivalent in approximation capabilities.
The challenge of assigning importance to individual neurons in a network is of interest when interpreting deep learning models. In recent work, Dhamdhere et al. proposed Total Conductance, a "natural refinement of Integrated Gradients" for attributing importance to internal neurons. Unfortunately, the authors found tha…