Quadratic autoencoder improves low-dose CT image denoising.
problem Low-dose CT image denoising.
method Quadratic autoencoder architecture applied to CT denoising.
result Quadratic autoencoder achieves superior denoising performance and efficiency.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “quadratic neurons”
Quadratic neurons enhance deep networks' approximation capabilities.
problem Improving deep networks' expressive power and efficiency.
method Introduced quadratic neurons and analyzed their performance in deep quadratic networks.
result Quadratic networks can approximate functions more efficiently and uniquely than conventional networks.
Over-parametrization speeds up learning a single neuron model.
problem Understanding why over-parametrization accelerates learning in neural networks.
method Studied a simple model of a single teacher neuron with quadratic activation, showing how over-parametrization can lead to faster convergence.
result Over-parametrization helps gradient descent enter the neighborhood of a global optimal solution faster.
The artificial neural network is a popular framework in machine learning. To empower individual neurons, we recently suggested that the current type of neurons could be upgraded to 2nd order counterparts, in which the linear operation between inputs to a neuron and the associated weights is replaced with a nonlinear qu…
A common analytical problem in neuroscience is the interpretation of neural activity with respect to sensory input or behavioral output. This is typically achieved by regressing measured neural activity against known stimuli or behavioral variables to produce a "tuning function" for each neuron. Unfortunately, because …
Study shows different training methods yield varying prediction risks for two-layers neural networks.
problem Understanding prediction risks in two-layers neural networks under different training regimes.
method Examined three training regimes: random features, neural tangent, and fully trained neural network.
result There is a significant gap in prediction risk between the random features and neural tangent regimes when the number of neurons is smaller than the ambient dimension.
The paper studies quadratic neural networks, proving existence of spurious minima and saddle points.
problem Understanding the loss landscape of neural networks with quadratic activations.
method Theoretical analysis of mean squared error loss for neural networks with quadratic activations.
result Proves existence of spurious local minima and saddle points in the training landscape of deep overparameterized quadratic neural networks.
SGD efficiently learns the XOR function with near-optimal sample complexity.
problem Learning the XOR function with a 2-layer neural network.
method Minibatch SGD on a 2-layer neural network with ReLU activations, focusing on signal-finding and signal-heavy phases.
result Achieves population error with samples.
New SDP method certifies neural network robustness across all classes efficiently.
problem Certifying robustness of neural networks across multiple classes.
method Quadratic model + SDP relaxation + pruning strategy.
result Significant computational speed-up and scalability to large datasets.
AGF explains feature learning in neural networks through alternating steps.
problem Understanding what features neural networks learn and how they learn them.
method AGF is an algorithmic framework that approximates the dynamics of feature learning in two-layer networks.
result AGF provides a unified framework to understand feature learning in neural networks, matching experimental results across various architectures.
New analysis shows capacity of treelike neural networks with various activations.
problem Analyzing the capacity of treelike neural networks with diverse activations.
method Utilized Random Duality Theory and its partially lifted version to handle various activations.
result The capacity of treelike neural networks decreases for large network width but converges to a constant value.
Neuron Shapley identifies key neurons in deep networks, improving model accuracy and fairness.
problem Identifying responsible neurons in deep networks for better model performance and fairness.
method Neuron Shapley framework quantifies neuron contributions, accounting for interactions.
result Removing just 30 critical filters can destroy model accuracy, revealing network function.
SNEPPPs use squared neural networks to efficiently model Poisson point processes.
problem Efficiently modeling Poisson point processes with flexibility.
method Parameterizing intensity function with squared norm of a two-layer neural network.
result Closed-form integration of intensity function for quadratic time computation.
Describes explaining neurons in deep representations using compositional logical concepts.
problem Interpreting neuron behavior in deep neural networks.
method Identifying compositional logical concepts that closely approximate neuron behavior.
result Compositional explanations provide insights into model performance and allow for adversarial example creation.
SeReNe prunes neurons with low sensitivity to reduce network size.
problem Large neural networks consume too many resources on resource-constrained devices.
method Exploits neural sensitivity as a regularizer to prune neurons with low sensitivity.
result Pruning neurons with low sensitivity achieves competitive compression ratios.
Under-parameterized networks can either copy or average teacher weights, leading to universal optimal solutions.
problem Approximating a teacher network with an under-parameterized student network.
method Analyzing shallow neural networks with erf activation function and unitary teacher weights, proving copy-average configurations are critical points and finding the optimal solution.
result The optimal solution for under-parameterized networks has a universal structure, whether copying or averaging teacher neurons.
This research investigates selectively pruning hyper and hypo neurons to improve neural network generalization.
problem Improving neural network generalization to unseen data.
method Investigates pruning hyper and hypo neurons selectively in fully connected layers of CNNs.
result Selective pruning of hyper and hypo neurons improves model performance on out-of-domain data.
Modeling hidden neurons in SNNs using mesoscopic approximations.
problem Underconstrained problem of modeling unobserved neurons in SNNs.
method Coarse-graining and mean-field approximations to derive neuLVM.
result neuLVM can efficiently model large SNNs and recover connectivity parameters.
Detection and skeletonization of single neurons and tracer injections using topological methodscs.CV
Topological methods improve neuron analysis and tracer injection summary.
problem Traditional methods fail to capture the tree-like structure of neurons.
method Discrete Morse (DM) Theory for neuron skeletonization and consensus tree summarization.
result Significant performance improvements over non-topological methods.
We propose a new generic type of stochastic neurons, called -neurons, that considers activation functions based on Jackson's -derivatives with stochastic parameters . Our generalization of neural network architectures with -neurons is shown to be both scalable and very easy to implement. We demonstrate expe…
Reduces learning periodic neural networks to lattice problems, proving hardness under cryptographic assumptions.
problem Learning single periodic neurons in noisy environments.
method Reduction to worst-case lattice problems, using LLL algorithm.
result Polynomial-time algorithms for learning these functions are hard under cryptographic assumptions.
We developed a faster method for calculating neuron importance in neural networks.
problem Assigning importance to individual neurons in deep learning models.
method We developed Neuron Integrated Gradients, a scalable implementation of Total Conductance.
result Neuron Integrated Gradients is faster and empirically stronger than DeepLIFT.
BEAN models neuronal correlations to create interpretable representations.
problem Hard interpretation of dense-layer representations in DNNs.
method Inspired by neuroscience, BEAN models neuronal correlations and dependencies.
result BEAN enables formation of interpretable neuronal clusters without sacrificing model performance.
SpaRCe optimizes reservoir computing by learning neuron thresholds to improve performance and prevent forgetting.
problem Improving performance and preventing forgetting in reservoir computing networks.
method Integrates neuron-specific learnable thresholds to optimize sparsity without altering dynamics, learning read-out weights and thresholds via gradient rule.
result Threshold learning improves performance and alleviates catastrophic forgetting.
A Note on Topology Preservation in Classification, and the Construction of a Universal Neuron Gridcs.NE
It will be shown that according to theorems of K. Menger, every neuron grid if identified with a curve is able to preserve the adopted qualitative structure of a data space. Furthermore, if this identification is made, the neuron grid structure can always be mapped to a subset of a universal neuron grid which is constr…
This work improves DNN interpretability by reducing neuron ambiguity.
problem Lack of interpretability in DNNs, especially in healthcare applications.
method Developed a metric to evaluate neuron consistency, used adversarial examples to identify ambiguous features, and proposed adversarial training to improve consistency.
result Reduced ambiguity of neurons in DNNs, improving interpretability.
A new method to understand neural networks by sampling the 'inverse set' of a neuron.
problem Understanding the internal representation of neurons in neural networks.
method Optimization-based sampling approach to characterize the input space that excites a neuron.
result Inspection of samples reveals regularities that help understand the neuron's representation.
Neural networks learn to mimic brain neurons with two-input activation functions, improving performance and robustness.
problem Training neural networks to mimic the complex interactions of brain neurons.
method Developed a network-in-network architecture with two-input activation functions, optimized hyperparameters, and compared to conventional ReLU networks.
result Two-input activation functions can learn soft XOR functions, improving network performance and robustness.
CHANI learns classification tasks with local transformations inspired by biology.
problem Proving neural networks can learn classification tasks with local transformations.
method CHANI uses spiking neurons modeled by Hawkes processes with expert aggregation for local learning.
result CHANI can learn and encode multiple classes, forming assemblies of neurons.
BinaryGAN uses binary neurons for GAN training.
problem Training GANs with binary outputs.
method End-to-end backpropagation with sigmoid-adjusted straight-through estimators.
result BinaryGAN can generate binary-valued predictions.
Researchers develop methods to learn neuron dynamics from colored noise.
problem Learning nonlocal stochastic neuron dynamics from colored noise.
method Proposed two methods for closing Fokker-Planck equations: nonlocal large-eddy-diffusivity closure and data-driven sparse regression.
result Mutual information and total correlation between stimulus and neuron states calculated for FHN neuron.
Single neuron with ADA learns XOR and outperforms other functions.
problem Classifying linearly non-separable data.
method Proposed a new artificial neuron with apical dendrite activation.
result ADA function achieves 100% accuracy on XOR and superior performance on benchmark datasets.
Despite our extensive knowledge of biophysical properties of neurons, there is no commonly accepted algorithmic theory of neuronal function. Here we explore the hypothesis that single-layer neuronal networks perform online symmetric nonnegative matrix factorization (SNMF) of the similarity matrix of the streamed data. …
Stable unactivated neurons reduce expressiveness in ReLU networks.
problem Reducing expressiveness in ReLU neural networks due to stably unactivated neurons.
method Investigated the probability of neurons being stably unactivated in ReLU networks with symmetric weight and bias distributions.
result Proved the probability of a neuron being stably unactivated in the second hidden layer of a ReLU network.
Novel chaotic neurons improve AI with minimal training data.
problem Limited training data for AI algorithms.
method Intrinsically chaotic neurons inspired by chaos theory.
result Classification accuracy up to 95.8% with just 2 training samples per class.
Single-spike neurons can approximate as well as multi-spike neurons.
problem Limitation of single-spike neurons in spiking neural networks.
method Comparison of single-spike and multi-spike neural networks.
result Single-spike and multi-spike neural networks are equivalent in approximation capabilities.
Optimal neuron activation functions improve neural network performance.
problem Limited expressive power of standard neuron activation functions in neural networks.
method Additive Gaussian process regression to construct individual neuron activation functions.
result Optimal neuron activation functions lead to better performance and reduced overfitting.
New neuron model learns and adapts its receptive field.
problem Learning and focusing on informative inputs.
method Adaptive locally connected neuron model using backpropagation.
result Focusing neurons outperform dense layers in classification tasks.
Novel 'strong neuron' improves deep learning efficiency and robustness.
problem Improving deep learning efficiency and robustness against adversarial attacks.
method Introducing a novel 'strong neuron' model and a constructive training algorithm.
result Achieved 10x-100x reduction in operations count and hardware requirements.
Deep CNN model predicts neuronal cell health from images.
problem Predicting the biological activity of chemical compounds on neuronal cells.
method Deep convolutional neural network (CNN) with residual connections.
result Achieved 99.6% accuracy in distinguishing treated from untreated cells.
New artificial neuron doubles weight for improved deep learning accuracy.
problem Improving deep learning models' accuracy.
method Introducing a double-weight neuron, tested on MNIST and CIFAR-10 datasets.
result Significant improvement in classification accuracy for MNIST and CIFAR-10 datasets.
When Neurons Failstat.ML
We view a neural network as a distributed system of which neurons can fail independently, and we evaluate its robustness in the absence of any (recovery) learning phase. We give tight bounds on the number of neurons that can fail without harming the result of a computation. To determine our bounds, we leverage the fact…
Single ReLU neuron's gradient dynamics reveal support vectors as key to generalization.
problem Understanding the generalization capability of ReLU networks.
method Examined gradient flow dynamics and support vectors in single ReLU neuron training.
result Support vectors play a crucial role in the generalization of ReLU networks.
It is widely conjectured that the reason that training algorithms for neural networks are successful because all local minima lead to similar performance, for example, see (LeCun et al., 2015, Choromanska et al., 2015, Dauphin et al., 2014). Performance is typically measured in terms of two metrics: training performanc…
Novel method decorrelates neurons for better deep learning model generalization.
problem High correlations between neurons limit deep learning model generalization.
method Regularization terms from minimum spanning tree of neuron cliques, using correlation dissimilarities.
result Our regularizers outperform existing methods and minimize neuron redundancies.
A neuron is a basic physiological and computational unit of the brain. While much is known about the physiological properties of a neuron, its computational role is poorly understood. Here we propose to view a neuron as a signal processing device that represents the incoming streaming data matrix as a sparse vector of …
Mesoscopic model infers neural population dynamics from spike trains.
problem Challenges in fitting mechanistic spiking networks to empirical population data.
method Fit mesoscopic model to aggregate population activity, using likelihood of single-neuron and connectivity parameters.
result Extracts posterior correlations between model parameters and defines subsets of parameters able to reproduce data.
New training method for ReLU networks achieves optimal weight size for memorization.
problem Approximate memorization of arbitrary real labels with neural networks.
method Complex recombination training procedure for ReLU networks.
result Approximate memorization with nearly optimal weight size and neuron count.