Neural networks with learned biases can approximate any function.
problem Whether neural networks with only learned biases can approximate any continuous function.
method Theoretical and numerical analysis of random weights and learned biases in neural networks.
result Feedforward and recurrent neural networks with random weights can approximate any continuous function and dynamical systems.
CoNBONet improves reliability analysis of complex systems with fast, energy-efficient predictions.
problem Time-dependent reliability analysis of nonlinear systems under stochastic excitations is computationally demanding.
method CoNBONet combines deep operator networks with neuroscience-inspired neuron models for fast, energy-efficient inference.
result CoNBONet provides reliable coverage of failure probabilities with theoretical guarantees.
Visualizes DNNs using topographic maps for better understanding.
problem Difficulty in understanding how DNNs solve tasks.
method Adapting neuroscience methods to visualize DNN activations.
result Improved transparency and interpretability of DNN-based systems.
Estimates curvature of network manifolds to understand community structure.
problem Understanding the geometry of network models to infer community structure.
method Develops hypothesis tests to determine manifold type, dimension, and curvature from noisy distance matrices.
result Consistently estimates manifold type, dimension, and curvature from Riemannian manifolds of constant curvature.
AKOrN uses synchronized neurons to improve AI tasks.
problem Improving AI performance through better neural representations.
method AKOrN introduces synchronized neurons to replace threshold units.
result AKOrN improves performance across various AI tasks.
Over the last several years, the use of machine learning (ML) in neuroscience has been rapidly increasing. Here, we review ML's contributions, both realized and potential, across several areas of systems neuroscience. We describe four primary roles of ML within neuroscience: 1) creating solutions to engineering problem…
We investigate time-dependent data analysis from the perspective of recurrent kernel machines, from which models with hidden units and gated memory cells arise naturally. By considering dynamic gating of the memory cell, a model closely related to the long short-term memory (LSTM) recurrent neural network is derived. E…
Deep neural networks (DNNs) transform stimuli across multiple processing stages to produce representations that can be used to solve complex tasks, such as object recognition in images. However, a full understanding of how they achieve this remains elusive. The complexity of biological neural networks substantially exc…
Power of network tests degrades when vertices are misaligned.
problem Power loss in network hypothesis testing due to vertex shuffling.
method Theoretical analysis and simulations of Frobenius norm differences in random dot product and stochastic block models.
result Shuffling vertices can significantly reduce the power of network tests.
A new approach to learning in brain-like networks using adversarial algorithms.
problem Complex inter-dependencies in brain-like networks not compatible with conditional independence assumptions.
method Adversarial algorithm for learning models of perceptual processing.
result The approach can mimic known neural phenomena and yields testable hypotheses.
A new method improves fitting neural data with spiking network models.
problem Fitting spiking network models to neural activity does not produce realistic data.
method Augment log-likelihood with dissimilarity terms measured by summary statistics and optimized via back-propagation.
result The new method generates more realistic neural activity statistics and improves network connectivity inference.
This work tackles catastrophic forgetting in neural networks by mimicking brain's metaplasticity.
problem Catastrophic forgetting in neural networks, where new tasks erase previously learned ones.
method Interpreting binarized neural networks as metaplastic systems, adjusting their training technique.
result Training technique reduces catastrophic forgetting without needing previously presented data.
Financial markets modeled like brain networks using dMNC.
problem Understanding latent dynamics in financial markets.
method Biologically inspired framework using dMNC.
result Structural persistence, regime shifts, and early warning signals identified.
New statistical inference method for high-dimensional Hawkes processes.
problem Uncertainty evaluation of network estimates in high-dimensional point process data.
method Develops a new statistical inference procedure using concentration inequalities and martingale central limit theory.
result Characterizes the convergence rate of test statistics for high-dimensional Hawkes processes.
Predictive coding networks use inference learning for efficient AI modeling.
problem Traditional AI methods struggle with complex neural patterns.
method Inference learning for hierarchical Bayesian inference models.
result PCNs outperform traditional BP methods in efficiency and flexibility.
PCGs encompass a broader range of neural networks.
problem Understanding the broader scope of neural network models.
method Proving PCGs as a superset of feedforward neural networks.
result PCGs represent a wider class of neural network models.
Although Deep Neural Networks have seen great success in recent years through various changes in overall architectures and optimization strategies, their fundamental underlying design remains largely unchanged. Computational neuroscience on the other hand provides more biologically realistic models of neural processing…
New framework for task-independent legged locomotion.
problem Building stable legged locomotion systems in robotics.
method Task-independent spiking central pattern generator using learning methods.
result Robotic legged locomotion at different speeds and within the same gait cycle.
Bayesian optimization generates personalized face stimuli for cognitive neuroscience.
problem Lack of personalized face stimuli in cognitive neuroscience studies.
method Combines GANs with Bayesian optimization to identify individual response patterns to faces.
result Algorithm efficiently generates optimal faces maximizing individual subject's response.
Proposes a method to estimate and infer networks from multiple high-dimensional point processes.
problem Estimating and inferring networks from multiple high-dimensional point processes with shared and unique structures.
method Joint estimation procedure for networks of high-dimensional point processes incorporating weights to encourage similarity.
result Powerful hierarchical multiple testing procedure for edges of all estimated networks, controlling family-wise error rate.
SBI uses neural networks to infer model parameters from simulators.
problem Computational infeasibility of Bayesian inference for complex models.
method Training neural networks on simulator-generated data.
result Efficient Bayesian inference without likelihood evaluations.
ST-GCN improves rs-fMRI prediction accuracy by modeling spatio-temporal graph connectivity.
problem Existing rs-fMRI methods neglect functional connectivity or temporal dynamics.
method Spatio-temporal graph convolutional network (ST-GCN) trained on BOLD time series.
result ST-GCN predicts gender and age more accurately than common methods.
A new model for sequential memory using temporal predictive coding.
problem Forming accurate memory of sequential stimuli in the brain.
method Proposes a novel PC-based model called temporal predictive coding (tPC).
result Shows that tPC models can accurately memorize and retrieve sequential inputs.
Proposes a model for classifying high-dimensional time series with interpretable parameters.
problem Challenges in classifying high-dimensional time series, especially in neuroscience.
method Model-based approach using sparsity in inverse spectral density matrices, with interpretability of model parameters.
result Model demonstrates consistency and sure screening property, enabling nuanced inferences.
Deep learning has sparked a network of mutual interactions between different disciplines and AI. Naturally, each discipline focuses and interprets the workings of deep learning in different ways. This diversity of perspectives on deep learning, from neuroscience to statistical physics, is a rich source of inspiration t…
Neuroscience is undergoing faster changes than ever before. Over 100 years our field qualitatively described and invasively manipulated single or few organisms to gain anatomical, physiological, and pharmacological insights. In the last 10 years neuroscience spawned quantitative big-sample datasets on microanatomy, syn…
One conjecture in both deep learning and classical connectionist viewpoint is that the biological brain implements certain kinds of deep networks as its back-end. However, to our knowledge, a detailed correspondence has not yet been set up, which is important if we want to bridge between neuroscience and machine learni…
Machine learning for ASD diagnosis using morphological MRI networks.
problem Challenging to diagnose ASD using MRI due to heterogeneity and incomplete network neuroscience.
method Crowdsourced Kaggle competition to develop and benchmark ML pipelines.
result First-ranked team achieved 70% accuracy, 72.5% sensitivity, and 67.5% specificity.
We can define a neural network that can learn to recognize objects in less than 100 lines of code. However, after training, it is characterized by millions of weights that contain the knowledge about many object types across visual scenes. Such networks are thus dramatically easier to understand in terms of the code th…
SNVI combines likelihood estimation with variational inference for efficient Bayesian inference.
problem Bayesian inference in models with intractable likelihoods.
method Sequential Neural Variational Inference (SNVI) that combines likelihood-estimation with variational inference.
result SNVI is more computationally efficient than previous algorithms without sacrificing accuracy.
Low-rank structure emerges in neural networks during learning.
problem Understanding the evolution of synaptic connectivity over learning.
method Investigated the rank of 3-tensor formed by weight matrices throughout learning.
result Inferred weights are low-tensor-rank and evolve in a fixed low-dimensional subspace.
Dual Variable Learning Rates improve neural network training efficiency.
problem Training neural networks efficiently and effectively.
method DVLR uses different learning rates for correct and incorrect responses, and adjusts rates based on network performance.
result DVLR consistently improves neural network accuracy across different types and domains.
S-VNNs improve VNNs by sparsifying covariance matrices.
problem Spurious correlations in covariance matrices degrade VNNs' performance and efficiency.
method Apply sparsification techniques on sample covariance matrix and integrate into VNN architecture.
result S-VNNs achieve improved performance, stability, and reduced computational time.
New method for nonlinear Granger causality improves predictive relationships.
problem Challenges in applying Granger causality to nonlinear data.
method Permutation of covariate set, artificial neural networks, consistent variance estimation.
result Permutation method outperforms other techniques in predicting nonlinear relationships.
A neuroscience method to understanding the brain is to find and study the preferred stimuli that highly activate an individual cell or groups of cells. Recent advances in machine learning enable a family of methods to synthesize preferred stimuli that cause a neuron in an artificial or biological brain to fire strongly…
Cluster Quilting clusters fragmented data sets for neuroscience and genomics.
problem Clustering fragmented data sets in neuroscience and genomics.
method Cluster Quilting method using patch ordering, patchwise SVD, sequential linear mapping, and k-means.
result Cluster Quilting discovers more accurate clusters than other methods.
Networks are a fundamental model of complex systems throughout the sciences, and network datasets are typically analyzed through lower-order connectivity patterns described at the level of individual nodes and edges. However, higher-order connectivity patterns captured by small subgraphs, also called network motifs, de…
Two synthetic likelihood methods learn EBM of likelihood from simulator data for SBI.
problem Conduct inference from experimental observations using high-fidelity simulators.
method Learn conditional EBM of likelihood using synthetic data conditioned on parameters.
result Learned likelihood combined with prior yields posterior estimate for sampling.
Convolutional Neural Networks (CNN) outperform traditional classification methods in many domains. Recently these methods have gained attention in neuroscience and particularly in brain-computer interface (BCI) community. Here, we introduce a CNN optimized for classification of brain states from magnetoencephalographic…
The study explores how brain development can inspire efficient deep learning models.
problem Efficient and robust optimization procedures for deep learning.
method Inspiration from biological neural development to improve deep learning models.
result Biological neural development can inspire efficient and robust optimization procedures.
NeuroQuery synthesizes brain mapping evidence across diverse concepts.
problem Lack of comprehensive meta-analysis of human brain mapping across different mental processes and mechanisms.
method A multivariate model that predicts the spatial distribution of neurological observations given text describing an experiment, cognitive process, or disease.
result Captures relationships and neural correlates of 7,547 neuroscience terms across 13,459 neuroimaging publications.
The uninformative ordering of artificial neurons in Deep Neural Networks complicates visualizing activations in deeper layers. This is one reason why the internal structure of such models is very unintuitive. In neuroscience, activity of real brains can be visualized by highlighting active regions. Inspired by those te…
We present a novel adversarial framework for training deep belief networks (DBNs), which includes replacing the generator network in the methodology of generative adversarial networks (GANs) with a DBN and developing a highly parallelizable numerical algorithm for training the resulting architecture in a stochastic man…
Perfect clustering achieved in hypergraphs with enough interactions.
problem Complexity and lack of tractable models for analyzing hypergraphs.
method Introduced an interaction hypergraph model for analyzing hypergraphs, defined latent embeddings, and analyzed spectral estimators.
result A spectral estimate of interaction latent positions can achieve perfect clustering with enough interactions.
Paper introduces CRP-O framework for uncertainty quantification in deep operators.
problem Uncertainty quantification in energy-efficient deep learning algorithms, especially in SNNs.
method CRP-O framework using RP networks and SCP, with Gaussian Process Regression for super-resolution.
result Enhanced uncertainty bounds improve UQ estimates compared to existing methods.
Framework for reconstructing nonlinear systems from multi-modal time series data.
problem Reconstructing nonlinear dynamical systems from multi-modal time series data.
method Dynamic interpretable recurrent neural networks coupled with generalized linear models for multi-modal data integration.
result Framework efficiently compensates for noisy or missing information in one data channel using other channels.
TSNPE improves SBI efficiency and scalability.
problem Efficient and scalable simulation-based inference for complex models.
method Sequential inference with truncated proposals.
result TSNPE performs on par with previous methods and scales to complex models.
New method for analyzing brain dynamics using HMMs and graph models.
problem Limited ability of current brain models to explain spontaneous dynamic state changes.
method Hidden Markov Graph Models (HMGMs) and spatiotemporal random walks.
result Identification of important brain community structures.