Aims to describe neural network training dynamics using two-time-scale models.
problem Lack of a general mathematical description of neural network training.
method Introduces a theoretical framework based on two-time-scale population dynamics.
result Derives selection-mutation equations and effective fitness for hyperparameters.
Active learning method for neural population dynamics using optogenetics.
problem Efficiently selecting neurons to stimulate for identifying neural population dynamics.
method Developed active learning procedure for low-rank regression to determine informative photostimulation patterns.
result Demonstrated a two-fold reduction in data required for predictive power using low-rank linear dynamical systems model.
Neural population activity often exhibits rich variability and temporal structure. This variability is thought to arise from single-neuron stochasticity, neural dynamics on short time-scales, as well as from modulations of neural firing properties on long time-scales, often referred to as "non-stationarity". To better …
A powerful approach for understanding neural population dynamics is to extract low-dimensional trajectories from population recordings using dimensionality reduction methods. Current approaches for dimensionality reduction on neural data are limited to single population recordings, and can not identify dynamics embedde…
dLDS models neural dynamics as sparse combinations of simpler components.
problem Understanding complex neural dynamics at a population level.
method Proposes a decomposed dynamical system model trained through dictionary learning.
result Model efficiently captures and demix diverse neural dynamics.
New method learns population dynamics from snapshots using JKO scheme and inverse optimization.
problem Recovering underlying process governing particle evolution from discrete time samples.
method Combines JKO scheme with inverse optimization techniques for end-to-end adversarial training.
result Improved performance over prior JKO-based methods with theoretical guarantees.
Bubblewrap predicts neural dynamics online, scaling to thousands of neurons.
problem Direct testing of neural population hypotheses requires online inference of neural state.
method Soft tiling of neural manifold with fast, stable dimensionality reduction.
result Bubblewrap model outperforms existing methods in noisy conditions.
MFM integrates multiple evolving populations using Wasserstein manifold flows.
problem Learning dynamics of multiple interacting populations evolving over time.
method Meta Flow Matching (MFM) integrates vector fields on Wasserstein manifold using amortized flow models and GNN embeddings.
result MFM improves prediction of individual treatment responses on multi-patient single-cell drug screen data.
When governed by underlying low-dimensional dynamics, the interdependence of simultaneously recorded population of neurons can be explained by a small number of shared factors, or a low-dimensional trajectory. Recovering these latent trajectories, particularly from single-trial population recordings, may help us unders…
In order to interact intelligently with objects in the world, animals must first transform neural population responses into estimates of the dynamic, unknown stimuli which caused them. The Bayesian solution to this problem is known as a Bayes filter, which applies Bayes' rule to combine population responses with the pr…
Neural networks model COVID-19 spread with partial isolation data.
problem Modeling the spread of COVID-19 with limited data.
method Semi-supervised neural networks solving inverse problems.
result Estimates optimal conditions for virus spread and passive population.
A body of recent work in modeling neural activity focuses on recovering low-dimensional latent features that capture the statistical structure of large-scale neural populations. Most such approaches have focused on linear generative models, where inference is computationally tractable. Here, we propose fLDS, a general …
The curse of dimensionality affects neural network optimization, especially with smooth functions.
problem The curse of dimensionality in neural network optimization.
method Examined through the evolution of the parameter distribution under 2-Wasserstein gradient flow.
result The curse of dimensionality persists in neural network optimization, even with smooth functions.
To understand how rich dynamics emerge in neural populations, we require models exhibiting a wide range of activity patterns while remaining interpretable in terms of connectivity and single-neuron dynamics. However, it has been challenging to fit such mechanistic spiking networks at the single neuron scale to empirica…
New framework for online control in evolving populations.
problem Control of evolving populations in real-world conditions.
method Online control framework for linear and non-linear dynamical systems.
result Near-optimal regret bounds for gradient-based controllers.
Gradient descent learns over-param neural nets better than NTK.
problem Learning over-parametrized neural networks with ReLU activations.
method Gradient descent from random initialization on a Gaussian input distribution.
result Gradient descent achieves population loss o(1/d), while NTK achieves Ω(1/d). New method learns population dynamics from snapshots, outperforming existing models.
problem Capturing periodic and other dynamical properties of population dynamics.
method Wasserstein Lagrangian Mechanics (WLM) for learning second-order dynamics from observed marginals.
result WLM outperforms existing methods across various dynamics, including vortex dynamics, embryonic development, and flocking.
Study uses machine learning to predict predator-prey dynamics without prior knowledge.
problem Predicting predator-prey interactions without prior knowledge of the system.
method Applied Neural Ordinary Differential Equations (Neural ODEs) and Universal Differential Equations (UDEs) to the Lotka-Volterra model.
result UDEs outperform Neural ODEs in predicting predator-prey dynamics, especially in noisy data.
New approach uses 'growth' and 'harvesting' concepts to improve deep learning models.
problem Current deep learning models lack transparency and high convergence rates.
method Reconsider neural networks as single-species population dynamics with balanced growth and harvesting rates.
result SGD with balanced growth and harvesting rates outperforms adaptive methods in all three requirements.
Develops CLDS models to model neural activity with nonlinear dynamics.
problem Complex, nonlinear dynamics in neural population activity.
method Conditionally Linear Dynamical System (CLDS) models using Gaussian Process (GP) priors.
result CLDS models can perform well even in data-limited conditions.
Low-rank structure emerges in neural networks during learning.
problem Understanding the evolution of synaptic connectivity over learning.
method Investigated the rank of 3-tensor formed by weight matrices throughout learning.
result Inferred weights are low-tensor-rank and evolve in a fixed low-dimensional subspace.
Neurons in higher cortical areas, such as the prefrontal cortex, are known to be tuned to a variety of sensory and motor variables. The resulting diversity of neural tuning often obscures the represented information. Here we introduce a novel dimensionality reduction technique, demixed principal component analysis (dPC…
New method infers dynamical systems from population data.
problem Inferring dynamical systems from population data.
method Deducing and estimating Fokker-Planck equation, projecting to test functions, sparse inference.
result Induces driving forces of dynamical systems.
New metric compares noisy neural trajectories using optimal transport.
problem Existing metrics fail to capture differences in noisy, dynamic neural responses.
method Proposed an optimal transport distance metric for Gaussian processes.
result Metric effectively compares neural dynamics in different systems.
New technologies for recording the activity of large neural populations during complex behavior provide exciting opportunities for investigating the neural computations that underlie perception, cognition, and decision-making. Nonlinear state space models provide an interpretable signal processing framework by combinin…
DICE learns population dynamics from discrete samples.
problem Learning smooth population dynamics from discrete data.
method Discrete Inverse Continuity Equation (DICE) method.
result DICE models are stable and generate representative samples.
FHRN uses continuous-time dynamics to stabilize reentrant neural computation.
problem Stabilizing reentrant neural computation.
method Formulated as a continuous-time neural-ODE system, revealing norm-regulated reentry.
result Achieves stable oscillatory trajectories through population-level gain modulation.
We propose a method to infer stochastic low-rank RNNs from neural data.
problem Fitting low-rank RNNs to noisy, stochastic neural data.
method Variational sequential Monte Carlo methods for stochastic low-rank RNNs.
result Lower dimensional latent dynamics compared to state-of-the-art methods.
Robust RL improves controller robustness to dynamics variations using adversarial populations.
problem Robustness issues in RL when dynamics are perturbed.
method Adversarial population augmentation to the Robust RL formulation.
result Population-based adversarial approach yields more robust and generalizable policies.
BayesFlow learns complex models using neural networks.
problem Estimating parameters in complex, non-likelihood models.
method Invertible neural networks for global Bayesian inference.
result Global probabilistic mapping from data to parameters.
This work learns models for population dynamics using variational methods and higher-order quadrature.
problem Modeling population dynamics of physical systems with stochastic and mean-field effects.
method Variational problem to infer gradient fields, combining Monte Carlo sampling with higher-order quadrature rules.
result Accurate prediction of population dynamics over a wide range of parameters.
New algorithm learns switching dynamics from multiple neural signals.
problem Learning accurate switching dynamical system models from multimodal neural data.
method Unsupervised learning algorithm for multiscale switching dynamical system models.
result Switching multiscale dynamical system models outperform single-scale models in behavior decoding.
Study on symmetries in wide neural networks' dynamics without bias.
problem Understanding symmetries in the dynamics of wide two-layer neural networks.
method Analyzing symmetries in gradient flow on population risk for infinitely wide networks.
result Symmetries can simplify the dynamics of predictors and reduce the dimensionality of the problem.
NeuPL learns diverse policies in strategy games efficiently.
problem Iterative training of policies in strategy games leads to under-trained good-responses and wasteful repetition.
method NeuPL uses a single conditional model to represent a population of policies, offering convergence guarantees and transfer learning.
result NeuPL achieves better performance and efficiency across various domains, enabling access to novel strategies.
The paper analyzes the dynamics of a simple neural network using a mean-field approach.
problem Understanding the training dynamics of neural networks, especially in classification tasks.
method Developed an analytic theory using a mean-field limit for a simple neural network.
result Explicitly solved the dynamics of a linearly separable dataset with a linear hinge loss.
Measures of wealth and production have been found to scale superlinearly with the population of a city. Therefore, it makes economic sense for humans to congregate together in dense settlements. A recent model of population dynamics showed that population growth can become superexponential due to the superlinear scalin…
Develops a method to model neural dynamics with flexible yet interpretable latent states.
problem Capturing complex nonlinear dynamics in neural time series while maintaining interpretability.
method Gaussian Process Switching Linear Dynamical System (gpSLDS) that balances expressiveness and interpretability.
result Favorable performance in comparison to rSLDS on synthetic and real neuroscience data.
Paper uses neural networks to calibrate Lee-Carter models for multiple populations.
problem Calibrating Lee-Carter models for multiple populations with neural networks.
method Developed neural network architectures to fit Lee-Carter and Poisson Lee-Carter models simultaneously.
result Smooth and less sensitive parameter estimates, improved forecasting performance.
A new method for learning gradient flows from population dynamics.
problem Reconstructing population dynamics from limited data.
method Residual approach to enforce continuity equations, combining with data-fitting divergence.
result Demonstrated state-of-the-art performance across trajectory inference benchmarks.
Study reveals sharp characterisation of local minima in neural network loss landscapes.
problem Characterizing local minima in high-dimensional two-layer ReLU neural networks.
method Exact low-dimensional representation of local minima using summary statistics and link with one-pass SGD dynamics.
result Local minima in overparameterized neural networks form discrete families with varying stability and reachability.
Populations of species in ecosystems are often constrained by availability of resources within their environment. In effect this means that a growth of one population, needs to be balanced by comparable reduction in populations of others. In neutral models of biodiversity all populations are assumed to change increment…
Paper introduces a new method for improving reinforcement learning performance using transfer learning.
problem Improving reinforcement learning performance with limited sample sizes in dynamic decision-making scenarios.
method Developed a novel ``re-weighted targeting procedure'' and ``transfer deep Q∗-learning'' approach. result Demonstrated improved reinforcement learning performance through strategic sample construction.
New method improves NN performance across various settings.
problem Improving neural network performance across different datasets and architectures.
method Population Gradients (PG) method to calculate non-local gradient estimates.
result Significantly improves final performance across architectures, data-sets, and hyper-parameters.
EGDL predicts TB outbreaks with deep learning, integrating epidemiological models.
problem Predicting TB outbreaks with complex spatiotemporal dynamics.
method Modified MN-SIR model with Bayesian inference, deep neural networks.
result EGDL delivers robust and accurate TB outbreak predictions.
The paper analyzes the training dynamics of neural networks using kernel methods.
problem Understanding the training dynamics of neural networks in high-dimensional settings.
method High-dimensional asymptotics and gradient flow on kernel least-squares objectives.
result The training dynamics of neural networks undergo three stages, characterized by behaviors in the Oracle and Empirical worlds.
Improved neural population modeling using shared features and ensemble detection.
problem Missing shared coding properties in neural latent variable models.
method Feature sharing across tuning curves and soft clustering of neurons.
result More interpretable and better-performing neural population models.
Develops geometry for Lotka-Volterra model of species competition.
problem Population dynamics of competing species.
method Least squares variational method, Lagrange-Hamilton geometry.
result Jacobi stability discussed for the Lotka-Volterra system.
The paper studies learning dynamics in two-layer neural networks.
problem Learning dynamics and time scales in two-layer neural networks.
method Gradient flow dynamics of a wide two-layer neural network in high-dimension, with data following a single-index model.
result The learning dynamics exhibit separation of timescales and intermittency.