New dynamics for SGD in small learning rate regime.
problem Improving stochastic gradient descent in small learning rate regime.
method Introducing stochastic modified flows and distribution dependent stochastic modified flows.
result Captures fluctuating dynamics of SGD in small learning rate - infinite width scaling regime.
Generative models for complex stochastic dynamics using adversarial learning.
problem Data-driven modeling of multistep stochastic dynamics.
method Adversarial learning with GANs and MMD for stable model classes.
result Stable generative models for long-time prediction and stochastic systems.
New framework compares two stochastic learning dynamics in games.
problem Inability to distinguish between different learning rules leading to the same steady-state behavior.
method Developed a framework for comparative analysis of stochastic learning dynamics with different update rules.
result Identified distinct behaviors in the paths to stochastically stable states for LLL and ML.
sFML learns stochastic dynamical systems from data.
problem Learning unknown stochastic dynamical systems from measurement data.
method sFML extends FML for deterministic systems, using a stochastic flow map composed of deterministic and stochastic sub-maps.
result sFML constructs a stochastic evolution model approximating unknown stochastic systems.
This work learns effective dynamics from short-term data of stochastic systems.
problem Learning effective dynamics from short-term data of stochastic systems.
method Proposes a novel algorithm using a neural network (Auto-SDE) to learn invariant slow manifold from data.
result Validated through numerical experiments to be accurate, stable, and effective.
Method learns dynamics of slow variables from stochastic data.
problem Modeling unknown multiscale stochastic systems with limited data.
method Data-driven approach to learn effective dynamics from bursts of observation data.
result Generative model accurately captures effective dynamics of slow variables.
Develops interpretable model for latent stochastic systems from noisy data.
problem Learning interpretable models of latent stochastic dynamical systems from noisy data.
method Semi-parametric model using Gaussian process for drift, inference of latent paths with sparse variational description.
result Flexible nonparametric model of dynamics with interpretable portraits.
Study on test risk dynamics in learning theory with stochastic gradient flow.
problem Understanding test risk in stochastic gradient flow dynamics.
method Path integral formulation for small learning rates, explicit computation for weak features.
result Explicit corrections due to stochastic term in dynamics, good agreement with simulations.
Machine learning infers time-reversible dynamics from data.
problem Learn time-reversible dynamics constrained by initial and final conditions.
method Machine learning algorithms solve boundary value problems for deterministic and stochastic dynamics.
result Inferred time-reversible dynamics for various types of systems.
New method learns dynamics from sparse data using geometric constraints.
problem Learning dynamics from sparse, undersampled data.
method Reformulates inference as a stochastic control problem, using geometry-driven path augmentation.
result Accurately recovers stochastic dynamics from extremely undersampled data.
Deep learning solves complex stochastic control with jumps.
problem Solving high-dimensional stochastic control tasks with jumps.
method Model-based approach using two neural networks, iteratively trained with objectives derived from the Hamilton-Jacobi-Bellman equation.
result Demonstrates effectiveness in solving complex high-dimensional stochastic control tasks.
SGD in DLNs reveals feature learning dynamics.
problem Understanding SGD dynamics in DLNs during saddle-to-saddle training.
method Stochastic Langevin dynamics with anisotropic, state-dependent noise; one-dimensional per-mode SDEs; Boltzmann distribution approximation.
result SGD noise encodes feature learning progression but does not alter saddle-to-saddle dynamics.
VSDN models sporadic time series with neural SDEs.
problem Modeling irregular and sparse time series data.
method Variational Bayesian method and neural SDEs.
result VSDNs outperform state-of-the-art models in prediction and interpolation.
Safe learning of stochastic dynamics with safety constraints.
problem Learning controlled stochastic dynamics with safety constraints.
method Iterative expansion of a safe control set using kernel-based confidence bounds.
result The method ensures safe exploration and efficient estimation of system dynamics.
This work extracts stochastic dynamical systems with α-stable Lévy noise.
problem Extracting data-driven governing laws of dynamical systems with non-Gaussian noise.
method End-to-end deep learning approach for learning drift and diffusion coefficients for α-stable Lévy noise. result Effectiveness of the method confirmed by numerical experiments.
New method learns stochastic thermodynamics from system currents.
problem Understanding entropy production in complex dynamical systems.
method Constructs learning framework using currents and machine learning loss functions.
result Derives loss functions for thermodynamic functions directly from dynamics.
In this paper we study how to learn stochastic, multimodal transition dynamics in reinforcement learning (RL) tasks. We focus on evaluating transition function estimation, while we defer planning over this model to future work. Stochasticity is a fundamental property of many task environments. However, discriminative f…
Framework infers Langevin dynamics from stochastic observations of latent systems.
problem Inferring non-stationary Langevin dynamics from indirect stochastic observations.
method Non-parametric framework explicitly modeling stochastic observation process and non-stationary latent dynamics.
result Correct inference of non-stationary dynamics requires accounting for non-equilibrium states and observation duration.
New model predicts stochastic dynamics with hidden variables.
problem Predicting state transitions in stochastic dynamical systems.
method Hierarchical Bayesian linear regression with local features and variational EM algorithm.
result Parsimonious model structures and fast, accurate predictions.
Analysis of SGD for Gaussian mixture classification using dynamical mean-field theory.
problem Learning dynamics of SGD for a neural network classifying Gaussian mixture.
method Applying dynamical mean-field theory to track SGD dynamics in high dimensions.
result Reveals how SGD navigates the non-convex loss landscape.
Derives effective continuous dynamics for adaptive SGD methods.
problem Analyzing noise in adaptive SGD methods.
method Stochastic modified equations framework and Malladi's scaling rules.
result Sampling-induced noise in SGD limits to independent Brownian motions.
PASTIS selects minimal models from stochastic dynamics data.
problem Overfitting in model selection for stochastic dynamics.
method Combining likelihood-estimation statistics with extreme value theory.
result PASTIS reliably identifies minimal models, even with low sampling rates or error.
Deep learning scheme identifies and reconstructs chaotic and stochastic systems from noisy data.
problem Challenging identification of governing equations from noisy and partial observations.
method Jointly learns inference model and governing laws using variational deep learning.
result Framework generalizes state-of-the-art methods and accounts for stochastic variabilities.
A framework learns multiscale dynamics from single trajectories using normalizing flows.
problem Learning effective stochastic dynamics from single observed paths of slow variables.
method Data-driven approach based on coupled multiscale SDEs, stochastic averaging, and normalizing flows for density modeling.
result Scalable approach to capturing epistemic uncertainty in multiscale systems.
The paper develops algorithms for computing in stochastic acceptors.
problem Computing input marginals and acceptance probabilities in stochastic acceptors.
method Dynamic programming for input marginals, EM for parameter estimation.
result Efficient algorithms for computing in stochastic acceptors.
DeepONet accelerates reliability analysis of stochastic nonlinear systems.
problem Time-dependent reliability analysis of systems with stochastic forcing.
method DeepONet, a novel operator network, learns function-to-function mappings.
result DeepONet efficiently and accurately predicts system responses.
Improved noise estimation in latent neural SDEs enhances model accuracy.
problem Latent neural SDEs underestimate noise, limiting their stochastic dynamics modeling.
method Explicit additional noise regularization in the loss function.
result Model accurately captures diffusion component of stochastic time series data.
Unified treatment of RC in stochastic and deterministic settings.
problem Understanding and generalizing reservoir computing in both deterministic and stochastic contexts.
method Investigation of state-space systems, analysis of fading memory and solution stability, introduction of stochastic echo states.
result Generality of fading memory and solution stability in state-space systems, even without the echo state property.
Study models forest transitions with deep learning for parameter estimation.
problem Complex dynamics of forest, agricultural, and abandoned lands.
method Developed a stochastic differential equation model and used deep learning for parameter estimation.
result Deep learning approach estimates model parameters from time-series data.
Method learns dynamics from noisy partial observations.
problem Reconstructing stochastic dynamical systems from indirect noisy data.
method Amortized path generation method for nonlinear stochastic filtering.
result Learned conditional path generator quantifies uncertainty.
A new algorithm flattens multi-modal distributions for better deep learning.
problem Bayesian learning in big data with multi-modal distributions.
method Contour Stochastic Gradient Langevin Dynamics (CSGLD) algorithm.
result The CSGLD algorithm avoids local traps in deep neural networks.
A new algorithm improves Bayesian federated learning by reducing communication overhead.
problem Bayesian federated learning constraints, including privacy, data ownership, and communication overhead.
method Proposes Quantised Langevin Stochastic Dynamics (QLSD) for Bayesian federated learning, using gradient compression and variance reduction techniques.
result Non-asymptotic and asymptotic convergence guarantees for QLSD and its improved versions.
PSAEM combines EM and particle methods for efficient dynamical system learning.
problem Learning dynamical systems with stochastic approximation and particle methods.
method Particle stochastic approximation EM (PSAEM) algorithm combining stochastic approximation EM and particle Gibbs with ancestor sampling (PGAS).
result PSAEM achieves superior computational performance and convergence compared to existing methods.
Extends SINDy to model stochastic dynamical systems.
problem Modeling stochastic dynamical systems from data.
method Sparse Identification of Nonlinear Dynamics (SINDy) extended to stochastic systems, with proof of asymptotic correctness and practical algorithms.
result Proves asymptotic correctness of stochastics SINDy in infinite data limit.
In this paper, we propose a novel technique to implement stochastic gradient methods, which are beneficial for learning from large datasets, through accelerated stochastic dynamics. A stochastic gradient method is based on mini-batch learning for reducing the computational cost when the amount of data is large. The sto…
Develops RL for dynamic risk assessment in stochastic optimization.
problem Time-consistent risk assessment in stochastic optimization problems.
method Model-free reinforcement learning with dynamic convex risk measures, time-consistent dynamic programming, policy gradient updates, actor-critic neural network optimization.
result Demonstrates optimal policies for statistical arbitrage, financial hedging, and robot control.
Framework learns continuous dynamics from sparse trajectories.
problem Learning dynamics from sparsely sampled and high-dimensional trajectories.
method Interpolative Multi-Marginal Flow Matching (IMMFM) framework.
result IMMFM outperforms existing methods in forecasting and downstream tasks.
Paper introduces a new volatility model for natural gas markets and discusses swing option pricing.
problem Modeling price and storage dynamics in natural gas markets with path-dependent volatility.
method Developed a novel stochastic path-dependent volatility model and used deep learning for swing option pricing.
result Proposed a deep learning method for numerical approximations of swing option pricing.
Study SGD dynamics in high-dimensional models, revealing consistent behavior across different batch sizes and learning rates.
problem Understanding SGD dynamics in high-dimensional multi-index models.
method Asymptotic analysis of SGD, developing mean-field equations and Gaussian diffusion approximations.
result Consistent SGD dynamics across different batch sizes and learning rates, distinct from gradient flow and online SGD.
The paper tackles learning optimal predictions from a single trajectory of a stochastic dynamical system.
problem Learning from a single finite trajectory of an ergodic stochastic dynamical system.
method The approach involves estimating the optimal one-step prediction function using nonlinear least squares and deriving high-probability guarantees.
result The study provides high-probability guarantees for the optimal prediction function, accounting for the non-independent and non-identically distributed nature of trajectory data.
Unified ML approach for SDEs in bounded domains.
problem Challenges in simulating SDEs with particle exit phenomena.
method Hybrid approach combining diffusion model and exit prediction network.
result Accurate modeling of interior dynamics and boundary interactions.
Analyzes high-dimensional SGD dynamics using DMFT.
problem Understanding the high-dimensional behavior of multi-pass SGD with small batch sizes.
method Derives DMFT equations for high-dimensional SGD dynamics.
result Proves DMFT equations characterize the asymptotic distribution of SGF parameters.
Study of accelerated dynamics for convex function minimization with noisy gradients.
problem Minimizing smooth convex functions with noisy gradients.
method Formulate and study continuous-time stochastic dynamics, prove convergence rates.
result Derive estimates of convergence rates for function values, both persistent and asymptotic.
SNP extends Neural Processes to handle temporal dependencies in sequences.
problem Handling temporal dependencies in sequences of stochastic processes.
method Integrates a temporal state-transition model into Neural Processes.
result First 4D model capable of dynamic 3D scene modeling.
Foundational brain dynamics model using stochastic optimal control.
problem Complex and noisy fMRI signals in brain dynamics.
method Continuous-discrete state space model with amortized inference and locally linear approximations.
result State-of-the-art results across various downstream tasks.
Researchers analyze SGD dynamics using von Mises-Fisher distributions.
problem Understanding the dynamics of stochastic gradient descent in high-dimensional spaces.
method Geometric analysis of minibatch gradient norms and directions through von Mises-Fisher distribution.
result Directional uniformity of minibatch gradients increases over SGD iterations.
Time-lagged autoencoders improve molecular dynamics data analysis.
problem Analyzing slow collective variables in molecular kinetics.
method Modified autoencoder neural network for dimension reduction.
result Time-lagged autoencoders reliably capture slow dynamics.
The paper uses machine learning to compute rare event probabilities in stochastic systems.
problem Characterizing rare events in stochastic dynamical systems with weak noise.
method Developed a neural network framework for computing quasipotential, most probable paths, and prefactors.
result Demonstrated higher effectiveness and accuracy of the algorithm in calculating mean exit times.