Deep learning networks are approximated using dynamical systems theory.
problem Understanding the approximation capabilities of deep learning networks.
method Modeling deep residual networks as continuous-time dynamical systems and using approximation theories in Lp. result Established general sufficient conditions for universal approximation of deep residual networks.
This work proposes a new algorithm for efficient reduced modeling of non-linear dynamical systems.
problem Reduced modeling of computationally demanding dynamical systems to balance accuracy and complexity.
method Embedding trajectories in a RKHS, solving low-rank constraint optimization problems, and exploiting kernel-based computations.
result The proposed algorithm achieves a gain in approximation accuracy and computational efficiency.
Unified approach to dynamic programming improves reinforcement learning performance.
problem Improving approximate dynamic programming for broader reinforcement learning applications.
method Proposes Generalized Value Iteration (GVI) and its approximated version, Approximate GVI (AGVI), unifying value iteration, advantage learning, and dynamic policy programming.
result Demonstrates performance guarantees for AGVI, including those for existing algorithms.
Review of algorithms for linear system approximations.
problem Linear approximation of high-dimensional dynamical systems.
method State-of-the-art algorithms for low-rank DMD.
result Provides additional details for comprehensive understanding.
Deep neural nets approximate random dynamical system trajectories uniformly in time.
problem Approximating trajectories of random dynamical systems over infinite time horizons.
method Recurrent neural networks with simple feedback structures.
result Certain random trajectories can be approximated uniformly in time to any desired accuracy.
Paper approximates risk measures using SGD with Langevin dynamics.
problem Approximating arbitrary law invariant risk measures.
method Stochastic Gradient Langevin Dynamics (SGD-Langevin) for general risk measures.
result Non-asymptotic convergence rates of the approximation algorithm.
Framework optimizes battery storage for markets by separating long-term degradation from short-term market dynamics.
problem Intractable computation due to timescale mismatch between battery degradation and market dynamics.
method Approximate dynamic programming with value function approximation and pseudo-time encoding.
result Policy outperforms benchmarks in real-time market scenarios.
Adaptive neural network approximates stochastic system densities.
problem Approximating high-dimensional stochastic dynamical systems.
method Temporal KRnet (tKRnet) trained with adaptive collocation points and temporal decomposition.
result Improves density approximation for stochastic systems without curse of dimensionality.
This paper finds an exact solution for low-rank DMD, improving model complexity and accuracy.
problem Approximating high-dimensional dynamical systems with tractable models.
method Formulates a low-rank constrained optimisation problem and finds an exact closed-form solution.
result Shows a polynomial-time algorithm for computing the optimal low-rank approximation.
PSAEM combines EM and particle methods for efficient dynamical system learning.
problem Learning dynamical systems with stochastic approximation and particle methods.
method Particle stochastic approximation EM (PSAEM) algorithm combining stochastic approximation EM and particle Gibbs with ancestor sampling (PGAS).
result PSAEM achieves superior computational performance and convergence compared to existing methods.
Transformers can learn noisy linear systems with depth and IID data.
problem Learning noisy linear dynamical systems with transformers.
method Theoretical analysis of multi-layer and single-layer transformers with respect to L2-testing loss. result Single-layer transformers have a non-diminishing lower bound on approximation error, suggesting depth separation.
Two heuristics solve dynamic multiple travelling salesmen problems.
problem Dynamic routing with unknown customers.
method Balanced dynamic closest vehicle heuristic and balanced dynamic assignment vehicle heuristic.
result Continuous approximation models for strategic dynamic routing.
New algorithm improves dynamic mode decomposition for high-dimensional data.
problem Reduced modeling in high-dimensional spaces.
method Low rank constraint optimization and kernel-based computation.
result Gain in approximation accuracy and computational efficiency.
Deep learning approximates system moments from data.
problem Approximating moments of spatial probabilistic systems.
method Dynamic Boltzmann Distributions (DBDs) with deep Boltzmann machines (DBMs).
result Learned moment closures improve generalization over traditional methods.
Classical algorithms approximate quantum dynamics using subsampling.
problem Simulating quantum mechanical systems efficiently on classical computers.
method Randomized numerical linear algebra and the Nyström method for approximating matrix exponentials.
result Classical algorithms can efficiently simulate quantum computations under specific conditions.
Novel autoencoder method approximates Koopman operator in low dimensions.
problem Challenges in approximating finite Koopman operators using data-driven methods.
method Mori-Zwanzig autoencoder (MZ-AE) for robust Koopman operator approximation.
result Improved predictive capability and robust long-term statistical performance.
Kernel method approximates dynamical operators from data.
problem Estimating eigenfunctions of dynamical operators from data.
method Kernel-based approach in reproducing kernel Hilbert spaces.
result Eigenfunctions estimated via matrix eigenvalue problems.
In this paper we show how to approximate a Heath-Jarrow-Morton dynamics for the forward prices in commodity markets with arbitrage-free models which have a finite dimensional state space. Moreover, we recover a closed form representation of the forward price dynamics in the approximation models and derive the rate of c…
Survey on statistical theories of neural networks, focusing on approximation, training dynamics, and generative models.
problem Understanding the statistical properties and training dynamics of neural networks.
method Review of existing literature on neural networks from three perspectives: approximation, training dynamics, and generative models.
result Theoretical insights into neural network training dynamics and generative models.
New method for optimistic planning in MDPs using regularization.
problem Optimistic planning in infinite-horizon discounted MDPs.
method Regularized dynamic programming for approximate value iteration.
result Achieves near-optimal statistical guarantees in learning policies.
Transformers learn rich in-context dependencies efficiently.
problem Understanding how transformers learn long-range dependencies efficiently.
method Approximation and dynamics analysis of induction head mechanisms.
result Abrupt transition from lazy to rich mechanisms during training.
Paper approximates solutions for complex decision processes with limited precision.
problem Approximating the set of all solutions for Multi-objective Markov Decision Processes.
method Limited precision approach based on White's multi-objective value-iteration dynamic programming algorithm.
result The number of calculated solutions is tractable and approximates the true Pareto front.
This study explains why approximate NGD works well in wide neural networks.
problem Understanding why NGD with approximate Fisher information converges fast in wide neural networks.
method Analyzing asymptotic training dynamics in function space via the neural tangent kernel.
result NGD with approximate Fisher information achieves the same fast convergence as exact NGD under specific conditions.
Ensembles dynamic models using random feature approximations.
problem Online scalable Bayesian learning with dynamic models and ensembling.
method Random feature approximations and dynamic models using random walks.
result Better performance with alternative basis expansions like Hilbert space Gaussian processes.
New framework for consistent submodular maximization with insertions and deletions.
problem Maintaining near-optimal solutions in a dynamic setting with insertions and deletions.
method Developed a general framework for fully dynamic submodular maximization, instantiated for cardinality and rank-k matroid constraints.
result First constant-factor approximations with sublinear consistency for both cardinality and rank-k matroid constraints.
ESNs trained with Tikhonov least squares approximate ergodic dynamical systems in L2(μ) norm.
problem Approximating ergodic dynamical systems using ESNs.
method Tikhonov least squares regression on ESNs trained on observations from an ergodic dynamical system.
result ESNs trained with Tikhonov least squares approximate the target function in the L2(μ) norm.
Approximate dynamic programming is a popular method for solving large Markov decision processes. This paper describes a new class of approximate dynamic programming (ADP) methods- distributionally robust ADP-that address the curse of dimensionality by minimizing a pessimistic bound on the policy loss. This approach tur…
In this paper we extend the work of Smith and Papamichail (1999) and present fast approximate Bayesian algorithms for learning in complex scenarios where at any time frame, the relationships between explanatory state space variables can be described by a Bayesian network that evolve dynamically over time and the observ…
This paper develops a general method for constructing Poisson integrators.
problem Lack of a general theory for Poisson integrators due to geometric challenges.
method Adapting structural results about symplectic realizations to create geometric approximations.
result Developed a general approach for constructing geometric integrators on Poisson manifolds.
Model approximates market prices and returns without prior market dynamics.
problem Simultaneously approximate market prices and log returns.
method GDN model of Kratsios and Papon (2022) for generalized Ornstein-Uhlenbeck process.
result Universal approximation guarantees for conditional distributions and contingent claims.
A novel model uses ODE-based random features to model nonlinear dynamical systems.
problem Modeling highly nonlinear dynamical systems with uncertainty quantification.
method Compositions of physics-informed random features derived from ODEs, combined with deep Gaussian processes and approximate Bayesian inference.
result The model effectively captures nonlinear behavior in real-world multivariate time series data and achieves comparable performance to other models on benchmark tasks.
Improved particle approximation for mean-field neural networks.
problem Particle approximation error for mean-field neural networks.
method Improved particle approximation error by leveraging the problem structure in risk minimization.
result Established an LSI-constant-free particle approximation error concerning the objective gap.
Improved online learning algorithms using ADP for adversarial environments.
problem Minimizing regret in adversarial online learning with vector-valued losses.
method Approximate dynamic programming to characterize lower Pareto frontier of expected losses.
result Improved performance bounds compared to existing online learning algorithms.
New algorithm tackles non-stationary reinforcement learning with general function approximation.
problem Understanding non-stationary MDPs with function approximation.
method Dynamic Bellman Eluder (DBE) dimension for complexity, sliding window mechanism, confidence set design.
result Upper bound on dynamic regret for proposed SW-OPEA algorithm.
New Banach spaces for ReLU networks enable better function approximation and gradient dynamics analysis.
problem Function approximation and gradient dynamics in multi-layer ReLU networks.
method Developed Banach spaces for ReLU networks, defined new function representations, and analyzed gradient flow dynamics.
result Gradient flow dynamics of the new representation is the continuous analog of gradient descent for ReLU networks.
Neural operators approximate Stackelberg game solutions.
problem Intractability of follower's best-response operator in dynamic Stackelberg games.
method Used attention-based neural operators to approximate the best-response operator.
result Approximate best-response operator yields close game value.
Study birth-death dynamics for sampling Gibbs measures with nonconvex potentials.
problem Sampling Gibbs measures with nonconvex potentials.
method Birth-death dynamics, Kullback-Leibler divergence, χ2 divergence, kernel-based approximations, Γ-convergence of gradient flows. result Probability density converges exponentially fast to Gibbs equilibrium measure with a universal rate.
Transformers approximate mean-field dynamics of indistinguishable particles.
problem Approximating the dynamics of indistinguishable particles in complex systems.
method Using transformers to model the mean-field dynamics of interacting particle systems.
result Theoretical bounds on the distance between true and transformer-obtained mean-field dynamics.
New algorithm speeds up spectral clustering for dynamic graphs.
problem Prohibitive complexity of spectral clustering for dynamic graphs.
method Reuse past cluster assignments and use fast Chebyshev graph filtering.
result Achieves clustering quality approximating spectral clustering with significant complexity benefits.
Study shows polynomial-width neural networks can closely approximate infinite-width networks in polynomial time.
problem Approximating dynamics of polynomial-width neural networks with infinite-width networks.
method Bounding approximation gap through a differential equation governed by mean-field dynamics, considering local Hessian.
result Polynomially many neurons are sufficient to closely approximate mean-field dynamics.
New metrics using Laplace approximation improve Gaussian process model selection.
problem Finding a balance between model accuracy, interpretability, and simplicity.
method Introducing multiple metrics based on the Laplace approximation to evaluate Gaussian process models.
result Our metrics provide comparable performance to dynamic nested sampling but are significantly faster.
The paper develops a neural network for learning dynamics from data.
problem Trade-off between representational capacity and overfitting in EDMD.
method Linear recurrent autoencoder network for Koopman operator approximation.
result Improved model reduction and nonlinear reconstruction techniques.
One approach to monitoring a dynamic system relies on decomposition of the system into weakly interacting subsystems. An earlier paper introduced a notion of weak interaction called separability, and showed that it leads to exact propagation of marginals for prediction. This paper addresses two questions left open by t…
Alternative dynamic paired comparison model using Gaussian Processes.
problem Sports prediction and ranking players or teams.
method Dynamic paired comparison model with Gaussian Process priors, incorporating covariates, and efficient Bayesian inference.
result The GP model outperforms Elo and Glicko on log loss, especially with surface covariates.
New algorithm tackles dynamic assortment optimization with knapsack constraints.
problem Optimizing retailer's assortment decisions under resource constraints with multi-nomial choice modeling.
method Epoch-based re-solving algorithm that transforms MNL's fractional structure into a linear program with slack variables.
result Regret scales logarithmically with time horizon and resource capacities.
New algebraic approach for approximating Hamiltonian dynamics.
problem Approximating Hamilton-Jacobi solutions on symplectic groupoids.
method Introducing a pre-Lie algebra and Butcher trees for symplectic groupoids.
result New class of Poisson integrators for Hamiltonian dynamics.
Paper develops a new framework for analyzing certainty equivalents and dynamic risk premia using Malliavin calculus and Wiener chaos analysis.
problem Limitations of Arrow-Pratt approximation for arbitrary sequences of vanishing risks.
method Develops a new framework based on Malliavin calculus and Wiener chaos analysis, combining Itô calculus, the Clark--Ocone representation, and the Wiener chaos decomposition.
result Establishes a unified framework linking expected utility theory, stochastic analysis, and Wiener chaos expansions, revealing higher-order certainty equivalents and dynamic risk premia.
The paper develops a diffusion model for limit order book dynamics.
problem Modeling the complex dynamics of limit order books in financial markets.
method Derives a diffusion approximation for discrete-time limit order book models with non-linear dynamics.
result The discrete dynamics converge to a diffusion limit under suitable assumptions.