New method uses backward SDEs for deep learning uncertainty.
problem Uncertainty quantification in deep learning models.
method Probabilistic machine learning with stochastic neural networks and stochastic optimal control.
result Effectiveness validated through numerical experiments.
Paper examines the structure of stochastic gradients in deep learning.
problem Exploring the structure and heavy tails of stochastic gradients in deep learning.
method Conducted formal statistical tests on stochastic gradients and gradient noise.
result Stochastic gradients and gradient noise do not exhibit power-law heavy tails, but their covariance spectra do.
Deep kernel learning combines the non-parametric flexibility of kernel methods with the inductive biases of deep learning architectures. We propose a novel deep kernel learning model and stochastic variational inference procedure which generalizes deep kernel learning approaches to enable classification, multi-task lea…
Deep learning solves complex stochastic control with jumps.
problem Solving high-dimensional stochastic control tasks with jumps.
method Model-based approach using two neural networks, iteratively trained with objectives derived from the Hamilton-Jacobi-Bellman equation.
result Demonstrates effectiveness in solving complex high-dimensional stochastic control tasks.
GenFormer uses deep learning to generate complex stochastic data.
problem Creating synthetic stochastic data that matches real-world statistical properties.
method Transformer-based deep learning model that maps Markov state sequences to time series values.
result GenFormer preserves target marginal distributions and other statistical properties in multivariate spatio-temporal data.
Exact second-order optimization for deep learning reduces computational cost and improves performance.
problem Inadequate use of second-order optimization methods in deep learning due to high computational cost and non-convexity.
method Developed an exact stochastic second-order Newton method that addresses the non-convexity issue and provides an expression for the stochastic Hessian.
result Exact second-order Newton direction formula and its application in deep learning datasets.
Paper proves deep learning method for stochastic control converges and outperforms existing algorithms.
problem Formulating and solving stochastic control problems using FBSDE and SMP.
method Deep learning algorithm based on SMP, with convergence proof and error bounds.
result Deep SMP-BSDE algorithm converges and outperforms existing methods in high-dimensional stochastic control problems.
New model solves complex SDEs with high-dimensional spatial and stochastic spaces.
problem Solving SDEs with high-dimensional spatial and stochastic spaces.
method Physics-informed deep generative model (sPI-GeM) combining PI-BasisNet and PI-GeM.
result Scalable solution for high-dimensional SDE problems.
INP accelerates stochastic simulations using deep Bayesian active learning.
problem Computational expense of stochastic simulations at fine-grained resolution.
method Interactive Neural Process (INP) framework combining spatiotemporal surrogate model and active learning acquisition function.
result STNP outperforms baselines in accelerating stochastic simulations and LIG achieves state-of-the-art for Bayesian active learning.
New method solves stochastic control problems with delays using deep learning.
problem Stochastic control problems with delayed control in drift and diffusion.
method Characterization via Riccati PDEs and deep learning scheme.
result Illustrates effect of delay on Markowitz portfolio allocation problem.
A new algorithm SRG-DQN reduces variance in deep Q-learning.
problem Inaccurate estimation of anchor points in SVRG for deep Q-learning.
method Introduces recursive gradient variance reduction for stochastic gradient updates.
result Demonstrates improved efficiency and effectiveness of SRG-DQN on reinforcement learning tasks.
Deep Hedging learns optimal strategies for various risk levels.
problem Finding optimal hedging policies for diverse risk aversions.
method Continuous Reinforcement Learning with actor-critic algorithm.
result Demonstrated effectiveness in a stochastic volatility model.
Paper introduces a new deep-learning method for quantum mechanics.
problem Simulating time-evolving Schrödinger equations efficiently.
method Generative diffusion models and stochastic mechanics.
result Significantly lower computational complexity compared to existing methods.
Deep learning model approximates stochastic responses.
problem Approximating stochastic responses using neural networks.
method Generative neural network with conditional maximum mean discrepancy (CMMD) loss.
result Excellent performance on benchmark problems.
Enhances deep kernel learning with stochastic latent variables for better model regularization.
problem Weak model regularization in deep kernel learning, especially on small datasets.
method Introduces DLVKL model with stochastic latent variables, NSDE for expressive posterior, and hybrid prior.
result DLVKL-NSDE outperforms existing deep GPs on large datasets.
Method uses deep learning to estimate traffic intensity.
problem Estimating stochastic intensity of traffic processes.
method Deep neural networks for nonlinear filtering.
result Deep learning method accurately estimates traffic intensity.
A new method approximates expected empirical loss for stochastic deep learning tasks.
problem Determining optimal step sizes for stochastic gradient descent in deep learning.
method Applying one-dimensional function fitting to noisy losses of vertical cross sections to approximate expected empirical loss.
result The method leads to a robust and straightforward optimization method that performs well across datasets and architectures.
NOVAS uses adaptive stochastic search for non-convex optimization in deep networks.
problem Non-convex optimization challenges in deep neural networks.
method Adaptive stochastic search for non-convex optimization.
result NOVAS outperforms existing alternatives in a structured prediction task.
New deep learning model robust to adversarial attacks using stochastic LWTA units.
problem Adversarial robustness in deep learning networks.
method Introduces deep networks with stochastic LWTA activations, combining them with Bayesian non-parametric tools.
result Achieves high robustness to adversarial perturbations, outperforming state-of-the-art methods.
Embeds sparsity in deep neural networks, allowing exact zero parameters.
problem Learning sparse structures in deep networks.
method Embeds sparsity into neural network structure, allowing exact zero parameters during training.
result Can learn both structured and unstructured sparsity.
JaxSGMC simplifies SG-MCMC for Bayesian deep learning.
problem Uncertainty quantification in deep learning models.
method Modular stochastic gradient MCMC in JAX.
result Facilitates trustworthy neural network predictions.
Deep learning approximates SPDE solutions from noise trajectories.
problem Approximating solutions to stochastic partial differential equations (SPDEs).
method Uses neural networks to approximate SPDE solutions based on noise realizations.
result Accurately estimates SPDE solutions and functionals like mean and variance.
We introduce two constructions in geometric deep learning for 1) transporting orientation-dependent convolutional filters over a manifold in a continuous way and thereby defining a convolution operator that naturally incorporates the rotational effect of holonomy; and 2) allowing efficient evaluation of manifold convol…
New deep learning method solves complex BSDEs efficiently.
problem Solving high-dimensional nonlinear BSDEs.
method Reformulate as global optimization, approximate solution with deep neural network, globally minimize quadratic local loss functions.
result Demonstrated effectiveness on various high-dimensional nonlinear BSDEs, including finance applications.
Deep fictitious play converges to Nash equilibrium in stochastic differential games.
problem Finding Nash equilibrium in large stochastic differential games.
method Decouples the game into sub-optimization problems and solves each player's optimal strategy with deep BSDE method.
result Deep fictitious play converges to the true Nash equilibrium.
Analyzes deep neural networks training errors with SGD and random init.
problem Lack of rigorous understanding of deep learning algorithms.
method Mathematical analysis of deep learning with SGD and random init.
result First full error analysis for deep learning with SGD and random init.
This paper organizes recent deep learning theory advances.
problem Lack of theoretical foundations in deep learning.
method Literature review and categorization into six groups.
result Organized recent advances in deep learning theory.
Continuous-time SGD converges under certain conditions, useful for deep learning.
problem Minimizing population expected loss in learning problems.
method Continuous-time approximation of stochastic gradient descent.
result Establishes sufficient conditions for convergence, applicable to overparametrized neural networks.
Deep ensembles effectively capture epistemic uncertainty through training stochasticity, providing a frequentist perspective.
problem Understanding and quantifying epistemic uncertainty in machine learning models.
method Bootstrap-based estimator and decomposition of deep ensembles into data variability and training stochasticity.
result Deep ensembles primarily capture training stochasticity, explaining their effectiveness in quantifying epistemic uncertainty.
VSDN models sporadic time series with neural SDEs.
problem Modeling irregular and sparse time series data.
method Variational Bayesian method and neural SDEs.
result VSDNs outperform state-of-the-art models in prediction and interpolation.
New deep learning method solves stochastic control problems.
problem Solving strongly coupled FBSDEs for stochastic control.
method Modified deep BSDE method with new loss function.
result Empirical convergence of the new method for three problems.
DeepRSCN models nonlinear systems using stochastic configurations.
problem Modeling nonlinear dynamic systems efficiently.
method Incrementally constructed deep reservoir computing framework with random parameters and online weight updates.
result DeepRSCN outperforms single-layer networks in efficiency, learning, and generalization.
New algorithms improve sampling from Bayesian deep learning models.
problem Sampling from the posterior of deep neural networks is inefficient.
method Adaptive SGMCMC algorithms with biased drift.
result Proposed algorithms significantly outperform existing methods.
Model-free learning for multi-agent stochastic games is an active area of research. Existing reinforcement learning algorithms, however, are often restricted to zero-sum games, and are applicable only in small state-action spaces or other simplified settings. Here, we develop a new data efficient Deep-Q-learning method…
New deep learning architecture learns martingales efficiently.
problem Efficiently learning martingales in financial derivatives pricing.
method High-order weak approximation algorithms of Runge-Kutta type.
result Deep neural networks based on this architecture learn martingales effectively.
A new optimization method improves deep learning accuracy without hyper-parameter tuning.
problem Computational demands and convergence behavior in deep learning training.
method Stochastic quasi-Gauss-Newton (SQGN) optimization method combining stochastic quasi-Newton, Gauss-Newton, and variance reduction.
result SQGN provides excellent accuracy without hyper-parameter experimentation, improving convergence and computational performance.
We propose an adaptive optimization method for deep learning that dynamically adjusts batch size.
problem Optimizing deep learning models with varying sensitivity to batch size selection.
method Adaptive regularization with dynamically determined stochastic batch size based on gradient norms.
result Our method outperforms state-of-the-art optimization algorithms in generalization and robustness.
Investigates portfolio selection with transaction costs and stochastic volatility, using deep learning for computation.
problem Optimal portfolio selection with transaction costs and stochastic volatility.
method Two-factor stochastic volatility model, option-implied utility function, deep learning policy iteration.
result Deep learning method effectively computes optimal investment decisions under transaction costs and stochastic volatility.
Deep learning models viewed through tame geometry for convergence guarantees.
problem Understanding convergence guarantees in deep learning models.
method Introducing tame geometry concepts and tools for nonsmooth nonconvex settings.
result Illustrates tame geometry as a natural framework for AI systems, especially deep learning.
We propose a generic framework to calibrate accuracy and confidence of a prediction in deep neural networks through stochastic inferences. We interpret stochastic regularization using a Bayesian model, and analyze the relation between predictive uncertainty of networks and variance of the prediction scores obtained by …
DPC uses physics and neural nets to solve SDEs.
problem Solving stochastic differential equations with missing physics.
method Physics-data fusion with conditional maximum mean discrepancy (CMMD) loss.
result DPC achieves highly accurate solutions on benchmark examples.
Paper introduces MCSD, a method for uncertainty estimation in deep learning.
problem Need for reliable uncertainty quantification in deep neural networks.
method Theoretical connection to variational inference and empirical benchmarking of MCSD.
result MCSD achieves competitive predictive accuracy and improves uncertainty ranking.
Two deep learning algorithms solve utility maximisation problems in finance.
problem Solving utility maximisation problems in finance with deep learning.
method Two algorithms: one for Markovian problems via HJB equation and 2BSDE, the other for non-Markovian problems via adjoint BSDE.
result Highly accurate results with low computational cost, solving problems with power, log, and non-HARA utilities in various models.
Study models forest transitions with deep learning for parameter estimation.
problem Complex dynamics of forest, agricultural, and abandoned lands.
method Developed a stochastic differential equation model and used deep learning for parameter estimation.
result Deep learning approach estimates model parameters from time-series data.
Recent advances in deep reinforcement learning have achieved human-level performance on a variety of real-world applications. However, the current algorithms still suffer from poor gradient estimation with excessive variance, resulting in unstable training and poor sample efficiency. In our paper, we proposed an innova…
Deep learning scheme identifies and reconstructs chaotic and stochastic systems from noisy data.
problem Challenging identification of governing equations from noisy and partial observations.
method Jointly learns inference model and governing laws using variational deep learning.
result Framework generalizes state-of-the-art methods and accounts for stochastic variabilities.
Although stochastic gradient descent (SGD) is a driving force behind the recent success of deep learning, our understanding of its dynamics in a high-dimensional parameter space is limited. In recent years, some researchers have used the stochasticity of minibatch gradients, or the signal-to-noise ratio, to better char…
Bayesian methods enhance deep learning models by improving reliability and uncertainty.
problem Improving reliability and uncertainty awareness in deep learning models.
method Approximate Bayesian inference techniques, including SG-MCMC and VI, applied to deep learning models.
result Enhanced posterior inference for deep learning models, particularly in neural networks and generative models.