Stochastic reservoir computing is shown to be a universal approximator.
problem Theoretical justification for using stochastic reservoirs in machine learning.
method Investigated stochastic reservoir computing using probabilities of reservoir states as readout.
result Stochastic reservoir computers are universal approximating classes.
Proposes a new method combining Reservoir Computing and Normalizing Flow for predicting stochastic dynamical systems.
problem Predicting and capturing long-term behaviors of stochastic dynamical systems.
method Data-driven framework combining Reservoir Computing and Normalizing Flow, integrating error modeling and both approaches virtues.
result Successfully predicts the long-term evolution of stochastic dynamical systems and replicates dynamical behaviors.
Stochastic Newton and quasi-Newton methods solve large linear least-squares problems efficiently.
problem Efficiently solve large linear least-squares problems with limited computational resources.
method Introduce stochasticity in Newton and quasi-Newton approaches to handle large datasets.
result Stochastic Newton iterates may not converge to the least-squares solution.
New method efficiently computes gradients for stochastic differential equations.
problem Computing gradients for stochastic differential equations efficiently.
method Generalized adjoint sensitivity method to stochastic differential equations.
result Time-efficient and memory-efficient computation of gradients with high-order solvers.
The paper develops algorithms for computing in stochastic acceptors.
problem Computing input marginals and acceptance probabilities in stochastic acceptors.
method Dynamic programming for input marginals, EM for parameter estimation.
result Efficient algorithms for computing in stochastic acceptors.
Paper introduces stochastic mesh methods for efficient CVA computation.
problem Efficient computation of CVA for large systems using Monte Carlo methods.
method Introduces two stochastic mesh methods for CVA computation.
result Demonstrates the rate of convergence of the methods to real CVA values.
RES, a regularized stochastic version of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton method is proposed to solve convex optimization problems with stochastic objectives. The use of stochastic gradient descent algorithms is widespread, but the number of iterations required to approximate optimal arguments c…
Asynchronous SVI scales up Bayesian computation for massive data.
problem Scaling stochastic variational inference to massive data.
method Lock-free asynchronous parallel implementation of SVI.
result Linear speed-up and asymptotic convergence rate O ( 1 / ( T ) ) O(1/\sqrt(T)) O ( 1/ ( T )) . Storchastic improves stochastic AD for complex models in RL and VI.
problem Handling intractable expectations in RL and VI.
method Introduces Storchastic, a framework for AD of stochastic computation graphs with various gradient estimation methods.
result Provable unbiasedness and variance reduction for higher-order gradients.
New method uses Chebyshev expansions to compute unbiased stochastic gradients for spectral functions.
problem Computing gradients of spectral functions is expensive and challenging.
method Combining randomized trace estimators with Chebyshev expansions for unbiased stochastic gradients.
result Developed methods for optimizing objectives involving spectral-sums with fast and stable convergence.
Enhances SGLD for log-concave posteriors with asynchronous computation.
problem Sampling log-concave posterior distributions efficiently.
method Integrates asynchronous computation into SGLD with delayed gradients.
result Convergence in measure is not significantly affected by delayed gradient information.
Quantum computing speeds up analysis of financial stochastic processes.
problem Challenging simulation and analysis of continuous time stochastic processes.
method Established a quantum framework for efficient state preparation and information extraction.
result Extraction of path-dependent and history-sensitive information from stochastic processes efficiently.
This paper contains a summary of mathematical researches of stochastic properties of the long time behavior of a continuously observed (and interactively controlled) quantum--field top. Applications to interactively controlled stochastic computer-graphic dynamical systems are also discussed.
Study computes option sensitivities using Malliavin calculus for hybrid stochastic models.
problem Computing option sensitivities (Greeks) under hybrid stochastic volatility and interest rate models.
method Integrates Malliavin calculus for Delta, Vega, and Rho computation; extends to non-differentiable payoffs.
result Malliavin calculus enables effective numerical implementations for various option types.
A new method for stochastic optimization using virtual gradients.
problem Stochastic optimization challenges in computational efficiency and memory usage.
method Inspired by dynamic programming, SVGD uses a computational graph and automatic differentiation for efficient optimization.
result Experimental results show SVGD outperforms other methods on multiple datasets and network models.
A new SVRG variant reduces variance with less computation.
problem High variance in stochastic gradient descent hinders convergence.
method CheapSVRG: Uses a surrogate full-gradient on a subset of data.
result Achieves linear convergence rate with complexity trade-off.
Efficiently prices options in stochastic volatility models using sparse grids.
problem Option pricing in stochastic volatility models.
method Sparse grid high-order ADI scheme for second-order in time and fourth-order in space.
result Computational efficiency gains confirmed by numerical experiments.
Parallel SGLD improves MCMC for large matrix factorisation problems.
problem Large-scale matrix factorisation problems.
method Distributed Markov Chain Monte Carlo (MCMC) based on stochastic gradient Langevin dynamics (SGLD).
result PSGLD achieves high performance and superior convergence compared to optimisation methods.
Optimizes stochastic and online optimization methods based on problem geometry.
problem Optimizing computational and statistical outcomes in stochastic and online optimization problems.
method Characterizes optimal methods based on constraint set and gradient geometry.
result Stochastic and adaptive-gradient methods are optimal for quadratically convex constraint sets.
Method learns software resource usage from snapshots.
problem Challenges in learning time-varying, correlated resource usage.
method Graph structured Schrödinger bridge problem for nonparametric learning.
result Predicts most-likely resource distributions.
AdaSub optimizes with second-order info in low-dims subspace.
problem Efficiently use second-order optimization methods with low computational cost.
method Adaptive subspace selection for second-order optimization.
result AdaSub outperforms other stochastic optimizers in time and iterations.
Adaptive methods optimize machine learning training faster.
problem Non-adaptive stochastic optimization requires tuning for each application.
method Develop adaptive stochastic optimization methods.
result Adaptive methods offer computational savings for large-scale systems.
Stochastic variational inference (SVI) lets us scale up Bayesian computation to massive data. It uses stochastic optimization to fit a variational distribution, following easy-to-compute noisy natural gradients. As with most traditional stochastic optimization methods, SVI takes precautions to use unbiased stochastic g…
Stochastic Q-learning tackles large action spaces with reduced computation.
problem Effective decision-making in complex environments with large discrete action spaces.
method Stochastic value-based RL approaches that consider a sublinear number of actions in each iteration.
result Stochastic Q-learning achieves near-optimal returns with significantly reduced computation time.
Stochastic gradient descent procedures have gained popularity for parameter estimation from large data sets. However, their statistical properties are not well understood, in theory. And in practice, avoiding numerical instability requires careful tuning of key parameters. Here, we introduce implicit stochastic gradien…
Paper designs a time delay reservoir using stochastic logic.
problem Designing a robust time delay reservoir for noise-tolerant classification problems.
method Stochastic logic approach with re-seeding method.
result The proposed design performs well on noise-tolerant classification problems.
Review of quantile regression methods for stochastic computer experiments.
problem Quantile regression in stochastic computer experiments.
method Six metamodels categorized by order statistics, functional approaches, and Bayesian methods tested on various problems.
result Metamodels reveal good contrasts, providing guidelines for selecting the best method.
MindFlayer SGD improves parallel SGD for heterogeneous, random compute times.
problem Minimizing nonconvex functions with heterogeneous, random compute times.
method MindFlayer SGD, designed for stochastic and heterogeneous delays.
result MindFlayer SGD outperforms existing methods in environments with heavy-tailed noise.
Backprop-Q extends standard backpropagation for stochastic computation graphs.
problem Applying standard backpropagation to stochastic computation graphs is challenging.
method Construct Q-functions for each stochastic node and use them to train the SCG with standard backpropagation.
result Generalized backpropagation for stochastic computation graphs is feasible and extends learning signals beyond gradients.
A new algorithm reduces Hessian computation for nonconvex optimization.
problem Finding local minimum in nonconvex optimization efficiently.
method Stochastic variance-reduced cubic regularization (Lite-SVRC) with constant batch size.
result Achieves lower Hessian sample complexity than existing methods.
The paper uses machine learning to compute rare event probabilities in stochastic systems.
problem Characterizing rare events in stochastic dynamical systems with weak noise.
method Developed a neural network framework for computing quasipotential, most probable paths, and prefactors.
result Demonstrated higher effectiveness and accuracy of the algorithm in calculating mean exit times.
Paper introduces a new deep-learning method for quantum mechanics.
problem Simulating time-evolving Schrödinger equations efficiently.
method Generative diffusion models and stochastic mechanics.
result Significantly lower computational complexity compared to existing methods.
Stochastic Volatility in Mean models with heavy-tailed distributions using Hidden Markov Models
problem Accurate inference for Stochastic Volatility in Mean models with heavy-tailed distributions
method Numerically stable estimation procedure and parallel computing
result Significant reduction in computational times
mS2GD improves S2GD for large-scale convex optimization.
problem Minimizing a strongly convex function with a large sum of smooth convex functions and a simple nonsmooth convex regularizer.
method mS2GD combines deterministic and stochastic gradient steps with mini-batching.
result mS2GD achieves faster convergence and parallelizable implementation.
Diamonds help compute volatility models efficiently.
problem Computing volatility models in forward variance form.
method Application of diamond trees and forests.
result Efficient computation of volatility models.
The paper uses GPR to speed up pricing of GMWB VA with stochastic vol and rate.
problem Pricing and computing Greeks of GMWB VA with stochastic vol and rate.
method Gaussian Process Regression for numerical solution of dynamic control problem.
result GPR significantly speeds up computation with high accuracy.
Optimal rates found for learning with Nyström stochastic gradient methods.
problem Nonparametric regression learning with improved computational efficiency.
method Combination of stochastic gradient methods with Nyström subsampling, allowing multiple passes and mini-batches.
result Derivation of optimal learning rates considering various parameters.
The paper tackles drift identification in Lévy α-stable stochastic systems, proposing a Fourier space approach.
problem Estimating the drift field of a stochastic differential equation driven by Lévy α-stable noise.
method Fourier space approach, parameterizing the drift field using Fourier series, minimizing a loss function with gradients computed via the adjoint method.
result The method is capable of learning drift fields in qualitative and/or quantitative agreement with ground truth fields.
Paper uses NMT to predict solutions to stochastic optimization problems quickly.
problem Predicting solutions to stochastic discrete optimization problems under uncertainty.
method Applied a state-of-the-art NMT algorithm with minimal adaptations and hyperparameter tuning.
result NMT can produce accurate solutions in milliseconds with less variability.
Study on test risk dynamics in learning theory with stochastic gradient flow.
problem Understanding test risk in stochastic gradient flow dynamics.
method Path integral formulation for small learning rates, explicit computation for weak features.
result Explicit corrections due to stochastic term in dynamics, good agreement with simulations.
SPIDER optimizes non-convex problems with reduced gradient computations.
problem Non-convex optimization problems with limited gradient information.
method Stochastic Path-Integrated Differential Estimator (SPIDER) combined with gradient descent.
result SPIDER-SFO and SPIDER-SFO extsuperscript{+} achieve optimal gradient computation costs for non-convex optimization.
Study approximates rough stochastic volatility models using diffusion processes.
problem High computational cost in simulating rough stochastic volatility models.
method Approximates stochastic Volterra equations with an N-dimensional diffusion process.
result Approximations converge strongly with superpolynomial rate in N.
The paper analyzes gradient descent algorithms using stochastic differential equations.
problem Understanding the asymptotic behaviors of gradient descent algorithms in statistical and computational contexts.
method Modeling gradient descent algorithms as stochastic differential equations and applying gradient flow central limit theorems.
result Identifies four factors affecting the local minima found by stochastic gradient descent.
VR-lite improves SGD performance without high memory or full gradient computations.
problem Limited practical use of variance reduction methods in distributed settings.
method VR-lite: A variance reduction method for distributed SGD that requires no full gradient computations or extra storage.
result VR-lite algorithms perform favorably compared to other stochastic methods in both sequential and distributed settings.
Stochastic methods tackle inexact Hessian and gradient computations in large-scale non-convex optimization.
problem Efficiently solving non-convex optimization problems with inexact Hessian and gradient computations.
method Stochastic trust region and cubic regularization methods with inexact gradient, Hessian, and function values.
result Achieves ε-approximate second-order optimality with similar iteration complexity as exact computations.
SVD-based methods reduce computational cost for stochastic systems.
problem High dimensionality and Monte Carlo runs in stochastic systems.
method Extending SVD-based model reduction to stochastic differential equations.
result Preserving symplectic structures improves accuracy and energy conservation.
New method estimates SDE parameters efficiently using WCE and SGD.
problem Parameter estimation for stochastic differential equations.
method Wiener Chaos Expansion and Stochastic Gradient Descent.
result Accurate parameter recovery from noisy observations.
GPU speeds up Monte Carlo simulations for large time steps.
problem Slow convergence and inaccurate solutions with large time steps in Monte Carlo simulations.
method Generalizes the Seven League scheme for GPU acceleration.
result Significantly improved computational speed.