Paper corrects and expands stochastic Lie systems theory.
problem Stochastic Lie systems and their properties.
method Corrected stochastic Lie theorem, introduced new stochastic Lie systems.
result Stochastic Lie systems can differ significantly between Stratonovich and Itô approaches.
The study analyzes stochastic Lie systems and their applications in various models.
problem Analyzing stochastic differential equations on manifolds.
method Coalgebra method for Hamiltonian stochastic Lie systems.
result New examples of stochastic Lie systems and Hamiltonian stochastic Lie systems are analyzed.
sFML learns stochastic dynamical systems from data.
problem Learning unknown stochastic dynamical systems from measurement data.
method sFML extends FML for deterministic systems, using a stochastic flow map composed of deterministic and stochastic sub-maps.
result sFML constructs a stochastic evolution model approximating unknown stochastic systems.
The existence of stationary Markov perfect equilibria in stochastic games is shown under a general condition called "(decomposable) coarser transition kernels". This result covers various earlier existence results on correlated equilibria, noisy stochastic games, stochastic games with finite actions and state-independe…
Bayesian neural networks can be partially stochastic without losing predictive power.
problem The necessity of fully stochastic parameters in Bayesian neural networks.
method Theoretical and empirical investigation of partially stochastic networks compared to fully stochastic ones.
result Expressive predictive distributions require only small amounts of stochasticity, and partially stochastic networks can match or outperform fully stochastic networks.
Stochastic approximation algorithms show exponential progress bounds.
problem Analyzing the convergence of stochastic approximation algorithms.
method Developed geometric ergodicity proofs to establish exponential concentration bounds.
result Proved faster convergence rates for specific algorithms.
Stochastic gradient methods can converge in expectation under heavy-tailed noise.
problem Convergence of stochastic gradient methods under heavy-tailed noise.
method Comprehensive study of stochastic optimization under heavy-tailed noise for extsfSGD, extsfSMD, extsfASMD, extsfSGDM in convex and nonconvex optimization. result Established in-expectation convergence results for various stochastic gradient methods.
Study on stochastic mean curvature flow on networks using Ito calculus.
problem Understanding the dynamics of network structures under random influences.
method Application of Ito calculus to derive a stochastic differential equation (SDE) for network edges.
result New insights into the stability, long-term behavior, and pattern formation of complex networks under stochastic influences.
New method reveals insights about stochastic optimization methods using modified equations.
problem Understanding the qualitative behavior of stochastic optimization algorithms.
method Developed a class of stochastic differential equations to approximate the dynamics of stochastic optimization methods.
result Mean-square stability of the modified equation provides qualitative insights about stochastic coordinate descent.
New method uses backward SDEs for deep learning uncertainty.
problem Uncertainty quantification in deep learning models.
method Probabilistic machine learning with stochastic neural networks and stochastic optimal control.
result Effectiveness validated through numerical experiments.
The article reviews how to set stochastic volatility model parameters.
problem Choosing parameters for stochastic volatility models.
method Examines existing literature on various methods.
result Different approaches to setting stochastic volatility parameters.
We extend Dupire's formula for stochastic interest rates and local volatility.
problem Deriving formulas for stochastic interest rates and local volatility.
method Generalizations of Dupire's formula for stochastic drift and local volatility.
result Validated the limits of the generalized Dupire formulae for specific cases.
This paper studies a non-stochastic version of Fernholz's stochastic portfolio theory for a simple model of stock markets with continuous price paths. It establishes non-stochastic versions of the most basic results of stochastic portfolio theory and discusses connections with Stroock-Varadhan martingales.
We develop the mathematical foundations of the stochastic modified equations (SME) framework for analyzing the dynamics of stochastic gradient algorithms, where the latter is approximated by a class of stochastic differential equations with small noise parameters. We prove that this approximation can be understood math…
New theorem handles stochastic Volterra semimartingales.
problem Classic Fubini theorem restrictions for Volterra semimartingales.
method Introduced measure-valued stochastic integration.
result Proved new stochastic Fubini theorem.
Proposes a new method combining Reservoir Computing and Normalizing Flow for predicting stochastic dynamical systems.
problem Predicting and capturing long-term behaviors of stochastic dynamical systems.
method Data-driven framework combining Reservoir Computing and Normalizing Flow, integrating error modeling and both approaches virtues.
result Successfully predicts the long-term evolution of stochastic dynamical systems and replicates dynamical behaviors.
Proposes new Monte Carlo methods for calibrating local volatility models with stochastic components.
problem Calibrating local volatility models with stochastic drift and diffusion.
method Developed Monte Carlo algorithms for three models: local volatility with stochastic interest rates, stochastic local volatility with deterministic interest rates, and stochastic local volatility with stochastic interest rates.
result Conditions for the existence of local volatility given European option prices, stochastic interest rate model parameters, and correlations.
We introduce a stochastic model for noisy vector fields on manifolds.
problem Noisy vector fields violate the assumption of parallel transport in stochastic analysis.
method We define a stochastic Lie bracket that induces torsion and analyze its consequences.
result The stochastic Lie bracket induces torsion in expectation.
New dynamics for SGD in small learning rate regime.
problem Improving stochastic gradient descent in small learning rate regime.
method Introducing stochastic modified flows and distribution dependent stochastic modified flows.
result Captures fluctuating dynamics of SGD in small learning rate - infinite width scaling regime.
Survey of methods for solving smooth stochastic variational inequalities.
problem Solving smooth (strongly) monotone stochastic variational inequalities.
method Deterministic foundation, general stochastic formulation, finite sum setup, recent advances.
result Review of various methods for solving smooth stochastic variational inequalities.
This paper provides a unifying theoretical framework for stochastic optimization algorithms by means of a latent stochastic variational problem. Using techniques from stochastic control, the solution to the variational problem is shown to be equivalent to that of a Forward Backward Stochastic Differential Equation (FBS…
The paper connects higher order risk measures and stochastic dominance, showing their equivalence and integrating them with optimization.
problem Comparing and characterizing random outcomes in risk assessment.
method Exploring the equivalence between higher order risk measures and stochastic dominance, using stochastic optimization and expectiles as examples.
result Higher order risk measures and stochastic dominance are equivalent and can be used to characterize random outcomes.
In this paper, we propose a novel technique to implement stochastic gradient methods, which are beneficial for learning from large datasets, through accelerated stochastic dynamics. A stochastic gradient method is based on mini-batch learning for reducing the computational cost when the amount of data is large. The sto…
Algorithm samples constrained stochastic differential equations.
problem Sampling stochastic differential equations with complex constraints.
method Pathspace Metropolis-adjusted manifold sampling.
result Demonstrated effectiveness in various constrained conditions.
New algorithm solves stochastic optimization problems with unknown gradients.
problem Solving nonlinear optimization problems with stochastic objectives and deterministic constraints.
method Adaptive SQP with differentiable exact augmented Lagrangian and stochastic line search.
result Global convergence established for both non-adaptive and adaptive SQP methods.
Paper uses second-order differential geometry to study stochastic mechanics.
problem Stochastic differential equations and their symmetries.
method Develops second-order differential geometry to study symmetries of SDEs and constructs stochastic mechanics.
result Establishes stochastic Lagrangian and Hamiltonian mechanics and their relations with HJB equations.
We develop the method of stochastic modified equations (SME), in which stochastic gradient algorithms are approximated in the weak sense by continuous-time stochastic differential equations. We exploit the continuous formulation together with optimal control theory to derive novel adaptive hyper-parameter adjustment po…
We develop a family of reformulations of an arbitrary consistent linear system into a stochastic problem. The reformulations are governed by two user-defined parameters: a positive definite matrix defining a norm, and an arbitrary discrete or continuous distribution over random matrices. Our reformulation has several e…
Bayesian inference using stochastic neural networks ensembles.
problem Approximating Bayesian posterior distributions.
method Formulate stochastic ensembles of neural networks, train with variational inference, and evaluate using Monte Carlo dropout.
result Stochastic ensembles provide more accurate posterior estimates than other methods.
High-probability bound for distributed stochastic approximation tracking error.
problem Analyzing the convergence of distributed stochastic approximation schemes.
method Analysis using ODE approach to stochastic approximation.
result High probability bound for tracking error between iterates and limiting differential equation.
Simplified calculus for semimartingales makes complex transformations easier.
problem Complex transformations of semimartingales.
method Unified treatment of transformations for real and complex semimartingales.
result Unified calculus for semimartingales simplifies various transformations.
Stochastic gradient descent on manifolds improves low-rank approximation.
problem Efficiently approximate large matrices with lower rank.
method Stochastic gradient descent on a manifold.
result Algorithm outperforms Euclidean space methods on Netflix Prize data.
In this paper, we consider multi-stage stochastic optimization problems with convex objectives and conic constraints at each stage. We present a new stochastic first-order method, namely the dynamic stochastic approximation (DSA) algorithm, for solving these types of stochastic optimization problems. We show that DSA c…
Neural networks can approximate complex stochastic equations well.
problem Approximating general stochastic differential equations.
method Identified neural network classes approximating continuous functions.
result Neural stochastic differential equations can approximate general stochastic differential equations arbitrarily well.
SON learns SPDE solutions and uncertainty from noisy data.
problem Uncertainty quantification in SPDEs with unknown model uncertainties.
method Combining DeepONet and SNNs, SON models stochasticity and predicts uncertainty.
result SON accurately captures solution structure and quantifies predictive uncertainty.
We show that asymptotically, completely asynchronous stochastic gradient procedures achieve optimal (even to constant factors) convergence rates for the solution of convex optimization problems under nearly the same conditions required for asymptotic optimality of standard stochastic gradient procedures. Roughly, the n…
New algorithm improves on EM for streaming data, outperforming existing methods.
problem Processing high-volume, streaming data efficiently.
method Incremental stochastic Majorization-Minimization (MM) algorithm.
result The algorithm converges to a stationary point with vanishing gradient.
Pathwise uniqueness shown for specific stochastic equations.
problem Stochastic Volterra equations with singular kernels and Hölder coefficients.
method Established pathwise uniqueness through Hölder continuity of coefficients.
result Pathwise uniqueness and existence of unique strong solutions.
Paper develops efficient methods for estimating Hessian inverses in stochastic optimization.
problem Estimating the inverse Hessian for convex function minimization.
method Robbins-Monro procedure for recursive estimation of the inverse Hessian.
result Develops universal stochastic Newton methods with improved efficiency.
Stochastic encoders outperform deterministic ones in 'perfect perceptual quality'.
problem Understanding when stochastic encoders outperform deterministic ones.
method Provided a toy example to illustrate performance.
result Stochastic encoders can significantly outperform deterministic ones in 'perfect perceptual quality'.
Entropy-minimal measure calculated for a stochastic volatility model.
problem Calculating the entropy-minimal equivalent martingale measure in a stochastic volatility model.
method Revised related theory, calculated entropy-minimal measure.
result Entropy-minimal measure for the exponential Ornstein-Uhlenbeck model.
Improves understanding of stochastic NGVI convergence rates.
problem Lack of knowledge about non-asymptotic convergence rates in stochastic NGVI.
method Proved non-asymptotic convergence rates for conjugate likelihoods and showed implicit optimization for non-conjugate likelihoods.
result First O(T1) non-asymptotic convergence rate for stochastic NGVI in conjugate likelihoods. Optimizes control of infectious disease spread using stochastic methods.
problem Optimizing control of highly infectious diseases like COVID-19.
method Reformulated Hamilton-Jacobi-Bellman equation as stochastic minimum principle, leading to forward-backward stochastic differential equations.
result Numerous numerical solutions presented under various scenarios.
The paper derives the QGS equations using stochastic central extensions.
problem Deriving the viscous quasi-geostrophic equations on the torus.
method Central extensions of Lie groups and Lie algebras, stochastic Lagrangian formulation, and Euler-Poincaré reduction.
result Stochastic perturbations to the central extension lead to solutions of the QGS equations.
In this paper we study several classes of stochastic optimization algorithms enriched with heavy ball momentum. Among the methods studied are: stochastic gradient descent, stochastic Newton, stochastic proximal point and stochastic dual subspace ascent. This is the first time momentum variants of several of these metho…
A new method improves stochastic gradient descent for faster and more efficient estimation.
problem Efficient and fast parametric estimation methods.
method Projected stochastic gradient descent corrected by Fisher scoring.
result The method is faster and more efficient than traditional methods.
The paper generalizes the construction by stochastic flows of consistent utility processes introduced by M. Mrad and N. El Karoui in (2010). The utilities random fields are defined from a general class of processes denoted by $\GX$. Making minimal assumptions and convex constraints on test-processes, we construct by co…
Study approximates rough stochastic volatility models using diffusion processes.
problem High computational cost in simulating rough stochastic volatility models.
method Approximates stochastic Volterra equations with an N-dimensional diffusion process.
result Approximations converge strongly with superpolynomial rate in N.