Algorithm samples constrained stochastic differential equations.
problem Sampling stochastic differential equations with complex constraints.
method Pathspace Metropolis-adjusted manifold sampling.
result Demonstrated effectiveness in various constrained conditions.
New method efficiently computes gradients for stochastic differential equations.
problem Computing gradients for stochastic differential equations efficiently.
method Generalized adjoint sensitivity method to stochastic differential equations.
result Time-efficient and memory-efficient computation of gradients with high-order solvers.
New algorithm optimizes nonlinear SDEs online with convergence guarantees.
problem Optimizing nonlinear stochastic differential equations (SDEs) is computationally challenging.
method Forward propagation algorithm that solves an SDE derived using forward differentiation.
result Convergence theorem for nonlinear dissipative SDEs with bounds on stochastic fluctuations.
Develops DP-SCD for stochastic coordinate descent, making it differentially private.
problem Privacy leak in auxiliary information during stochastic coordinate descent training.
method Develops DP-SCD, leveraging independent noise addition and decoupling/parallelizing coordinate updates.
result Demonstrates competitive performance against DP-SGD with less tuning.
Deep fictitious play converges to Nash equilibrium in stochastic differential games.
problem Finding Nash equilibrium in large stochastic differential games.
method Decouples the game into sub-optimization problems and solves each player's optimal strategy with deep BSDE method.
result Deep fictitious play converges to the true Nash equilibrium.
Improved method for estimating derivatives of discontinuous functions using stochastic algorithmic differentiation and regression.
problem High Monte-Carlo error in finite difference approximation of discontinuous functions.
method Combining stochastic algorithmic differentiation and regression to estimate derivative of expectations of discontinuous functions.
result Reduction in Monte-Carlo error through decoupling integration of Dirac delta and conditional expectation.
We generalize stochastic smoothing for gradient estimation of non-differentiable functions.
problem Gradient estimation for non-differentiable functions.
method Developed a general framework for relaxation and gradient estimation of non-differentiable black-box functions using stochastic smoothing with reduced assumptions.
result Empirically validated the effectiveness of variance reduction strategies for various non-differentiable tasks.
Develops mathematical framework for analyzing stochastic gradient algorithms.
problem Analyzing the dynamics of stochastic gradient algorithms.
method Stochastic modified equations (SME) framework to approximate stochastic gradient algorithms as stochastic differential equations.
result Proves that the approximation leads to precise results on SGD, momentum SGD, and Nesterov's accelerated gradient method.
Paper addresses privacy and robustness in stochastic linear bandits.
problem Stochastic linear bandits with differential privacy and adversarial robustness.
method Logarithmic batch queries, arm elimination algorithm, two privacy models.
result First algorithms providing differential privacy and adversarial robustness.
Two new algorithms solve privacy-constrained SVI and SSP problems.
problem Privacy-constrained stochastic variational inequality and saddle-point problems.
method Proposed Noisy Stochastic Extragradient (NSEG) and Noisy Inexact Stochastic Proximal Point (NISPP) algorithms.
result Optimal risk bounds for weak gap function with sampling with replacement.
Quantum algorithms speed up financial model calculations.
problem Computing financial model expectations efficiently.
method Quantum-accelerated multilevel Monte Carlo methods.
result Improved speed-up for financial model calculations.
New framework for DP-SMO with near-optimal privacy-loss trade-off.
problem Optimal trade-off between privacy and population loss in DP-SMO.
method General framework using Phased-ERM method and black-box optimization.
result Near-linear time algorithms with near-optimal guarantees.
Paper solves complex control problems using novel SDEs.
problem Solving stochastic differential games for nonlinear systems.
method Uses Deep Forward-Backward SDEs with neural networks.
result Numerical solution validated on two example systems.
A new method for stochastic optimization using virtual gradients.
problem Stochastic optimization challenges in computational efficiency and memory usage.
method Inspired by dynamic programming, SVGD uses a computational graph and automatic differentiation for efficient optimization.
result Experimental results show SVGD outperforms other methods on multiple datasets and network models.
Paper improves privacy in SGD with low noise, achieving optimal risk rates.
problem Privacy-preserving machine learning with good performance.
method Differentially private SGD with low-noise analysis.
result Achieves optimal excess risk rates for non-smooth losses.
New stochastic algorithms solve DC functions and non-convex problems efficiently.
problem Solving non-convex, non-smooth, and non-differentiable functions efficiently.
method Proposed new stochastic optimization algorithms for DC functions and non-convex problems.
result First non-asymptotic convergence for non-convex optimization with general non-convex non-differentiable regularizers.
New algorithms for approximating stochastic processes efficiently.
problem Finding accurate finite approximations for stochastic processes.
method Develops new algorithms and fast implementations for approximating stochastic processes.
result Efficient approximations for stochastic processes can be found.
New method optimizes SDE models using continuous-time gradient descent.
problem Optimizing over the stationary distribution of SDE models.
method Continuous-time stochastic gradient descent for SDE models.
result Asymptotic convergence to the direction of steepest descent.
New algorithm reduces privacy loss in SGD without learning rate tuning.
problem Locally differentially private stochastic optimization with high privacy loss.
method BANCO (Betting Algorithm for Noisy COins) for ε ε ε -LDP SGD. result Matches convergence rate of tuned SGD without learning rate tuning.
New deep learning method solves complex BSDEs efficiently.
problem Solving high-dimensional nonlinear BSDEs.
method Reformulate as global optimization, approximate solution with deep neural network, globally minimize quadratic local loss functions.
result Demonstrated effectiveness on various high-dimensional nonlinear BSDEs, including finance applications.
We develop the method of stochastic modified equations (SME), in which stochastic gradient algorithms are approximated in the weak sense by continuous-time stochastic differential equations. We exploit the continuous formulation together with optimal control theory to derive novel adaptive hyper-parameter adjustment po…
New estimator for SDEs is shown to be an adjoint state method.
problem Estimating gradients for overparameterized SDEs efficiently.
method Demonstrates generator gradient estimator as an adjoint state method.
result Generator gradient estimator is an adjoint state method for SDEs.
New algorithm solves stochastic optimization problems with unknown gradients.
problem Solving nonlinear optimization problems with stochastic objectives and deterministic constraints.
method Adaptive SQP with differentiable exact augmented Lagrangian and stochastic line search.
result Global convergence established for both non-adaptive and adaptive SQP methods.
A new algorithm solves high-dimensional nonlinear BSDEs efficiently.
problem Solving high-dimensional nonlinear backward stochastic differential equations (BSDEs).
method Transformed BSDE into a differential deep learning problem using Malliavin calculus. Discretized integrals using Euler-Maruyama method. Approximated solution with three deep neural networks. Optimized parameters using a differential learning loss function.
result Our algorithm is more accurate and faster than other methods.
New algorithm closes empirical gap in PFSGD performance.
problem Empirical performance gap between tuned SGD and PFSGD.
method Parameter-free algorithm based on Coin-Betting ODE updates.
result New algorithm outperforms tuned baselines and matches optimal performance.
Neural networks model financial data with Lévy processes.
problem Forecasting chaotic financial time series with big jumps.
method Lévy-induced stochastic differential equation network approximated by neural networks.
result The method improves prediction accuracy using non-Gaussian Lévy processes.
Data that is gathered adaptively --- via bandit algorithms, for example --- exhibits bias. This is true both when gathering simple numeric valued data --- the empirical means kept track of by stochastic bandit algorithms are biased downwards --- and when gathering more complicated data --- running hypothesis tests on c…
We propose a numerical algorithm for backward stochastic differential equations based on time discretization and trigonometric wavelets. This method combines the effectiveness of Fourier-based methods and the simplicity of a wavelet-based formula, resulting in an algorithm that is both accurate and easy to implement. F…
The paper develops a deep signature approach for option pricing under non-Markovian stochastic volatility models.
problem Pricing options under non-Markovian stochastic volatility models is challenging due to the dependence on historical paths.
method Reformulate the asset dynamics as a rough stochastic differential equation and represent rough paths via signatures. Apply standard analytical tools to solve the transformed equation.
result The deep signature approach provides a theoretically grounded and computationally efficient framework for option pricing.
Paper develops a consistent algorithm for learning graph structure from continuous-time stochastic differential equations.
problem Learning structure from continuous-time stochastic differential equations.
method Score-based structure learning using Neural Ordinary Differential Equations with adaptive regularization.
result The method consistently recovers directed graphs of local independencies in systems of stochastic differential equations.
New algorithm solves complex equations using deep learning.
problem High-dimensional nonlinear PDEs and BSDEs.
method Iterated time discretization, deep neural networks, stochastic gradient descent.
result Increased accuracy and reduced complexity compared to existing methods.
Paper tackles good arm identification in stochastic bandits.
problem Identifying good arms with minimal samples.
method Proposes DGAI, a differentiable algorithm to improve sample complexity.
result DGAI outperforms baseline algorithms in synthetic and real-world datasets.
Private algorithms minimize population loss with optimal rate.
problem Private optimization of convex functions with stochastic samples.
method Differentially private algorithms based on algorithmic stability.
result Optimal rate of 1 / n 1/\sqrt{n} 1/ n for population loss. Deep learning model solves high-dimensional PDEs using Actor-Critic approach.
problem Solving high-dimensional nonlinear PDEs efficiently.
method Reformulated PDE into BSDE system, inspired by Actor-Critic algorithm for deep RL.
result Improved model with fewer parameters, faster convergence, and less hyperparameter tuning.
Efficient private algorithms for estimating block models and mixture models.
problem Estimating block models and mixture models in high-dimensional settings.
method General tools for designing efficient private estimation algorithms.
result First efficient private algorithms for weak and exact recovery of stochastic block models.
This paper investigates asymptotic behaviors of gradient descent algorithms (particularly accelerated gradient descent and stochastic gradient descent) in the context of stochastic optimization arising in statistics and machine learning where objective functions are estimated from available data. We show that these alg…
New method reveals insights about stochastic optimization methods using modified equations.
problem Understanding the qualitative behavior of stochastic optimization algorithms.
method Developed a class of stochastic differential equations to approximate the dynamics of stochastic optimization methods.
result Mean-square stability of the modified equation provides qualitative insights about stochastic coordinate descent.
New methods improve neural network training by sampling from target distributions.
problem Optimization procedures like SGD drive parameters to local minima.
method Thermodynamic parameterization using discretized stochastic differential equations.
result Partitioned numerical algorithms converge faster and more robustly.
Develops a mathematical model for automatic differentiation in machine learning.
problem Current automatic differentiation lacks a simple mathematical model for machine learning.
method Articulates relationships between program differentiation and nonsmooth functions, provides a class of functions and nonsmooth calculus.
result Shows how nonsmooth calculus applies to stochastic approximation methods and evidence of artificial critical points.
New algorithm for privacy-preserving nonconvex optimization.
problem Privacy-preserving nonconvex empirical risk minimization.
method Differentially private stochastic gradient descent algorithm.
result Achieves strong privacy guarantees efficiently with improved utility.
Unified framework connects stochastic optimization to Bayesian inference.
problem Stochastic optimization algorithms and their theoretical underpinnings.
method Latent variational problem and Forward Backward Stochastic Differential Equations (FBSDE).
result Recovery of various adaptive stochastic gradient descent methods.
New approach achieves optimal rates for differentially private stochastic convex optimization with heavy-tailed gradients.
problem Differentially private stochastic convex optimization with heavy-tailed gradients.
method Reduction-based approach to achieve optimal rates.
result Achieved optimal rates up to logarithmic factors, nearly matching a lower bound.
We present a new algorithm for stochastic variational inference that targets at models with non-differentiable densities. One of the key challenges in stochastic variational inference is to come up with a low-variance estimator of the gradient of a variational objective. We tackle the challenge by generalizing the repa…
Quantum algorithm samples from SDEs using DQCs and quantile mechanics.
problem Sampling from solutions of stochastic differential equations.
method Differentiable quantum circuits (DQCs) encoding latent variables, quantile mechanics.
result Quantum algorithm generates time-series from SDEs.
Deep learning theory for Nash equilibrium in stochastic games.
problem Computing Nash equilibrium in non-zero-sum stochastic differential games.
method Fictitious play applied to deep neural networks for solving N N N -player optimization problems. result Deep learning algorithm converges to open-loop Nash equilibrium under appropriate assumptions.
Two deep learning algorithms solve utility maximisation problems in finance.
problem Solving utility maximisation problems in finance with deep learning.
method Two algorithms: one for Markovian problems via HJB equation and 2BSDE, the other for non-Markovian problems via adjoint BSDE.
result Highly accurate results with low computational cost, solving problems with power, log, and non-HARA utilities in various models.
New algorithms for differentially private optimization in convex and non-convex settings with near-optimal rates.
problem Differentially private optimization in convex and non-convex settings.
method Developed algorithms for convex and non-convex settings with near-optimal excess population risk.
result Achieved near-optimal rates in near-linear time for convex settings and nearly dimension independent rates for non-convex settings.
Deep learning approximates SPDE solutions from noise trajectories.
problem Approximating solutions to stochastic partial differential equations (SPDEs).
method Uses neural networks to approximate SPDE solutions based on noise realizations.
result Accurately estimates SPDE solutions and functionals like mean and variance.