arXiv research

Review of quantile regression methods for stochastic computer experiments.

Proposes a new method combining Reservoir Computing and Normalizing Flow for predicting stochastic dynamical systems.

We study distributed stochastic convex optimization under the delayed gradient model where the server nodes perform parameter updates, while the worker nodes compute stochastic gradients. We discuss, analyze, and experiment with a setup motivated by the behavior of real-world distributed computation networks, where the…

RES, a regularized stochastic version of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton method is proposed to solve convex optimization problems with stochastic objectives. The use of stochastic gradient descent algorithms is widespread, but the number of iterations required to approximate optimal arguments c…

Enhances SGLD for log-concave posteriors with asynchronous computation.

A new algorithm reduces Hessian computation for nonconvex optimization.

The paper uses machine learning to compute rare event probabilities in stochastic systems.

We present a sparse grid high-order alternating direction implicit (ADI) scheme for option pricing in stochastic volatility models. The scheme is second-order in time and fourth-order in space. Numerical experiments confirm the computational efficiency gains achieved by the sparse grid combination technique.

SVD-based methods reduce computational cost for stochastic systems.

A large class of machine learning techniques requires the solution of optimization problems involving spectral functions of parametric matrices, e.g. log-determinant and nuclear norm. Unfortunately, computing the gradient of a spectral function is generally of cubic complexity, as such gradient descent methods are rath…

Stochastic gradient descent procedures have gained popularity for parameter estimation from large data sets. However, their statistical properties are not well understood, in theory. And in practice, avoiding numerical instability requires careful tuning of key parameters. Here, we introduce implicit stochastic gradien…

Investigates portfolio selection with transaction costs and stochastic volatility, using deep learning for computation.

New method estimates SDE parameters efficiently using WCE and SGD.

NSFs learn SDE transition laws for efficient sampling.

Develops methods to simulate option prices for a specific stochastic volatility model.

Study on stochastic hypergradient computation for machine learning problems.

Improved multilevel scheme for value-at-risk computation.

Improved loss scaling for stochastic momentum algorithms in high dimensions.

New algorithms improve linear bandit performance with low computation.

Improved first-order algorithm for entropy regularized OT with faster convergence.

The paper uses GPR to speed up pricing of GMWB VA with stochastic vol and rate.

SIM-Shapley improves SV approximation efficiency and stability.

As the size of modern data sets exceeds the disk and memory capacities of a single computer, machine learning practitioners have resorted to parallel and distributed computing. Given that optimization is one of the pillars of machine learning and predictive modeling, distributed optimization methods have recently garne…

EigenGame improves eigendecomposition by offering unbiased updates for larger datasets.

We propose a stochastic variance reduced optimization algorithm for solving sparse learning problems with cardinality constraints. Sufficient conditions are provided, under which the proposed algorithm enjoys strong linear convergence guarantees and optimal estimation accuracy in high dimensions. We further extend the …

Improved scheme for option pricing in stochastic volatility models with jumps.

Optimal transport (OT) distances are finding evermore applications in machine learning and computer vision, but their wide spread use in larger-scale problems is impeded by their high computational cost. In this work we develop a family of fast and practical stochastic algorithms for solving the optimal transport probl…

This paper uses Malliavin calculus to price and compute delta of financial derivatives in jump-diffusion models.

We revisit the stochastic limited-memory BFGS (L-BFGS) algorithm. By proposing a new framework for the convergence analysis, we prove improved convergence rates and computational complexities of the stochastic L-BFGS algorithms compared to previous works. In addition, we propose several practical acceleration strategie…

Paper discusses new stochastic algorithms for sparse signal recovery.

A new algorithm solves constrained optimization problems with stochastic gradients.

This work discusses AAD for financial model calibration and its parallelization benefits.

Modern large scale machine learning applications require stochastic optimization algorithms to be implemented on distributed computational architectures. A key bottleneck is the communication overhead for exchanging information such as stochastic gradients among different workers. In this paper, to reduce the communica…

Proposes a method to reduce parallel complexity of MLMC in SGD.

A new algorithm speeds up deep learning training by decoupling computation and communication.

New method improves stochastic kriging for high-dimensional simulations.

In this paper, we present an online adaptive PCA algorithm that is able to compute the full dimensional eigenspace per new time-step of sequential data. The algorithm is based on a one-step update rule that considers all second order correlations between previous samples and the new time-step. Our algorithm has O(n) co…

FSGLD improves federated data sampling by correcting noisy gradients.

New method improves parameter estimation in complex stochastic models.

We propose a numerical algorithm for backward stochastic differential equations based on time discretization and trigonometric wavelets. This method combines the effectiveness of Fourier-based methods and the simplicity of a wavelet-based formula, resulting in an algorithm that is both accurate and easy to implement. F…

Stochastic momentum methods trade compute efficiency for serial runtime.

Method learns SDEs from one trajectory using GP priors and randomized cross-validation.

Method extracts governing laws from non-Gaussian stochastic systems data.

Proposes a new method for optimizing large-scale models using Nyström approximation of the Hessian.

New MLMC method reduces evidence estimation cost.

We propose a randomized block-coordinate variant of the classic Frank-Wolfe algorithm for convex optimization with block-separable constraints. Despite its lower iteration cost, we show that it achieves a similar convergence rate in duality gap as the full Frank-Wolfe algorithm. We also show that, when applied to the d…

Hessian-free (HF) optimization has been successfully used for training deep autoencoders and recurrent networks. HF uses the conjugate gradient algorithm to construct update directions through curvature-vector products that can be computed on the same order of time as gradients. In this paper we exploit this property a…

The paper analyzes the variance of different shuffling methods in stochastic gradient descent.