A new family of stochastic dominance orders based on distortion functions.
problem Determining a continuum of dominance relations for risk assessment.
method Introducing H-distorted stochastic dominance, a generalized family of stochastic orders.
result Power-distorted stochastic dominance is particularly appealing due to its simplicity and statistical interpretations.
Proposes SPFB method for optimizing partition functions in stochastic learning.
problem Optimizing partition functions in stochastic learning settings.
method Stochastic Gradient Bound (SPFB) method based on upper-bounding the partition function with a quadratic surrogate.
result Sub-linear convergence rate of SPFB method and efficient training of deep learning models.
New method uses Chebyshev expansions to compute unbiased stochastic gradients for spectral functions.
problem Computing gradients of spectral functions is expensive and challenging.
method Combining randomized trace estimators with Chebyshev expansions for unbiased stochastic gradients.
result Developed methods for optimizing objectives involving spectral-sums with fast and stable convergence.
Paper tackles non-convex inf-projection problems with stochastic optimization.
problem Non-convex and possibly non-smooth inf-projection minimization problems.
method Developed stochastic algorithms for finding (nearly) stationary solutions.
result Established first-order convergence for non-convex inf-projection problems.
Functional approach calculates path probabilities in stochastic motion.
problem Calculating path probabilities in stochastic motion.
method Functional technique applied to derive path probability distribution.
result General formula derived for path probability distribution.
New approach solves utility maximization problems using Delta family.
problem Utility maximization in stochastic control problems.
method Directly solving DP equation with Delta function representation.
result Explicit series representation of value function.
Functional-analytic method for stochastic parallel transport in bundles.
problem Stochastic parallel transport in Hermitian bundles over Riemannian manifolds.
method Purely functional-analytic construction.
result Obtained a general Feynman-Kac formula in vector bundles.
New method solves complex optimization problems with real-time learning.
problem Nonconvex nonsmooth conditional stochastic optimization problems.
method Single time-scale stochastic method with parametric model approximation.
result Method converges with probability one using differential inclusions and Lyapunov function.
New stochastic algorithms solve DC functions and non-convex problems efficiently.
problem Solving non-convex, non-smooth, and non-differentiable functions efficiently.
method Proposed new stochastic optimization algorithms for DC functions and non-convex problems.
result First non-asymptotic convergence for non-convex optimization with general non-convex non-differentiable regularizers.
New q-neurons improve neural network performance.
problem Improving neural network activation functions.
method Introducing q-neurons based on Jackson's q-derivatives with stochastic parameters. result Consistently improved performance over state-of-the-art activation functions.
We generalize stochastic smoothing for gradient estimation of non-differentiable functions.
problem Gradient estimation for non-differentiable functions.
method Developed a general framework for relaxation and gradient estimation of non-differentiable black-box functions using stochastic smoothing with reduced assumptions.
result Empirically validated the effectiveness of variance reduction strategies for various non-differentiable tasks.
Stochastic models analyze traffic network performance.
problem Evaluate traffic system performance.
method Stochastic cell transmission models, preference functionals, Gaussian process regression.
result Illustrated in two case studies.
Solves risk minimization problem with SSD constraints.
problem Finding SSD-minimal quantile function under mixed constraints.
method Explicitly works out SSD-minimal solution and relates to Skorokhod problem.
result Explicit solution to risk minimizing problem.
Paper analyzes error in stochastic approximation for discontinuous functions.
problem Estimating expected error in discontinuous stochastic approximation.
method Uses finite differences and O(n−1/5) error estimate for discontinuous functions. result Achieves error estimate of O(n−1/5) for discontinuous stochastic representation. The paper guarantees global stability for stochastic subgradient methods in nonsmooth nonconvex optimization.
problem Minimizing nonsmooth nonconvex functions with convergence guarantees.
method Developed a framework for stochastic subgradient methods with global stability guarantees.
result Iterates are uniformly bounded and asymptotically stabilize around the stable set of the differential inclusion.
We present a stochastic setting for optimization problems with nonsmooth convex separable objective functions over linear equality constraints. To solve such problems, we propose a stochastic Alternating Direction Method of Multipliers (ADMM) algorithm. Our algorithm applies to a more general class of nonsmooth convex …
AdaGrad-Norm achieves linear convergence for certain functions.
problem Proving linear convergence for specific types of functions.
method Introducing RUIG, a measure of gradient balance; developing a two-stage framework.
result AdaGrad-Norm achieves linear convergence for certain functions.
New algorithm minimizes convex functions with smooth and non-smooth parts.
problem Minimizing convex functions with smooth and non-smooth components.
method Proximal stochastic quasi-Newton method incorporating Hessian and multistage variance reduction.
result Achieves linear rate of convergence.
Deterministic GD can behave stochastically in large learning rates for multiscale functions.
problem Understanding deterministic GD's stochastic behavior in large learning rates for multiscale objectives.
method Established a sufficient condition for deterministic GD to converge to a rescaled Gibbs distribution in large learning rates for multiscale functions.
result Deterministic GD can converge to a statistical distribution in large learning rates for multiscale functions.
SGD method converges to minima for non-convex functions.
problem Non-convex optimization problems in machine learning.
method Stochastic gradient descent method for non-convex objective functions.
result Estimates on the rate of convergence to minima.
Stochastic neural networks use Gaussian noise with monotonic activation functions.
problem Training RBM with non-linearities.
method Laplace approximation with Gaussian noise for monotonic activation functions.
result Exp-RBM learns useful representations using stochastic units.
This is a follow up of our previous paper - Trybuła and Zawisza \cite{TryZaw}, where we considered a modification of a monotone mean-variance functional in continuous time in stochastic factor model. In this article we address the problem of optimizing the mentioned functional in a market with a stochastic interest rat…
Stochastic conditional gradient methods improve optimization for convex and submodular functions.
problem Optimization of large-scale stochastic problems with high-dimensional constraints.
method Proposes averaging technique for gradient approximations and linear programming for descent/ascent directions.
result Achieves optimal or near-optimal guarantees for various submodular maximization problems.
Optimized method tackles convex optimization with heavy-tailed noise.
problem Convex optimization problems with noisy gradients.
method Vanilla stochastic proximal subgradient method without gradient clipping or normalization.
result Achieves optimal complexity for various convex optimization types under heavy-tailed noise.
The article constructs stochastic integration in Riemannian manifolds.
problem No specific problem stated; focuses on the construction of stochastic integration.
method Functional-analytic approach to stochastic integration in Riemannian manifolds.
result There are infinitely many stochastic integrals, and they are related by a simple formula.
Paper proves large deviation principle for stochastic approximations.
problem Asymptotic estimates of learning algorithm deviations.
method Weak convergence approach to large deviations.
result Identifies appropriate scaling sequence and new representation for rate function.
Develops robust methods for infinite-dimensional stochastic processes.
problem Measuring covariations in stochastic evolution equations in infinite dimensions.
method Asymptotic theory for jump robust measurement of covariations.
result Identifies scaling limits for realized covariations.
New method optimizes discrete submodular problems using stochastic gradient descent.
problem Optimizing submodular functions in discrete settings with stochastic elements.
method Projected stochastic gradient ascent for continuous optimization, followed by rounding.
result Achieves optimal approximation guarantees with significantly reduced computational cost.
Stochastic gradient descent's long-term fluctuations are described by a diffusion limit.
problem Long-term behavior of stochastic gradient descent in non-smooth settings.
method Functional central limit theorem applied to rescaled trajectory of SGD.
result Characterization of long-term fluctuations around the minimizer.
New algorithms estimate Hessians using random directions for faster stochastic optimization.
problem Efficiently estimating Hessians for stochastic optimization.
method Generalized Hessian estimators using random directions and noisy function measurements.
result Asymptotically unbiased estimators with lower bias for more measurements.
New algorithm reduces complexity for optimizing complex machine learning tasks.
problem Optimizing complex machine learning objectives like reinforcement learning and portfolio management.
method Developed SARAH-Compositional algorithm using Stochastic Recursive Gradient Descent.
result Achieved optimal IFO complexity bounds for stochastic compositional optimization.
Flow Annealing Posterior Sampling unifies stochastic-process regression and PDE inverse problems.
problem Function-space posterior sampling for stochastic processes and inverse problems.
method Flow Annealing Posterior Sampling (FAPS) using pretrained function-space flow-matching priors.
result Coherent posterior samples with accurate uncertainty quantification.
Optimal strategies in stochastic control problems with two parameters are identified.
problem Optimal control strategies in stochastic processes with two parameters.
method First, parameters are chosen by continuous/smooth fit conditions. Then, optimality is shown using verification arguments.
result Optimal strategies can be concisely expressed via scale functions.
New framework for ranking distributions using variable fractional parameters.
problem Ordering distributions with varying steepness and local non-concavities.
method Introducing a function γ:Ro[0,1] to replace the fixed parameter in fractional SD. result Enables ranking of a broader range of distributions and incorporates dynamic greediness.
Study of accelerated dynamics for convex function minimization with noisy gradients.
problem Minimizing smooth convex functions with noisy gradients.
method Formulate and study continuous-time stochastic dynamics, prove convergence rates.
result Derive estimates of convergence rates for function values, both persistent and asymptotic.
Derives new equations for stochastic volatility models.
problem Modeling local-stochastic-volatility models and their derivatives.
method Conditional forward equation, Dupire stochastic PDE, rolling expiry vanilla option SPDE.
result New equations for LSV models and their derivatives.
SUSTAIN algorithm tackles stochastic bilevel optimization with near-optimal complexity.
problem Stochastic bilevel optimization problems with specific convexity and smoothness properties.
method SUSTAIN algorithm using single-timescale double-momentum stochastic approximation.
result SUSTAIN achieves near-optimal complexity for finding ε-stationary solutions.
In this work we consider the stochastic minimization of nonsmooth convex loss functions, a central problem in machine learning. We propose a novel algorithm called Accelerated Nonsmooth Stochastic Gradient Descent (ANSGD), which exploits the structure of common nonsmooth loss functions to achieve optimal convergence ra…
SGD generalization bounds derived from information theory.
problem Understanding generalization of SGD for non-convex functions.
method Combining information-theoretic bounds with perturbation analysis.
result Upper bounds on SGD's generalization error based on gradient variance and function smoothness.
New algorithm reduces error in regression problems.
problem Minimizing composite objective functions with quadratic and convex components.
method Stochastic dual averaging with constant step-size, proving convergence rate O(1/n).
result Extends least-squares regression to various convex regularizers and geometries.
Universal approximation for stochastic processes using Brownian motion.
problem Approximating stochastic processes with linear functionals.
method Establishing Lp-type universal approximation theorems for rough path spaces. result Linear functionals on the signature of time-extended Brownian motion can approximate any p-integrable stochastic process. Two new algorithms solve privacy-constrained SVI and SSP problems.
problem Privacy-constrained stochastic variational inequality and saddle-point problems.
method Proposed Noisy Stochastic Extragradient (NSEG) and Noisy Inexact Stochastic Proximal Point (NISPP) algorithms.
result Optimal risk bounds for weak gap function with sampling with replacement.
In this paper we investigate general linear stochastic volatility models with correlated Brownian noises. In such models the asset price satisfies a linear SDE with coefficient of linearity being the volatility process. This class contains among others Black-Scholes model, a log-normal stochastic volatility model and H…
Paper introduces cubature method for stochastic Volterra equations.
problem Solving stochastic Volterra integral equations efficiently.
method Derive stochastic Taylor expansion, introduce cubature measure.
result Cubature method is more efficient than Euler scheme under certain conditions.
We consider the problem of minimizing the sum of two convex functions: one is the average of a large number of smooth component functions, and the other is a general convex function that admits a simple proximal mapping. We assume the whole objective function is strongly convex. Such problems often arise in machine lea…
Paper develops a constant step stochastic Douglas-Rachford algorithm for convex minimization.
problem Finding solutions to convex minimization problems with random functions.
method Stochastic Douglas-Rachford algorithm with constant step size.
result Iterates stay close to the solution set with high probability.
New algorithm finds local minima faster than SGD for nonconvex functions.
problem Finding local minima in nonconvex optimization efficiently.
method Stochastic cubic regularization of Newton method.
result Matches best-known result for local minima without acceleration.
Paper analyzes SARAH for nonconvex optimization with mini-batches.
problem Solving nonconvex optimization problems with mini-batches.
method Stochastic Recursive Gradient Algorithm (SARAH) for nonconvex losses.
result Sublinear and linear convergence rates for different types of nonconvex functions.