Entropy-minimal measure calculated for a stochastic volatility model.
problem Calculating the entropy-minimal equivalent martingale measure in a stochastic volatility model.
method Revised related theory, calculated entropy-minimal measure.
result Entropy-minimal measure for the exponential Ornstein-Uhlenbeck model.
Solves risk minimization problem with SSD constraints.
problem Finding SSD-minimal quantile function under mixed constraints.
method Explicitly works out SSD-minimal solution and relates to Skorokhod problem.
result Explicit solution to risk minimizing problem.
Stochastic Gradient Descent prefers minimizers with flat basins in nonconvex problems.
problem Understanding why SGD prefers minimizers with flat basins in nonconvex problems.
method Detailed analysis of a generic stochastic quadratic problem, deriving a deterministic mechanism.
result Derives a deterministic mechanism explaining why SGD prefers flat minimizers.
Optimizes CM for stochastic convex optimization with progressive precision.
problem Stochastic nature of objective function in convex optimization.
method Iterative coordinate minimization with optimal precision control.
result Order-optimal regret performance for strongly convex and nonsmooth functions.
Optimal bounds for exp-concave stochastic minimization in terms of effective dimension.
problem Finding optimal statistical and computational complexity for exp-concave stochastic minimization.
method Derives optimal bounds using effective dimension and sketching techniques.
result Reveals connections between algorithmic stability and ridge leverage scores.
Stochastic heavy ball method achieves linear convergence for general loss minimization.
problem Minimizing generalization error in machine learning models.
method SGD steps with heavy ball momentum, focusing on expected loss, not finite-sum minimization.
result Established the first linear convergence result for the stochastic heavy ball method.
We assume that an individual invests in a financial market with one riskless and one risky asset, with the latter's price following a diffusion with stochastic volatility. In the current financial market especially, it is important to include stochastic volatility in the risky asset's price process. Given the rate of c…
Many machine learning algorithms minimize a regularized risk, and stochastic optimization is widely used for this task. When working with massive data, it is desirable to perform stochastic optimization in parallel. Unfortunately, many existing stochastic optimization algorithms cannot be parallelized efficiently. In t…
In this work we consider the stochastic minimization of nonsmooth convex loss functions, a central problem in machine learning. We propose a novel algorithm called Accelerated Nonsmooth Stochastic Gradient Descent (ANSGD), which exploits the structure of common nonsmooth loss functions to achieve optimal convergence ra…
Total variation minimization clusters partially labeled data points.
problem Clustering partially labeled data points in stochastic block models.
method Total variation minimization as a clustering method.
result Total variation minimization allows for accurate clustering under certain model parameters.
New approach uses SGLD to minimize CVaR for portfolio weights.
problem Minimizing CVaR for portfolio weights with complete theoretical guarantees.
method Stochastic Gradient Langevin Dynamics (SGLD) with discontinuous updating.
result Theoretical guarantees for convergence in Wasserstein distances for convex and non-convex functions.
Stochastic gradient descent outperforms traditional force-directed methods.
problem Improving graph layout quality and efficiency.
method Applying stochastic gradient descent for stress minimization.
result Stochastic gradient descent is simpler and more robust than traditional methods.
Unified analysis of stochastic ADMM variants via SME.
problem Analyzing and optimizing stochastic ADMM variants for machine learning.
method Unified mathematical framework of SME for continuous-time analysis.
result Dynamics of stochastic ADMM approximated by SDEs with small noise.
We consider a composite convex minimization problem associated with regularized empirical risk minimization, which often arises in machine learning. We propose two new stochastic gradient methods that are based on stochastic dual averaging method with variance reduction. Our methods generate a sparser solution than the…
Paper develops efficient methods for estimating Hessian inverses in stochastic optimization.
problem Estimating the inverse Hessian for convex function minimization.
method Robbins-Monro procedure for recursive estimation of the inverse Hessian.
result Develops universal stochastic Newton methods with improved efficiency.
This study explains how different training methods affect the minimizer of neural networks.
problem How training methods influence the minimizer of neural networks.
method Explains how initialization size, adaptive optimization (AdaGrad), and stochastic mini-batch training affect the minimizer.
result Different training methods lead to different minimizers, even in overparameterized networks.
New algorithm minimizes worst-case regret in uncertain, time-varying dynamics.
problem Model-based policy learning in uncertain, time-varying dynamics.
method Planning regret metric and iterative algorithm for minimizing it.
result Empirical evidence shows the proposed algorithm outperforms existing methods.
Paper develops a constant step stochastic Douglas-Rachford algorithm for convex minimization.
problem Finding solutions to convex minimization problems with random functions.
method Stochastic Douglas-Rachford algorithm with constant step size.
result Iterates stay close to the solution set with high probability.
Algorithm minimizes risk for multiclass classification of stochastic diffusion paths.
problem Multiclass classification of stochastic diffusion paths with distinct drift functions.
method Empirical risk minimization using L2 risk.
result Achieves fast rates of convergence under margin assumption.
We develop a family of accelerated stochastic algorithms that minimize sums of convex functions. Our algorithms improve upon the fastest running time for empirical risk minimization (ERM), and in particular linear least-squares regression, across a wide range of problem settings. To achieve this, we establish a framewo…
Derives conditions for no arbitrage in financial markets with stochastic or diffusion models.
problem Existence and absence of arbitrage in financial markets with stochastic or diffusion models.
method Integral tests, martingale and strict local martingale properties of stochastic exponentials, Markov switching models.
result Conditions for the existence of minimal martingale measure and its preservation under Markov switching.
Adaptive machine learning framework for optimizing with changing data.
problem Optimizing machine learning models with changing data.
method Adaptive selection of samples using estimates of minimizer change and optimization algorithm properties.
result Excess risk is controlled to ensure model performance.
Algorithm improves stochastic gradient optimization with normalized steps.
problem Stochastic and finite sum minimization problems.
method Trust region algorithm with normalized steps.
result Algorithm converges similarly to traditional stochastic gradient under certain conditions.
Paper tackles non-convex inf-projection problems with stochastic optimization.
problem Non-convex and possibly non-smooth inf-projection minimization problems.
method Developed stochastic algorithms for finding (nearly) stationary solutions.
result Established first-order convergence for non-convex inf-projection problems.
Katyusha X adds momentum to SVRG for faster non-convex optimization.
problem Minimizing sum-of-nonconvex functions in machine learning.
method Adding momentum to SVRG method.
result Provable accelerated stochastic algorithm for sum-of-nonconvex functions.
New method solves stochastic optimization problems with affine constraints.
problem Stochastic optimization problems with affine constraints.
method Stochastic Frank-Wolfe method.
result Guarantees convergence rates for objective residual and feasibility gap.
Paper shows faster convergence to local-minimizers in over-parametrized models under interpolation-like conditions.
problem Escaping saddle-points in over-parametrized models.
method Stochastic and deterministic optimization algorithms under interpolation-like conditions.
result Oracle complexity of PSGD and SCRN algorithms to reach ε ε ε -local-minimizer matches or improves upon deterministic rates. Optimizes reinsurance and investment strategies to minimize ruin probability.
problem Optimizing reinsurance and investment strategies to minimize ruin probability.
method Stochastic projected gradient method based on Malliavin calculus.
result Effectiveness of the proposed method demonstrated through numerical experiments.
New algorithm adapts to unknown smoothness in stochastic bandits with polynomial cost.
problem Adapting to unknown smoothness in stochastic bandits.
method Reconsidered Locatelli and Carpentier's lower bound, defined admissible rate functions, and developed a new algorithm.
result New algorithm matches minimal rate functions and provides polynomial cost of adaptation.
New method gives high confidence bounds for stochastic convex optimization with minimal overhead.
problem Rare high probability guarantees in stochastic convex optimization.
method ProxBoost algorithm combining robust distance estimation and proximal point method.
result Wide class of stochastic optimization algorithms can achieve high confidence bounds with logarithmic and polylogarithmic overhead.
Two SGD-like algorithms reduce memory usage for stochastic optimization with infinite data.
problem Optimizing with infinite data sets resulting from random noise.
method Proposes SSAG and S-SAGA algorithms for expected risk minimization.
result SSAG has faster convergence rate than SGD with comparable space requirement.
The study proves theorems about minimal and H-surfaces in 3D space.
problem Characterizing minimal and H-surfaces in 3D space.
method Analytical proofs and stochastic methods.
result Conditions for minimal and H-surfaces in specific spaces.
PASTIS selects minimal models from stochastic dynamics data.
problem Overfitting in model selection for stochastic dynamics.
method Combining likelihood-estimation statistics with extreme value theory.
result PASTIS reliably identifies minimal models, even with low sampling rates or error.
New algorithm finds local minima in non-convex problems efficiently.
problem Finding local minima in non-convex finite-sum minimization problems.
method Stochastic Trust Region (STR) algorithm combining inexact gradient and Hessian estimation.
result STR finds ( ε , ε ) (ε, \sqrtε) ( ε , ε ) -approximate local minimum with improved efficiency. PF-based FSO methods improve on SGD and IPM for large-scale empirical risk minimization.
problem Optimizing large-scale empirical risk minimization problems efficiently.
method Developed PF-based stochastic optimizers (PFSOs) based on FSO methods.
result PFSOs outperform SGD, vanilla IPM, and KF-type FSO methods in stability, speed, and flexibility.
New algorithm improves on EM for streaming data, outperforming existing methods.
problem Processing high-volume, streaming data efficiently.
method Incremental stochastic Majorization-Minimization (MM) algorithm.
result The algorithm converges to a stationary point with vanishing gradient.
A new k-means variant minimizes pairwise distances within clusters.
problem The need for improved clustering methods.
method A stochastic optimization procedure that minimizes the k-sums target function.
result The new k-sums method outperforms k-means and its variants.
SGD optimality proven for convex objectives without smoothness assumptions.
problem Proving optimality of SGD for convex objectives without smoothness assumptions.
method Stochastic Gradient Descent (SGD) for convex objectives without smoothness or strict convexity assumptions.
result With high probability, the objective evaluated at the final candidate minimizer is close to the minimal value of the objective.
Study proves no minimal surfaces can be contained in certain half-spaces or cones.
problem Prohibiting minimal surfaces from certain geometric configurations.
method Analyzes weighted minimal surfaces in R 3 \mathbb{R}^3 R 3 with height-dependent weights. result No proper surfaces can be contained in specific half-spaces or cones.
New adaptive scheduler improves SAM for better model training.
problem Training machine learning models requires selecting a learning rate, which is often difficult and time-consuming.
method Derive Polyak schedulers tailored to SAM-style updates, proving linear convergence for strongly convex objectives and an O(1/T) rate for convex objectives.
result Polyak schedulers achieve comparable or better performance than tuned SAM baselines, reducing the need for learning-rate tuning.
SVRN accelerates Newton methods by reducing variance and improving performance.
problem Improving the efficiency of Newton methods for large-scale optimization problems.
method Stochastic Variance-Reduced Newton (SVRN) algorithm that accelerates Subsampled Newton and Iterative Hessian Sketch algorithms.
result SVRN accelerates Newton methods by reducing the number of passes over the data, achieving a significant improvement in performance.
New algorithm optimally minimizes convex functions with noisy gradients.
problem Minimizing strongly convex, smooth functions with noisy gradient estimates.
method A multistage accelerated stochastic gradient method with restarts.
result Achieves optimal convergence rate in deterministic and stochastic cases.
Study bank salvage model with stochastic impulse controls to minimize costs.
problem Minimize total cost of saving a bank from default with unpredictable default time.
method Impulse stochastic controls to address the bank's default risk.
result Unique viscosity solution exists for the QVI, with Lipschitz and Holder continuity properties.
New algorithm reduces exploration in structured stochastic bandits.
problem Wide class of stochastic bandit problems with known structural properties.
method Developed OSSB algorithm that matches minimal exploration rates of sub-optimal arms.
result OSSB's regret matches asymptotic instance-specific regret lower bound.
SGD achieves a O ( ε − 4 ) O(ε^{-4}) O ( ε − 4 ) bound for minimizing gradient norm of smooth functions.
problem Finding stationary points with SGD for gradient norm minimization.
method Stochastic Gradient Descent (SGD) for smooth, possibly nonconvex functions.
result The O ( ε − 4 ) O(ε^{-4}) O ( ε − 4 ) bound for gradient norm minimization cannot be improved upon. The paper studies a stochastic majority vote approach to improve classifier accuracy.
problem Improving classifier accuracy over ensembles of classifiers.
method Minimizing a PAC-Bayes generalization bound with Dirichlet distributions.
result Achieves state-of-the-art accuracy and tight generalization bounds.
Deep ResNets with constant weights approach Neural ODEs in the limit.
problem Understanding the deep limit behavior of ResNets.
method Proof based on Fokker-Planck equations and stochastic gradient descent convergence.
result Stochastic gradient descent on ResNets converges to Neural ODEs in the deep limit.
Paper extends SMM to weakly convex and multi-convex surrogates for non-convex optimization.
problem Non-convex optimization with weakly convex or multi-convex surrogates.
method Stochastic majorization-minimization with proximal regularization or block-minimization.
result Convergence rates for empirical and expected losses under non-i.i.d. data.