Paper derives second order approximations for limit order books.
problem Modeling fluctuations in price and volume processes.
method Second order approximation of infinite dimensional limit order book dynamics.
result Confidence intervals for optimal portfolio liquidation models.
We develop a second-order model for limit order books in a single scaling regime.
problem Modeling price and volume dynamics in a limit order book with market and limit orders at a common time scale.
method Established a first- and second-order approximation for an infinite dimensional limit order book model.
result Proved the existence and uniqueness of a solution for the second-order approximation.
New geometric proof of convex function differentiability and approximation.
problem Second-order differentiability of convex functions and their approximations.
method Elementary geometric approach to prove classical and recent results.
result New proofs of Lusin approximation of convex functions and bodies by C1,1 functions. Paper introduces STSL, a second-order Tweedie sampler for efficient posterior sampling in inverse problems.
problem Computational challenges in sampling from posterior distributions using latent diffusion models.
method Introduces STSL, a novel second-order Tweedie sampler with tractable reverse process.
result STSL achieves 4X and 8X reduction in neural function evaluations compared to state-of-the-art solvers.
Paper examines risk measure expansions under FGM dependence, improving accuracy at extreme levels.
problem Capturing higher-order tail behavior and dependence effects in risk measures.
method Second-order asymptotic expansions using extreme value theory and regular variation theory.
result Second-order approximations reduce approximation errors, especially at extreme confidence levels.
New algorithm finds approximate stationary points in non-convex optimization.
problem Finding approximate stationary points in non-convex stochastic optimization.
method Design of an algorithm using O(ε−3) stochastic gradient and Hessian-vector products. result Optimal rate of O(ε−3) for finding ε-approximate stationary points, matching lower bounds. New matrix approximation method speeds up optimization for deep learning.
problem Efficient computation of matrix inverse and square root for high-dimensional optimization.
method Divide matrix into blocks and represent each block by one or two numbers.
result Improved performance of AdaGrad in training deep neural networks compared to diagonal approximation.
Disputes the empirical Fisher approximation for natural gradient descent.
problem The empirical Fisher approximation fails to capture second-order information in general.
method Comparison of empirical Fisher and Fisher information matrices.
result The empirical Fisher does not generally approximate the Fisher or Hessian.
Second-order economic theory considers new variables to improve price volatility predictions.
problem Current economic models focus on first-order variables, missing second-order variables that affect price volatility.
method Introduces second-order economic theory with new variables composed of sums of squares of agents' transactions.
result Second-order economic theory complements first-order variables and introduces new macroeconomic variables.
The paper debiases mini-batch approximations in deep learning for more accurate optimization and uncertainty quantification.
problem Bias in mini-batch approximations distorts the shape of quadratic approximations used in deep learning.
method Developed and evaluated debiasing strategies for mini-batch approximations.
result Debiasing strategies improve the accuracy of second-order optimization and uncertainty quantification in deep learning.
A new method for optimizing deep neural networks using TKFAC.
problem Optimizing deep neural networks with second-order methods.
method Proposes Trace-restricted Kronecker-factored Approximate Curvature (TKFAC) for Fisher information matrix approximation.
result TKFAC improves performance on deep network architectures compared to state-of-the-art algorithms.
Method provides bounds for sparse PCA and nuclear norm problems.
problem Semidefinite optimization problems (SDOs).
method Cutting-plane method with focus on initial outer approximation as a second-order cone approximation.
result Method provides bound gaps of 0.5-6.5% for sparse PCA problems with 1000 covariates and solves nuclear norm problems over 500x500 matrices.
Proposes second-order influence functions for identifying influential groups in test-time predictions.
problem Identifying influential groups in test-time predictions for black-box models.
method Second-order approximations of the effect of removing a group of training samples on model predictions.
result Improves the correlation between computed influence values and ground truth values for linear models.
Enhances SMC² with Hessian info for more efficient posterior approximation.
problem Improving accuracy and efficiency in Bayesian inference.
method Integrates second-order information (Hessian) into SMC²'s proposal distribution.
result Second-order proposals lead to more accurate posterior approximations and better step-size selection.
A new method improves ICA performance by approximating MDI.
problem Improving F astICA's performance with nonlinear functions.
method Second-order approximation of MDI for joint maximization.
result Efficiency validated through experiments compared to other ICA algorithms.
Paper introduces arctan pinball loss for XGBoost quantile regression.
problem Efficiently predicting multiple quantiles with XGBoost.
method Smooth approximation of pinball loss for XGBoost, using arctan pinball loss.
result Arctan pinball loss reduces quantile crossings and improves efficiency.
SOAR improves deep networks' robustness against adversarial examples.
problem Improving deep neural networks' robustness against adversarial examples.
method Formulated adversarial robustness problem under robust optimization framework, approximated loss function using second-order Taylor series expansion.
result SOAR significantly improves robustness of networks against adversarial perturbations.
The quantification of diversification benefits due to risk aggregation plays a prominent role in the (regulatory) capital management of large firms within the financial industry. However, the complexity of today's risk landscape makes a quantifiable reduction of risk concentration a challenging task. In the present pap…
Improved robustness in optimization methods using second-order information.
problem Scalability and sensitivity to mini-batch size in optimization methods.
method Mini-Batch Stochastic Variance-Reduced Newton (extttMb−SVRN) algorithm incorporating partial second-order information. result Achieves a fast linear convergence rate independent of mini-batch size for large data sizes.
Paper proposes a new method for efficient second-order neural network training.
problem Infeasibility of Hessian calculation and noisy second-order information in deep learning.
method Adopting complex-step directional derivative (CSFD) for accurate Hessian computation and designing an effective Newton Krylov procedure.
result Our method outperforms existing methods and often converges one-order faster.
Proposes a new method for optimizing large-scale models using Nyström approximation of the Hessian.
problem Optimizing non-convex functions like deep learning models using second-order methods.
method Nyström-approximated curvature for stochastic optimization of large-scale empirical risk minimization.
result The proposed method achieves performance competitive with state-of-the-art first-order and stochastic quasi-Newton methods.
Efficient method classifies locally stationary time series based on second-order characteristics.
problem Classifying locally stationary time series for various applications.
method Autoregressive approximation, ensemble aggregation, distance-based threshold.
result Zero misclassification error rate asymptotically for mildly differing second-order characteristics.
Multiscale stochastic volatility models have been developed as an efficient way to capture the principle effects on derivative pricing and portfolio optimization of randomly varying volatility. The recent book Fouque, Papanicolaou, Sircar and Sølna (2011, CUP) analyzes models in which the volatility of the underlying i…
Two new algorithms improve federated optimization under second-order similarity.
problem Federated learning under communication constraints and second-order similarity.
method SVRP and Catalyzed SVRP algorithms combining proximal point evaluations, client sampling, and variance reduction.
result Achieves superior performance and uniformly improves upon existing algorithms for federated optimization under second-order similarity and strong convexity.
New pruning method captures global correlations for efficient neural network inference.
problem Efficiently pruning neural networks for faster inference and reduced memory usage.
method Second-order structured pruning (SOSP-H) with innovative saliency-based approaches.
result SOSP-H scales to large-scale vision tasks and improves accuracy without compromising efficiency.
Stein variational Newton method accelerates SVGD for faster inference.
problem Efficient nonparametric variational inference.
method Accelerates SVGD by incorporating second-order information and optimal kernel selection.
result Significant computational gains over original SVGD in multiple test cases.
Paper proposes SNAP algorithm for finding approximate SOSPs efficiently.
problem Finding approximate second-order stationary points of non-convex problems with linear constraints.
method SNAP algorithm uses strict complementarity condition and negative curvature projections.
result SNAP and SNAP+ achieve polynomial per-iteration complexity and global sublinear rate for finding SOSPs. Optimizes hyperparameter tuning for models using approximate leave-one-out cross-validation.
problem Finding optimal hyperparameters for regularized models using approximate leave-one-out cross-validation.
method Derive efficient formulas for gradient and hessian of approximate leave-one-out cross-validation, apply second-order optimization.
result Demonstrates the effectiveness of the approach on real-world data sets.
Paper proposes a method to find approximate SOSP for nonconvex conic optimization problems.
problem Finding approximate second-order stationary points in nonconvex conic optimization.
method Newton-CG based barrier method with complexity guarantees.
result Achieves iteration complexity of O(ε^(-3/2)) for finding (ε,√ε)-SOSP.
New distributed algorithm for second-order optimization with reduced Hessian computation.
problem Efficiently using second-order methods in distributed optimization with limited communication.
method Distributed training of generalized linear models with adaptive trust-region approach.
result Achieves state-of-the-art results on multiple large datasets.
Paper improves sampling from smooth, log-concave densities with inaccurate gradients.
problem Sampling from smooth, log-concave densities with inaccurate gradient evaluations.
method Approximate sampling using discretizations of the Langevin diffusion with optimized step sizes and various gradient approximations.
result Improved guarantees on sampling error, including logarithmic improvements and nonasymptotic bounds for second-order methods.
New high-order approximations for CIR process using random grids.
problem Approximating the Cox-Ingersoll-Ross process with high order.
method Combining discretization schemes on different random grids.
result Weak approximations of order 2k for all k∈N∗. Paper proposes a second-order method for faster SVI convergence.
problem Poor convergence rate of first-order SVI algorithms.
method Derives Hessian matrix and implements two numerical schemes for efficient second-order SVI.
result Proposed approach achieves faster convergence compared to first-order SVI.
ISAAC Newton uses input-based curvature for efficient training.
problem Efficient training in small-batch stochastic regimes.
method ISAAC Newton conditions gradients using selected second-order information based on input.
result Effective training even in small-batch stochastic regimes, competitive to first-order and second-order methods.
We study the sensitivity of the expected utility maximization problem in a continuous semi-martingale market with respect to small changes in the market price of risk. Assuming that the preferences of a rational economic agent are modeled with a general utility function, we obtain a second-order expansion of the value …
Develops high-order approximations for financial models, proving convergence and regularity.
problem Challenges in approximating and regularizing the Heston model due to its square root diffusion term.
method Random grid technique, Cox-Ingersoll-Ross (CIR) process, log-Heston process, PDE analysis.
result Achieves weak approximations of any order for smooth test functions in the Heston model, extending to log-Heston process.
We develop high-order approximations for the Heston model.
problem Modeling the Heston model with high accuracy and efficiency.
method Combining approximation schemes on different random grids to achieve any order of convergence.
result Achieve any order of convergence for the Heston model.
Simplifies convolutions using tensor networks and einsum for efficient second-order methods.
problem Complexity in analyzing and applying convolutions in deep learning.
method Viewing convolutions as tensor networks, drawing diagrams, and using einsum for efficient computation.
result Accelerates a KFAC variant up to 4.5x with reduced memory overhead.
Introduces a new price measure and a second-order economic theory for volatility forecasting.
problem Forecasting price volatility in financial markets.
method Develops a new price measure and a second-order economic theory to model price volatility.
result Shows that second-order economic theory improves forecasting of price volatility.
PDHAMS improves sampling for discrete distributions with quadratic potential functions.
problem Sampling discrete distributions efficiently and accurately.
method Integrates a second-order approximation of the potential function and uses Gaussian integral trick.
result PDHAMS yields superior performance compared to other methods.
Paper proposes a new Hessian-aware zeroth-order optimization for improving black-box adversarial attacks.
problem Improving black-box adversarial attacks on neural networks.
method Introduces a Hessian-aware zeroth-order optimization algorithm called ZO-HessAware.
result ZO-HessAware achieves improved success rates with lower query complexity.
Faster convergence and handling larger mini-batches for deep neural networks.
problem Generalization gap in large-scale distributed training of deep neural networks.
method Second-order optimization using Kronecker-factored approximate curvature.
result Achieved 75% Top-1 validation accuracy with mini-batch size of 131,072 in 978 iterations.
New method avoids saddle points without gradients.
problem Optimizing non-convex functions efficiently.
method Zero-order derivative-free algorithm using only function evaluations.
result Converges to second-order stationary points efficiently.
In the present work, a novel second-order approximation for ATM option prices is derived for a large class of exponential Lévy models with or without Brownian component. The results hereafter shed new light on the connection between both the volatility of the continuous component and the jump parameters and the behavio…
Optimal algorithms for Riemannian optimization with reduced complexity.
problem Stochastic optimization on Riemannian manifolds with limited data.
method Zeroth-order Riemannian Averaging Stochastic Approximation algorithms using Riemannian moving-average estimators and novel geometric conditions.
result Achieves optimal sample complexities for generating approximate first-order stationary solutions.
Paper identifies neural network weights from few samples using differentiation and tensor products.
problem Identifying weights of shallow neural networks from limited data.
method Uses second-order differentiation and tensor product decomposition to identify weights from a small number of samples.
result Proves successful identification of weight vectors close to orthonormal and constructsively reduces generality.
Develops first and second-order pseudo-mirror descent methods for nonnegative function estimation.
problem Nonnegative function estimation in settings like MLE and trajectory optimization.
method First and second-order pseudo-mirror descent with pseudo-gradients and projections.
result Establishes tradeoffs and non-asymptotic bounds on model complexity.
Study shows rate of convergence for particle approximation of PDEs in Wasserstein space.
problem Analyzing convergence rates for particle approximations of PDEs in Wasserstein space.
method Backward stochastic differential equations techniques.
result Proved a rate of convergence of order 1/N for pathwise error and 1/sqrt(N) for L2-error on the derivative.