Taylorized training improves neural network training at finite width.
problem Understanding and improving neural network training at finite width.
method Training the k-th order Taylor expansion of the neural network at initialization.
result Taylorized training agrees with full neural network training better as k increases and can significantly close the performance gap.
Paper provides convergence guarantees for rectifier networks using neural Taylor approximations.
problem Smoothness and convexity issues in modern convolutional networks.
method Neural Taylor approximations and Taylor loss for optimization.
result Guarantees match lower bounds for convex nonsmooth functions and accurately capture optimization dynamics.
New sampling scheme improves ML accuracy in physics simulations.
problem Improving accuracy of ML models in physics simulations.
method Taylor-based data sampling scheme for DNNs.
result Reduces error in DNN solutions of ODE systems.
Paper finds dropout noise approximation invalid for logistic regression and neural networks.
problem Invalidity of dropout noise approximation for logistic regression and neural networks.
method Derived equivalence between dropout noise injection and L2 regularisation using divergent Taylor expansion. result Approximation approach is invalid for robust constraints and general neural network topologies.
TEAM uses Taylor expansion to generate adversarial examples.
problem Vulnerability of deep neural networks to adversarial examples.
method Approximates DNN output using Taylor expansion and optimizes with Lagrange multiplier method.
result Improves robustness of DNNs through adversarial training.
SOAR improves deep networks' robustness against adversarial examples.
problem Improving deep neural networks' robustness against adversarial examples.
method Formulated adversarial robustness problem under robust optimization framework, approximated loss function using second-order Taylor series expansion.
result SOAR significantly improves robustness of networks against adversarial perturbations.
New samplers reduce NFEs for diffusion models.
problem High NFEs in diffusion models.
method Quasi-Taylor samplers based on ideal derivatives.
result Reduced NFEs for image synthesis.
Neural networks learn higher-order derivatives for physics problems.
problem Lack of higher-order derivatives in neural networks for theoretical physics.
method Graph-theoretical approach to assign diagrams to partial derivatives, iterative NN perturbation theory.
result NNs can learn higher-order derivatives, improving machine-learned approximations.
New approximations for Asian basket spread options using stochastic Taylor expansions.
problem Pricing Asian basket spread options under the Black-Scholes model.
method Stochastic Taylor expansion applied to a log-normal proxy model.
result Highly accurate approximations for Asian and spread options, without numerical integration.
A framework connects Taylor methods with Fisher-efficient NG.
problem Combining Taylor-based methods with Fisher-efficient NG.
method Constructs a theoretical framework linking Taylor approximation and NG.
result Mathematical justification for combining higher order methods with NG.
Proposes a Taylor framework to unify and analyze attribution methods.
problem Lack of a unified guideline for feature contribution assignment in machine learning models.
method Introduces a Taylor attribution framework to model the attribution problem and reformulates fourteen mainstream methods.
result Empirically validates the Taylor reformulations and reveals a positive correlation between performance and principles followed.
Unified framework for analyzing machine learning model attributions.
problem Lack of a general and theoretical framework for understanding attribution methods.
method Proposes a Taylor attribution framework to unify and analyze seven mainstream attribution methods.
result Established three principles for good attribution and empirically validated the Taylor reformulations.
Deep ReLU networks can approximate smooth functions nearly optimally.
problem Approximating smooth functions with deep neural networks.
method Using Taylor expansions and deep ReLU network approximations, the paper establishes optimal approximation error bounds.
result Deep ReLU networks of width and depth O(NlnN) and O(LlnL) can approximate f∈Cs([0,1]d) with an error O(∥f∥Cs([0,1]d)N−2s/dL−2s/d). New approach uncovers neural network performance beyond linearized models.
problem Empirical performance of neural networks exceeds linearized models.
method Randomizing neural networks to couple with quadratic and higher-order terms in Taylor expansion.
result Concrete generalization and expressivity results on randomized networks match NTK and can be better by a dimension factor.
Graph hypernetworks improve molecule property prediction and classification.
problem Improving molecule property prediction and classification using graph neural networks.
method Replacing underlying networks with hypernetworks and addressing training instability.
result Demonstrated state-of-the-art performance in various benchmarks.
This paper tackles catastrophic forgetting in neural networks by providing a unified framework for regularization-based continual learning.
problem Catastrophic forgetting in neural networks trained sequentially on multiple tasks.
method Formulates regularization-based continual learning as a second-order Taylor approximation of the loss function, leading to a unified framework.
result Theoretical results indicate the importance of accurate approximation of the Hessian matrix for optimization and generalization.
A method estimates and prunes neural network filters to reduce computation and improve accuracy.
problem Reduction of neural network parameters to save computation and energy.
method Estimates each neuron's contribution to loss using first and second-order Taylor expansions; iteratively removes less important neurons.
result High (>93%) correlation between estimated and true importance; 40% FLOPS reduction with 0.02% top-1 accuracy loss.
Study finds Deep Taylor Decomposition is unreliable for explaining neural networks.
problem Reliability of Deep Taylor Decomposition for explaining neural networks.
method Investigated the theoretical foundations of Deep Taylor Decomposition (DTD) and found it under-constrained.
result DTD is unreliable because its theoretical foundations are under-constrained and roots do not align with input.
Develops AMITE for analyzing neural network nonlinearities.
problem Addressing difficulties in verification, explainability, and security in neural network analysis.
method Analytically modified integral transform expansion (AMITE) for neural network nonlinearities.
result First to provide six mutually exclusive desired expansion properties.
We develop closed-form approximations for European put options under stochastic volatility models.
problem Tackling the pricing of European put options under stochastic volatility models with time-dependent parameters.
method Using a second-order Taylor expansion around the mean of the argument, we write the option price as an expectation of a Black-Scholes formula. We then simplify the resulting expectations and derive closed-form pricing formulas under the assumption of piecewise-constant parameters.
result We derive closed-form pricing formulas and bounds on the remainder term generated by the Taylor expansion, showing that the errors are well within acceptable ranges for practical applications.
New method improves neural network performance by focusing on steep function regions.
problem Improving neural network performance by focusing on steep function regions.
method Variance Based Samples Weighting (VBSW) using labels local variance to weight training points.
result Significantly increases the performances of neural networks for various tasks.
New learning algorithm for real analytic functions without gradient descent.
problem Learning real analytic functions without gradient descent.
method Taylor approximation and sampling data distribution.
result Nonuniform learning result for real analytic functions.
Using classical Taylor series techniques, we develop a unified approach to pricing and implied volatility for European-style options in a general local-stochastic volatility setting. Our price approximations require only a normal CDF and our implied volatility approximations are fully explicit (ie, they require no spec…
Paper proposes a new Taylor moment expansion for non-linear Gaussian filtering and smoothing.
problem Non-linear Gaussian filtering and smoothing in continuous-discrete state-space models.
method Taylor moment expansion (TME) for moment functions directly and in time variable.
result Significantly outperforms state-of-the-art methods in terms of estimation accuracy and numerical stability.
In this paper we propose a closed-form approximation for the price of basket options under a multivariate Black-Scholes model, based on Taylor expansions and the calculation of mixed exponential-power moments of a Gaussian distribution. Our numerical results show that a second order expansion provides accurate prices o…
Study exchange option pricing with stochastic volatility and correlation.
problem Pricing exchange options under stochastic volatility and correlation.
method Approximation using a closed-form solution with Taylor expansion.
result Numerical results show the effectiveness of the proposed method.
Shallow neural networks can represent polynomials efficiently.
problem Representing polynomials using shallow neural networks.
method Using shallow neural networks of width 2(R+d)d to represent d-variate polynomials of degree R. result Derives minimax optimal convergence rate for shallow networks to unknown univariate regression functions.
A mathematical framework connects neural networks and polynomial regression for better model understanding.
problem Neural networks are black boxes with challenges in dimensioning and prediction error evaluation.
method Developed a mathematical framework using Taylor expansion to relate neural networks and polynomial regression.
result Polynomial approximations from neural networks trained on polynomial data are accurate locally.
Estimates neural drift for stochastic equations, improving inference on noisy data.
problem Estimating drift in stochastic differential equations with neural networks.
method Non-parametric estimation using ReLU neural networks, enforcing theoretical bounds.
result Practical method for inference on noisy and rough functional data.
Approximates option prices in Barndorff-Nielsen and Shephard models using Taylor expansion.
problem Approximating option prices in complex stochastic volatility models.
method Taylor expansion and recursive algorithm for closed-form approximations.
result Explicit results for inverse Gaussian and gamma stationary distributions, with favorable comparisons to characteristic function.
TEAM generates more powerful adversarial examples for DNNs.
problem Vulnerability of DNNs to imperceptible adversarial examples.
method TEAM uses Taylor expansion and Lagrangian multiplier method to craft adversarial examples.
result TEAM generates adversarial examples with 100% attack success rate using smaller perturbations.
Study embeddings between Barron spaces with various activation functions, focusing on RePU.
problem Understanding the influence of activation functions on infinitely wide neural networks.
method Prove embeddings by constructing push-forward maps on measures representing functions.
result Barron spaces with RePU activation have a hierarchical structure similar to Sobolev spaces.
In this article we develop a method for the strong approximation of stochastic differential equations (SDEs) driven by Lévy processes or general semimartingales. The main ingredients of our method is the perturbation of the SDE and the Taylor expansion of the resulting parameterized curve. We apply this method to devel…
The paper modifies asset pricing models using Taylor series expansions and market-based averages.
problem Improving asset pricing models to better reflect market dynamics.
method Derives new pricing equations using Taylor series expansions and market-based averages.
result New expressions for asset prices and volatilities derived from market data.
Prunes CNNs for resource efficiency in inference.
problem Efficient inference of large neural networks.
method Greedy pruning with backpropagation, Taylor expansion criterion.
result Pruning leads to significant reduction in filter count with minimal accuracy loss.
We approximate prices of various financial claims using a combination of expansions.
problem Approximating prices of financial claims in a complex volatility setting.
method Combining Taylor series expansions of diffusion coefficients with an expansion in correlation parameter.
result Rigorous accuracy results for European-style claims, and numerical examples for barrier-style claims.
AdaRound improves post-training quantization of neural networks.
problem Improving the accuracy of quantized weights in neural networks.
method Adaptive rounding mechanism that adapts to data and task loss.
result AdaRound outperforms rounding-to-nearest and achieves state-of-the-art performance.
Manifold calculus of functors, due to M. Weiss, studies contravariant functors from the poset of open subsets of a smooth manifold to topological spaces. We introduce "multivariable" manifold calculus of functors which is a generalization of this theory to functors whose domain is a product of categories of open sets. …
This paper provides intuition on the relationship of accrual and mark-to-market valuation for cash and forward interest rate trades. Discounted cashflow valuation is compared to spread-based valuation for forward trades, which explains the trader's view on valuation. This is followed by Taylor series approximation for …
AON improves neural network generalization by making weights approximately orthogonal.
problem Improving generalization of deep neural networks.
method Approximated orthonormal normalisation (AON) technique to make weight vectors approximately orthogonal.
result AON yields promising validation performance compared to orthonormal regularisation.
Study shows how noise and distractions affect neural network explanations.
problem Lack of consideration for noise and distractions in neural network explanations.
method Examined the impact of noise and distracting elements on neural network interpretation models.
result Noise and distracting elements influence the results of neural network explanations.
Paper improves kernel approximations for better statistical learning.
problem Improving kernel approximations for better statistical learning.
method Taylor series approximations of radial kernel functions.
result Establishes upper bounds for eigenfunctions, leading to better approximations.
Paper proposes a closed-form formula for geometric Istanbul call options.
problem Pricing geometric Istanbul call options under the Black-Scholes model.
method Second-order Taylor expansion to derive a closed-form approximation.
result The proposed formula accurately approximates GIC values compared to Monte-Carlo simulations.
Agents learn state ambiguity from non-linear sensor data using Gaussian approximations.
problem Learning state representation from non-linear sensor data.
method Second-order Taylor approximation of Gaussian distribution for non-linear measurement functions.
result Induces a preference for states based on inferability from observations.
New method finds better loss functions for neural nets.
problem Finding effective loss functions for deep neural networks.
method Optimizes multivariate Taylor polynomial parameterizations using CMA-ES.
result TaylorGLO finds loss functions that outperform existing methods.
New insights on pruning deep networks by preserving function locality.
problem Designing effective pruning methods for deep neural networks.
method Revisited loss modeling using first and second order Taylor expansions, emphasizing locality.
result Both first and second order Taylor expansions can achieve similar performance in pruning.
Paper proves existence of a CMC hypertorus in 4D sphere using numerical methods.
problem Proving the existence of a constant mean curvature (CMC) hypertorus in \(S^4\).
method Employed the round Taylor method with rational arithmetic and the Poincare-Miranda theorem.
result Existence of a constant mean curvature (CMC) hypertorus in \(S^4\).
The paper calculates Bachelier option prices using Taylor expansions and applies it as a variance reduction technique.
problem Calculating Bachelier option prices and variance reduction in correlated cases.
method Taylor expansions and classical Itô calculus to derive option prices, uses negative powers of future mean volatility.
result The paper provides a new method to calculate Bachelier option prices and applies it to reduce variance in Monte Carlo simulations.