Study finds Deep Taylor Decomposition is unreliable for explaining neural networks.
problem Reliability of Deep Taylor Decomposition for explaining neural networks.
method Investigated the theoretical foundations of Deep Taylor Decomposition (DTD) and found it under-constrained.
result DTD is unreliable because its theoretical foundations are under-constrained and roots do not align with input.
Taylorized training improves neural network training at finite width.
problem Understanding and improving neural network training at finite width.
method Training the k-th order Taylor expansion of the neural network at initialization.
result Taylorized training agrees with full neural network training better as k increases and can significantly close the performance gap.
A new method explains anomalies in one-class models using deep Taylor decomposition.
problem Understanding why one-class models classify data points as anomalies.
method Recompose one-class SVM as a neural network, perform deep Taylor decomposition.
result The method reliably explains a wide set of data anomalies and outperforms baselines.
Deep networks analyze video snippets to predict outcomes, revealing a border effect that can be adjusted for better accuracy.
problem Improving the accuracy of deep networks trained on small video snippets.
method Applied the deep Taylor / LRP technique to understand and identify a border effect, tuning the step size to improve accuracy.
result The step size used to build video snippets can be adjusted to improve deep network accuracy without retraining.
New sampling scheme improves ML accuracy in physics simulations.
problem Improving accuracy of ML models in physics simulations.
method Taylor-based data sampling scheme for DNNs.
result Reduces error in DNN solutions of ODE systems.
Unified framework for analyzing machine learning model attributions.
problem Lack of a general and theoretical framework for understanding attribution methods.
method Proposes a Taylor attribution framework to unify and analyze seven mainstream attribution methods.
result Established three principles for good attribution and empirically validated the Taylor reformulations.
Analyzes explaining nonlinear model predictions.
problem Understanding the contribution of inputs to outputs in nonlinear models.
method Merges integrated gradient and deep Taylor decomposition methods.
result Provides a natural reference point for model at use.
TEAM uses Taylor expansion to generate adversarial examples.
problem Vulnerability of deep neural networks to adversarial examples.
method Approximates DNN output using Taylor expansion and optimizes with Lagrange multiplier method.
result Improves robustness of DNNs through adversarial training.
SOAR improves deep networks' robustness against adversarial examples.
problem Improving deep neural networks' robustness against adversarial examples.
method Formulated adversarial robustness problem under robust optimization framework, approximated loss function using second-order Taylor series expansion.
result SOAR significantly improves robustness of networks against adversarial perturbations.
New insights on pruning deep networks by preserving function locality.
problem Designing effective pruning methods for deep neural networks.
method Revisited loss modeling using first and second order Taylor expansions, emphasizing locality.
result Both first and second order Taylor expansions can achieve similar performance in pruning.
Proposes a Taylor framework to unify and analyze attribution methods.
problem Lack of a unified guideline for feature contribution assignment in machine learning models.
method Introduces a Taylor attribution framework to model the attribution problem and reformulates fourteen mainstream methods.
result Empirically validates the Taylor reformulations and reveals a positive correlation between performance and principles followed.
Paper presents a new backward deep BSDE method for solving nonlinear FBSDE problems.
problem Nonlinear Forward Backward Stochastic Differential Equations (FBSDE) with terminal conditions.
method Backward deep BSDE method applied to FBSDE with nonlinear generators and random initial conditions.
result Derives exact and Taylor-based approximations for time-stepping nonlinear BSDEs.
Modern convolutional networks, incorporating rectifiers and max-pooling, are neither smooth nor convex; standard guarantees therefore do not apply. Nevertheless, methods from convex optimization such as gradient descent and Adam are widely used as building blocks for deep learning algorithms. This paper provides the fi…
Deep ReLU networks can approximate smooth functions nearly optimally.
problem Approximating smooth functions with deep neural networks.
method Using Taylor expansions and deep ReLU network approximations, the paper establishes optimal approximation error bounds.
result Deep ReLU networks of width and depth O(NlnN) and O(LlnL) can approximate f∈Cs([0,1]d) with an error O(∥f∥Cs([0,1]d)N−2s/dL−2s/d). New method finds better loss functions for neural nets.
problem Finding effective loss functions for deep neural networks.
method Optimizes multivariate Taylor polynomial parameterizations using CMA-ES.
result TaylorGLO finds loss functions that outperform existing methods.
Neural networks learn higher-order derivatives for physics problems.
problem Lack of higher-order derivatives in neural networks for theoretical physics.
method Graph-theoretical approach to assign diagrams to partial derivatives, iterative NN perturbation theory.
result NNs can learn higher-order derivatives, improving machine-learned approximations.
A new method exposes motion-related relevance in video frames.
problem Deconstructing relevance in spatio-temporal models for video processing.
method Proposes a discriminative method to separate spatial and temporal relevance.
result Demonstrates effectiveness on UCF-101 action recognition dataset.
Kernelized Taylor diagram visualizes data populations with fewer assumptions.
problem Limitations of Taylor diagram in capturing non-linear relationships and sensitivity to outliers.
method Proposes a kernelized version of the Taylor diagram that uses maximum mean discrepancy and kernel mean embedding.
result Kernelized Taylor diagram visualizes data populations with minimal assumptions of data distributions.
Let M and N be smooth manifolds without boundary. Immersion theory suggests that an understanding of the space of smooth embeddings emb(M,N) should come from an analysis of the cofunctor V |--> emb(V,N) from the poset O of open subsets of M to spaces. We therefore abstract some of the properties of this cofunctor, and …
A new DP method for deep learning with faster convergence and better privacy.
problem Challenges in differentially private training of deep neural networks.
method Method of auxiliary coordinates with perturbed Taylor expansion for privacy.
result Empirically shows decent trained model quality with modest privacy budget.
Taylor expansions improve reinforcement learning policies.
problem Improving reinforcement learning policy optimization.
method Taylor expansion policy optimization.
result Taylor expansions enhance performance of distributed algorithms.
Nonlinear methods such as Deep Neural Networks (DNNs) are the gold standard for various challenging machine learning problems, e.g., image classification, natural language processing or human action recognition. Although these methods perform impressively well, they have a significant disadvantage, the lack of transpar…
NN2Poly converts deep neural networks into polynomial models for better understanding.
problem Improving neural network interpretability and theoretical understanding.
method Taylor expansion on activation functions, combinatorial properties, and polynomial coefficients calculation.
result NN2Poly accurately represents deep feed-forward neural networks as polynomial models.
Improved saliency maps for deep neural networks with reduced noise.
problem Noisy explanations in Integrated Gradients for deep neural networks.
method SmoothTaylor, adaptive noising, and SmoothGrad techniques.
result SmoothTaylor and adaptive noising generate better quality saliency maps.
TEAM generates more powerful adversarial examples for DNNs.
problem Vulnerability of DNNs to imperceptible adversarial examples.
method TEAM uses Taylor expansion and Lagrangian multiplier method to craft adversarial examples.
result TEAM generates adversarial examples with 100% attack success rate using smaller perturbations.
Paper proposes a new Taylor moment expansion for non-linear Gaussian filtering and smoothing.
problem Non-linear Gaussian filtering and smoothing in continuous-discrete state-space models.
method Taylor moment expansion (TME) for moment functions directly and in time variable.
result Significantly outperforms state-of-the-art methods in terms of estimation accuracy and numerical stability.
New samplers reduce NFEs for diffusion models.
problem High NFEs in diffusion models.
method Quasi-Taylor samplers based on ideal derivatives.
result Reduced NFEs for image synthesis.
Interpretable deep learning classifies two-prong jets using jet spectra.
problem Lack of interpretability in deep learning for jet classification.
method Truncated Taylor series jet spectrum for interpretability.
result Interpretable network performs similarly to CNN but is simpler.
A framework connects Taylor methods with Fisher-efficient NG.
problem Combining Taylor-based methods with Fisher-efficient NG.
method Constructs a theoretical framework linking Taylor approximation and NG.
result Mathematical justification for combining higher order methods with NG.
The first aperiodic monotiling, introduced by Taylor, was based on a trapezoidal prototile equipped with 14 distinct decorations. A presentation of the closely related Taylor-Socolar aperiodic monotiling is based on a hexagonal prototile equipped with 7 decorations. This paper gives decoration-free algebraic descriptio…
New method improves neural network performance by focusing on steep function regions.
problem Improving neural network performance by focusing on steep function regions.
method Variance Based Samples Weighting (VBSW) using labels local variance to weight training points.
result Significantly increases the performances of neural networks for various tasks.
Develops state-space deep Gaussian processes for irregular signals.
problem Solving deep Gaussian process regression problems for irregular signals/functions.
method Represent DGPs as SDEs, solve using state-space filtering and smoothing methods.
result Rich class of priors compatible with irregular signals/functions.
FFCP improves FCP's speed without sacrificing accuracy.
problem Inefficient feature transformation in FCP.
method Introduces FFCP using Taylor expansion for faster computation.
result FFCP achieves a 50x speedup with comparable accuracy.
New approximations for Asian basket spread options using stochastic Taylor expansions.
problem Pricing Asian basket spread options under the Black-Scholes model.
method Stochastic Taylor expansion applied to a log-normal proxy model.
result Highly accurate approximations for Asian and spread options, without numerical integration.
TaylorPODA uses Taylor expansions to improve feature attributions for opaque models.
problem Lack of systematic framework for quantifying feature contributions in opaque models.
method Taylor expansion framework with postulates (precision, federation, zero-discrepancy, adaptation).
result TaylorPODA achieves competitive results and provides principled explanations.
Paper develops a new algorithm to find shortest paths on surfaces.
problem Finding shortest paths on surfaces with defined metrics.
method Uses Taylor expansion of exponential map for numerical computation.
result Developed a new algorithm to find geodesics efficiently.
Developed Taylor series for muscle-finger system analysis.
problem Understanding the complex relationship between muscle activity and finger movement.
method Used Dendrite Net to develop Taylor series and construct relation spectrum.
result Found muscle synergy and coupling in hand movement.
The Adomian decomposition method is shown to be equivalent to the Taylor series approach.
problem Incorrectly perceived complexity of the Adomian decomposition method.
method Demonstrates the Adomian decomposition method as equivalent to the Taylor series approach.
result The Adomian decomposition method is simpler and more straightforward.
Proves a special case of the Gaussian kinematic formula using large sphere limits.
problem Proving a special case of the Gaussian kinematic formula.
method Viewing the GKF as the limit of spherical kinematic formulas for large dimension spheres.
result Proves a special case of the Gaussian kinematic formula.
We introduce Taylor expansions that do not require the differentiability. We also provide new solutions to partial differential equations. We apply our methods to finance.
An expanding literature articulates the view that Taylor rules are helpful in predicting exchange rates. In a changing world however, Taylor rule parameters may be subject to structural instabilities, for example during the Global Financial Crisis. This paper forecasts exchange rates using such Taylor rules with Time V…
In this work we consider the Taylor expansion of the exponential map of a submanifold immersed in R^n up to order three, in order to introduce the concepts of lateral and frontal deviation. We compute the directions of extreme lateral and frontal deviation for surfaces in R^3. Also we compute, by using the Taylor expan…
New resonance theory for Anosov flows connects spectral properties to mixing measures.
problem Defining and analyzing Ruelle-Taylor resonances for Anosov actions.
method Combining microlocal methods and J. Taylor's cohomological theory, defining Ruelle-Taylor resonances and proving Fredholm theory.
result Ruelle-Taylor resonances form a discrete subset of Cκ with λ=0 being a leading resonance. We provide new exact Taylor's series with fixed coefficients and without the remainder. We demonstrate the usefulness of this contribution by using it to obtain very simple solutions to (non-linear) PDEs. We also apply the method to the portfolio model.
Examines how central bank policies affect stock markets and asset prices.
problem Understanding the impact of monetary policy on stock markets and asset prices.
method Used Taylor rule equations to analyze data from 1990 to 2020 for US and UK, testing with various econometric methods.
result Monetary policy can explain asset price volatility and output gap better than just inflation rate.
We develop closed-form approximations for European put options under stochastic volatility models.
problem Tackling the pricing of European put options under stochastic volatility models with time-dependent parameters.
method Using a second-order Taylor expansion around the mean of the argument, we write the option price as an expectation of a Black-Scholes formula. We then simplify the resulting expectations and derive closed-form pricing formulas under the assumption of piecewise-constant parameters.
result We derive closed-form pricing formulas and bounds on the remainder term generated by the Taylor expansion, showing that the errors are well within acceptable ranges for practical applications.
Study complex structures with totally real sections, providing integrability equations.
problem Existence and integrability of complex structures with totally real sections.
method Explicit integrability equations derived from fiberwise Taylor expansions.
result Detailed fiberwise Taylor expansions and integrability equations in a geometric case.
In this paper we study the Taylor series of an operator-valued function related to the differential of the exponential map. For a smooth manifold M with a torsion-free affine connection the operator Ep(v) acting on the space TpM is defined to be the composition of the differential …