A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
This paper considers multi-dimensional affine processes with continuous sample paths. By analyzing the Riccati system, which is associated with affine processes via the transform formula, we fully characterize the regions of exponents in which exponential moments of a given process do not explode at any time or explode…
Solves Multivariate-MAB problem with path planning and Thompson sampling.
We examine the small expiry behaviour of European call options in stock price models of exponential Lévy type. In most cases of interest, we are able to identify the exact small expiry asymptotics. In "complete generality" we are able to show that the time value of the call option has O(τ) decay as τ(time to expiry) go…
Classical (Itô diffusions) stochastic volatility models are not able to capture the steepness of small-maturity implied volatility smiles. Jumps, in particular exponential Lévy and affine models, which exhibit small-maturity exploding smiles, have historically been proposed to remedy this (see \cite{Tank} for an overvi…
This paper extends de Rham theory of smooth manifolds to exploded manifolds. Included are versions of Stokes' theorem, De Rham cohomology, Poincare duality, and integration along the fiber. The resulting cohomology theory is used to define Gromov Witten invariants of exploded manifolds in a separate paper.
Recurrent neural networks (RNNs) are particularly well-suited for modeling long-term dependencies in sequential data, but are notoriously hard to train because the error backpropagated in time either vanishes or explodes at an exponential rate. While a number of works attempt to mitigate this effect through gated recur…
Improves diffusion models by controlling total variance and signal-to-noise-ratio.
Notes for a short lecture series, covering exploded manifolds, the moduli stack of curves in exploded manifolds, and a tropical gluing formula for Gromov-Witten invariants: a gluing formula providing a degeneration formula for Gromov-Witten invariants in normal-crossing degenerations. I gave the original lecture series…
We give a rigorous analysis of the statistical behavior of gradients in a randomly initialized fully connected network N with ReLU activations. Our results show that the empirical variance of the squares of the entries in the input-output Jacobian of N is exponential in a simple architecture-dependent constant beta, gi…
New neural network approach mitigates vanishing/exploding gradients.
We present a gluing formula for Gromov-Witten invariants in the case of a triple product. This gluing formula is a simple case of a much more general gluing formula proved and stated using exploded manifolds. We present this simple case because it is relatively easy to explain without any knowledge of exploded manifold…
This work tackles exploding inverses in INNs, revealing and mitigating their numerical non-invertibility.
Normalization layers are widely used in deep neural networks to stabilize training. In this paper, we consider the training of convolutional neural networks with gradient descent on a single training example. This optimization problem arises in recent approaches for solving inverse problems such as the deep image prior…
Developing active inference agents for edge devices with limited resources.
The exploding and vanishing gradient problem has been the major conceptual principle behind most architecture and training improvements in recurrent neural networks (RNNs) during the last decade. In this paper, we argue that this principle, while powerful, might need some refinement to explain recent developments. We r…
TSGO optimizes gradients in tensor networks to avoid vanishing/exploding issues.
Derives a new formula for optimal stopping problems with exploding derivatives.
Extends LIBOR market model to reduce exploding scenarios.
We prove here a general closed-form expansion formula for forward-start options and the forward implied volatility smile in a large class of models, including the Heston stochastic volatility and time-changed exponential Lévy models. This expansion applies to both small and large maturities and is based solely on the p…
BatchNorm helps BNNs avoid exploding gradients during training.
Signature volatility models are analyzed for existence, arbitrage, completeness, and hedging-error decomposition.
We introduce and show the existence of a Hawkes self-exciting point process with exponentially-decreasing kernel and where parameters are time-varying. The quantity of interest is defined as the integrated parameter , where is the time-varying parameter, and we consider the high-frequency…
A long-standing obstacle to progress in deep learning is the problem of vanishing and exploding gradients. Although, the problem has largely been overcome via carefully constructed initializations and batch normalization, architectures incorporating skip-connections such as highway and resnets perform much better than …
We define Gromov--Witten invariants of exploded manifolds. The technical heart of this paper is a construction of a virtual fundamental class of any Kuranishi category (which is a simplified, more general version of an embedded Kuranishi structure.) We also show how to integrate differential…
The weight initialization and the activation function of deep neural networks have a crucial impact on the performance of the training procedure. An inappropriate selection can lead to the loss of information of the input during forward propagation and the exponential vanishing/exploding of gradients during back-propag…
We identify and study two common failure modes for early training in deep ReLU nets. For each we give a rigorous proof of when it occurs and how to avoid it, for fully connected and residual architectures. The first failure mode, exploding/vanishing mean activation length, can be avoided by initializing weights from a …
Blind Descent avoids gradient issues, using a different learning approach.
New insights on stability in reservoir computing for better performance.
A major challenge in the training of recurrent neural networks is the so-called vanishing or exploding gradient problem. The use of a norm-preserving transition operator can address this issue, but parametrization is challenging. In this work we focus on unitary operators and describe a parametrization using the Lie al…
Maxout networks study gradients and propose initialization strategies.
Volume-preserving neural networks prevent gradient issues.
A new RNN model tackles long-time dependencies with fast, invertible, and memory-efficient hidden states.
ADMMiRNN solves RNN training issues with stable convergence.
In \cite{FOinteger}, Fukaya and Ono outlined a way of counting pseudo-holomorphic curves in a general compact symplectic manifold to obtain integer valued invariants. This paper contains the details of Fukaya and Ono's suggested construction for any compact symplectic manifold and a large class of exploded manifolds.
Paper presents a GAN model for realistic river image synthesis.
ENRNN uses eigenvalue normalization for short-term memory in RNNs.
A simple gating mechanism improves deep learning convergence.
Vanishing nodes cause hidden nodes to behave similarly, complicating deep neural network training.
iEFM trains CNF models from unnormalized densities efficiently.
We prove two tropical gluing formulae for Gromov-Witten invariants of exploded manifolds, useful for calculating Gromov-Witten invariants of a symplectic manifold using a normal-crossing degeneration. The first formula generalizes the symplectic-sum formula for Gromov-Witten invariants. The second formula is stronger, …
The weight initialization and the activation function of deep neural networks have a crucial impact on the performance of the training procedure. An inappropriate selection can lead to the loss of information of the input during forward propagation and the exponential vanishing/exploding of gradients during back-propag…
New RNN model handles long-term dependencies in irregularly-sampled time series.
Recurrent Neural Networks (RNN), Long Short-Term Memory Networks (LSTM), and Memory Networks which contain memory are popularly used to learn patterns in sequential data. Sequential data has long sequences that hold relationships. RNN can handle long sequences but suffers from the vanishing and exploding gradient probl…
A well-conditioned Jacobian spectrum has a vital role in preventing exploding or vanishing gradients and speeding up learning of deep neural networks. Free probability theory helps us to understand and handle the Jacobian spectrum. We rigorously show almost sure asymptotic freeness of layer-wise Jacobians of deep neura…
New method improves DAG learning by using large coefficients for higher-order terms.
Hamiltonian RNN controls hidden states gradient for long-term dependencies.
Conformally invariant functionals on the space of knots are introduced via extrinsic conformal geometry of the knot and integral geometry on the space of spheres. Our functionals are expressed in terms of a complex-valued 2-form which can be considered as the cross-ratio of a pair of infinitesimal segments of the knot.…
Residual Network (ResNet) is the state-of-the-art architecture that realizes successful training of really deep neural network. It is also known that good weight initialization of neural network avoids problem of vanishing/exploding gradients. In this paper, simplified models of ResNets are analyzed. We argue that good…