arXiv research

Researchers study heavy-tail properties of SGD using stochastic recurrence equations.

CRUs model irregular time series with continuous hidden states.

We present an adaptive approach for valuing the European call option on assets with stochastic volatility. The essential feature of the method is a reduction of uncertainty in latent volatility due to a Bayesian learning procedure. Starting from a discrete-time stochastic volatility model, we derive a recurrence equati…

Noise in RNNs promotes flatter minima and more stable dynamics.

This paper presents a novel approach to numerically solve stochastic differential games for nonlinear systems. The proposed approach relies on the nonlinear Feynman-Kac theorem that establishes a connection between parabolic deterministic partial differential equations and forward-backward stochastic differential equat…

We present a deep recurrent neural network architecture to solve a class of stochastic optimal control problems described by fully nonlinear Hamilton Jacobi Bellmanpartial differential equations. Such PDEs arise when one considers stochastic dynamics characterized by uncertainties that are additive and control multipli…

Leveraging advances in variational inference, we propose to enhance recurrent neural networks with latent variables, resulting in Stochastic Recurrent Networks (STORNs). The model i) can be trained with stochastic gradient methods, ii) allows structured and multi-modal conditionals at each time step, iii) features a re…

This work introduces a new model for complex stochastic processes.

VSDN models sporadic time series with neural SDEs.

With latent variables, stochastic recurrent models have achieved state-of-the-art performance in modeling sound-wave sequence. However, opposite results are also observed in other domains, where standard recurrent networks often outperform stochastic models. To better understand this discrepancy, we re-examine the role…

How can we efficiently propagate uncertainty in a latent state representation with recurrent neural networks? This paper introduces stochastic recurrent neural networks which glue a deterministic recurrent neural network and a state space model together to form a stochastic and sequential neural generative model. The c…

New algorithm guarantees optimal convergence rate for stochastic optimization.

Method combines deep learning and elicitability for solving complex stochastic equations.

Parameter inference for stochastic differential equations is challenging due to the presence of a latent diffusion process. Working with an Euler-Maruyama discretisation for the diffusion, we use variational inference to jointly learn the parameters and the diffusion paths. We use a standard mean-field variational appr…

RNN operators solve Newton's equations with large timesteps for molecular dynamics.

Deep signature/log-signature FBSDE algorithm improves accuracy and training time.

In this work we propose a new class of long-memory models with time-varying fractional parameter. In particular, the dynamics of the long-memory coefficient, $d$, is specified through a stochastic recurrence equation driven by the score of the predictive likelihood, as suggested by Creal et al. (2013) and Harvey (2013)…

This work combines recurrent models with diffusion for probabilistic time series forecasting.

Deep neural networks solve stochastic control problems with delay.

Designing video prediction models that account for the inherent uncertainty of the future is challenging. Most works in the literature are based on stochastic image-autoregressive recurrent networks, which raises several performance and applicability issues. An alternative is to use fully latent temporal models which u…

The paper introduces Causal Neural Operators to approximate operators in stochastic analysis.

Improved GRU model with weighted time-delay feedback for long-term dependencies.

Stochastic recurrent neural networks with latent random variables of complex dependency structures have shown to be more successful in modeling sequential data than deterministic deep models. However, the majority of existing methods have limited expressive power due to the Gaussian assumption of latent variables. In t…

DeepBayes uses neural networks to efficiently estimate parameters in complex dynamical models.

We study discrete time dynamical systems governed by the state equation $h_{t+1}=φ(Ah_t+Bu_t)$. Here $A,B$ are weight matrices, $φ$ is an activation function, and $u_t$ is the input data. This relation is the backbone of recurrent neural networks (e.g. LSTMs) which have broad applications in sequential learning tasks. …

In this paper, we show that the recent integration of statistical models with deep recurrent neural networks provides a new way of formulating volatility (the degree of variation of time series) models that have been widely used in time series analysis and prediction in finance. The model comprises a pair of complement…

Many efforts have been devoted to training generative latent variable models with autoregressive decoders, such as recurrent neural networks (RNN). Stochastic recurrent models have been successful in capturing the variability observed in natural sequential data such as speech. We unify successful ideas from recently pr…

SORSCNs improve nonstationary data modeling by self-organizing and adjusting network parameters.

Modeling wind dynamics in Saudi Arabia using deep learning and stochastic PDEs.

In an asset return series there is a conditional asymmetric dependence between current return and past volatility depending on the current return's sign. To take into account the conditional asymmetry, we introduce new models for asset return dynamics in which frequencies of the up and down movements of asset price hav…

Neural differential equations combine deep learning and differential equations for modeling complex systems.

Dynamical systems can be modelled by partial differential equations and numerical computations are used everywhere in science and engineering. In this work, we investigate the performance of recurrent and convolutional deep neural network architectures to predict the surface waves. The system is governed by the Saint-V…

Recurrent neural networks have gained widespread use in modeling sequential data. Learning long-term dependencies using these models remains difficult though, due to exploding or vanishing gradients. In this paper, we draw connections between recurrent networks and ordinary differential equations. A special form of rec…

Unified approach combining BSDEs and PINNs for solving PDEs.

Paper characterizes equilibrium strategies for stochastic control with higher-order moments.

The Stochastic Volatility (SV) model and its variants are widely used in the financial sector while recurrent neural network (RNN) models are successfully used in many large-scale industrial applications of Deep Learning. Our article combines these two methods in a non-trivial way and proposes a model, which we call th…

Autonomous robots need to interact with unknown, unstructured and changing environments, constantly facing novel challenges. Therefore, continuous online adaptation for lifelong-learning and the need of sample-efficient mechanisms to adapt to changes in the environment, the constraints, the tasks, or the robot itself a…

We present a model of a basic recurrent neural network (or bRNN) that includes a separate linear term with a slightly "stable" fixed matrix to guarantee bounded solutions and fast dynamic response. We formulate a state space viewpoint and adapt the constrained optimization Lagrange Multiplier (CLM) technique and the ve…

The Einstein Equation on 4-dimensional Lorentzian manifolds admitting recurrent null vector fields is discussed. Several examples of a special form are constructed. The holonomy algebras, Petrov types and the Lie algebras of Killing vector fields of the obtained metrics are found.

The simplest non-trivial solutions of WDVV equations are A_n and B_n-potentials, which describe metrics of K.Saito on spaces of versal deformation of A_n and B_n-singularities. These are some polynomials, which were known for $n\leqslant$ 4. We find some recurrence relations, which give a possibility to find all A_n an…

This paper studies the performance of a recently proposed preconditioned stochastic gradient descent (PSGD) algorithm on recurrent neural network (RNN) training. PSGD adaptively estimates a preconditioner to accelerate gradient descent, and is designed to be simple, general and easy to use, as stochastic gradient desce…

The paper develops a model for sovereign debt dynamics with explicit maturity structure.

Sig-DEG speeds up diffusion models by distilling them into faster approximations.

Recurrent iterated function systems (RIFSs) are improvements of iterated function systems (IFSs) using elements of the theory of Marcovian stochastic processes which can produce more natural looking images. We construct new RIFSs consisting substantially of a vertical contraction factor function and nonlinear transform…

Nonlinear RNNs' memory capacity varies widely, making it impractical.

Stochastic RNNs classify biological neural network paths with robust error bounds.

Extended recurrent pseudo-Riemannian manifolds were introduced by Mileva Prvanovic'. We reconsider her work in the light of recent results and show that the manifold is conformally flat, and it is a space of quasi-constant curvature. We also show that an extended recurrent Lorentzian manifold, with time-like associated…

Optimal stock price prediction model using recurrent neural networks with RMSprop optimizer.