Leveraging advances in variational inference, we propose to enhance recurrent neural networks with latent variables, resulting in Stochastic Recurrent Networks (STORNs). The model i) can be trained with stochastic gradient methods, ii) allows structured and multi-modal conditionals at each time step, iii) features a re…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,181 papers · 148 categories
Trend · papers per month
48 results for “stochastic recurrent networks”
Stochastic models fail to outperform standard recurrent networks in sequence modeling.
problem Discrepancy in performance between stochastic and standard recurrent models in sequence modeling.
method Re-examine roles of latent variables, remove restriction on fully factorized output distribution, compare auto-regressive models.
result Standard recurrent models consistently outperform stochastic models in sequence modeling.
SIS-RNN improves model flexibility for sequential data.
problem Limited expressive power of existing stochastic RNNs.
method Semi-implicit variational inference for implicit latent representations.
result SIS-RNN outperforms existing methods in various tasks.
How can we efficiently propagate uncertainty in a latent state representation with recurrent neural networks? This paper introduces stochastic recurrent neural networks which glue a deterministic recurrent neural network and a state space model together to form a stochastic and sequential neural generative model. The c…
Deep neural networks solve stochastic control problems with delay.
problem Challenges in stochastic control problems with delay due to path-dependence and high dimensions.
method Employing recurrent neural networks (RNNs) to parameterize policies and optimize objectives.
result RNNs, especially LSTMs, efficiently capture path-dependence and outperform feedforward networks in training and performance.
This work introduces a new model for complex stochastic processes.
problem Difficulties in representing non-stationary distributions with conventional models.
method Recurrent Autoregressive Flows using normalizing flows with recurrent neural connections.
result Demonstrates the effectiveness of the proposed model through experiments.
Unified approach to training stochastic RNNs with latent variables.
problem Training generative latent variable models with autoregressive decoders.
method Amortized variational inference with backward RNN conditioning and auxiliary reconstruction cost.
result Improved performance on speech and sequential MNIST benchmarks.
This work combines recurrent models with diffusion for probabilistic time series forecasting.
problem Scalability and capturing high-dimensional distributions and cross-feature dependencies in time series forecasting.
method Combines recurrent neural networks' efficiency with diffusion models' probabilistic modeling, using stochastic interpolants and conditional generation.
result Offers scalable probabilistic time series forecasting methods.
SORSCNs improve nonstationary data modeling by self-organizing and adjusting network parameters.
problem Nonstationary data challenges traditional models in continuous learning.
method SORSCNs autonomously adjust network parameters and structure in real-time using adaptive algorithms.
result SORSCNs outperform other models in generalizing to nonstationary data.
DeepBayes uses neural networks to efficiently estimate parameters in complex dynamical models.
problem Estimating parameters in stochastic, nonlinear dynamical models is challenging.
method DeepBayes leverages deep recurrent neural networks to learn an estimator that minimizes mean-squared error.
result DeepBayes achieves asymptotically equivalent performance to Bayesian estimation methods.
Neural model improves volatility estimation and prediction in finance.
problem Improving volatility estimation and prediction in finance.
method Integrates deep neural networks with stochastic volatility models.
result Proposed model outperforms existing methods on average negative log-likelihood.
Robots learn quickly from few interactions using mental replay and intrinsic motivation.
problem Continuous online adaptation for robots in changing environments.
method Bio-inspired stochastic recurrent neural network with learning signals and mental replay.
result Robots can adapt to novel environments in seconds from few interactions.
Stochastic RNNs classify biological neural network paths with robust error bounds.
problem Classifying biological neural network paths.
method Modelled as a continuous-time stochastic recurrent neural network (RNN) with identity activation function, analysed in the robust regime.
result Generalisation error bound holds with high probability, showing the empirical risk minimiser is the best-in-class hypothesis.
Unified framework for efficient online training of RNNs.
problem Efficient and biologically plausible online training of recurrent neural networks.
method Organizes algorithms based on criteria like past vs. future facing, tensor structure, stochastic vs. deterministic, and closed form vs. numerical.
result Algorithms cluster according to criteria, revealing conceptual connections.
Combines SV and RNN models for stock market volatility.
problem Capturing complex volatility effects in stock markets.
method Statistical Recurrent Stochastic Volatility (SR-SV) model.
result Out-of-sample forecast performance is impressive.
Self-organized action hierarchy and compositionality learned by RNNs.
problem Improving RNN architectures for reinforcement learning.
method Multiple-timescale, stochastic RNN for RL.
result Network autonomously learns sub-goals and develops an action hierarchy.
Optimal stock price prediction model using recurrent neural networks with RMSprop optimizer.
problem Stock price prediction using neural networks.
method Comparison of fully connected, convolutional, and recurrent architectures; inclusion of three optimization techniques.
result Single layer recurrent neural network with RMSprop optimizer produces optimal results with validation and test MAE of 0.0150 and 0.0148 respectively.
Noise in RNNs promotes flatter minima and more stable dynamics.
problem Understanding and optimizing the training of RNNs with noise.
method Formalizing RNNs as stochastic differential equations and analyzing the effect of noise in the hidden states.
result Noise injection in RNNs leads to flatter minima, more stable dynamics, and improved robustness.
Nonlinear RNNs' memory capacity varies widely, making it impractical.
problem The usefulness of memory capacity as a metric for linear RNNs is questioned.
method Analysis of random nonlinear RNNs with varying input scales.
result Memory capacity of nonlinear RNNs is arbitrary and impractical.
Recurrent Ladder Networks improve iterative inference and temporal modeling.
problem Complex learning tasks requiring iterative inference and temporal modeling.
method Proposes a recurrent extension of Ladder networks.
result Shows close-to-optimal results on temporal modeling of video data and competitive results on music modeling.
Paper predicts travel costs across regions using neural networks.
problem Predicting travel costs in sparse, stochastic OD matrices.
method Recurrent Multi-Graph Neural Networks (R-MGNN) for sparse, stochastic OD matrix forecasting.
result Framework effectively predicts future OD matrices without empty elements.
This paper studies the performance of a recently proposed preconditioned stochastic gradient descent (PSGD) algorithm on recurrent neural network (RNN) training. PSGD adaptively estimates a preconditioner to accelerate gradient descent, and is designed to be simple, general and easy to use, as stochastic gradient desce…
Enhances RSCNs with hybrid regularization for nonlinear dynamics.
problem Modeling nonlinear dynamic systems with uncertainties.
method Recurrent stochastic configuration networks with hybrid regularization.
result The method outperforms other models in nonlinear system identification and industrial tasks.
Neural networks predict traffic flow in smart cities.
problem Forecasting stochastic and nonlinear traffic flow.
method Various recurrent neural networks trained on intersection data.
result Vector output model with gated recurrent units performed best.
CRUs model irregular time series with continuous hidden states.
problem Handling irregular time intervals in sequential data.
method Continuous Recurrent Units (CRUs) that integrate hidden states via a linear stochastic differential equation.
result CRUs outperform methods based on neural ordinary differential equations in irregular time series interpolation.
CPGAs converge to locally optimal policies for coagent networks.
problem Training stochastic neural networks using reinforcement learning.
method Proved convergence of CPGAs and extended prior theory to asynchronous and recurrent networks.
result CPGAs converge to locally optimal policies for coagent networks, including asynchronous and recurrent networks.
Paper introduces a new method for generating diverse human motion predictions.
problem Stochastic human motion prediction with limited flexibility.
method Stochastically combines root variations with previous pose information in a recurrent network.
result Model generates more diverse motion sequences than existing techniques.
RANP improves neural processes for sequential data.
problem Capturing temporal order and recurrent structure from sequential data.
method Incorporated ANP into a recurrent neural network.
result RANP outperforms NPs and LSTMs in 1D regression and autonomous-driving tasks.
Paper shows SGD can learn RNNs efficiently for certain functions.
problem Understanding what concept class RNNs can learn and how efficiently.
method Vanilla stochastic gradient descent (SGD) approach.
result RNNs can learn some functions efficiently with polynomial complexity in input length.
Inference for SDEs using variational methods and neural networks.
problem Parameter inference for stochastic differential equations is challenging due to latent diffusion processes.
method Variational inference with a mean-field approximation for parameters and a recurrent neural network for diffusion paths.
result Accurate parameter estimates for SDE systems, demonstrated on Lotka-Volterra and epidemic models.
DeepRSCN models nonlinear systems using stochastic configurations.
problem Modeling nonlinear dynamic systems efficiently.
method Incrementally constructed deep reservoir computing framework with random parameters and online weight updates.
result DeepRSCN outperforms single-layer networks in efficiency, learning, and generalization.
Mack-Net model combines Mack's model with RNNs for better insurance liability estimation.
problem Accurate estimation of insurance liabilities for better financial decision-making.
method Integrates Mack's reserving model with Recurrent Neural Networks (RNNs).
result Improves accuracy of general insurance liability assessment.
In this work we compare different batch construction methods for mini-batch training of recurrent neural networks. While popular implementations like TensorFlow and MXNet suggest a bucketing approach to improve the parallelization capabilities of the recurrent training process, we propose a simple ordering strategy tha…
Expressivity of Hidden Markov Chains vs. Recurrent Neural Networks from a system theoretic viewpointeess.SY
This paper compares HMC and RNN expressivity using SRT.
problem Comparing expressivity of HMC and RNN models.
method Embed HMC and RNN in a GUM, use SRT to compare structured covariance series.
result Conditions for realizing covariance series by GUM, HMC, or RNN.
Method learns to predict agent interactions from partial observations.
problem Predicting interactions between multiple agents from incomplete data.
method Graph-Structured Variational Recurrent Neural Network (Graph-VRNN) trained end-to-end.
result Graph-VRNN outperforms baselines on sports datasets.
Proposes a new RNN for language generation capturing long-range dependencies.
problem Capturing long-range word dependencies and sentence order in text corpora.
method Recurrent Hierarchical Topic-Guided RNN with dynamic deep topic model.
result Outperforms larger-context RNN-based language models and learns interpretable topics.
Study on state dynamics in Deep Echo State Networks, revealing the importance of inter-reservoir connections.
problem Understanding state dynamics in multi-layered RNNs.
method Tools from information theory and numerical analysis.
result Inter-reservoir connections enrich representations in higher layers of DeepESNs.
Deep neural nets solve complex stochastic control problems.
problem Solving stochastic optimal control problems with control multiplicative noise.
method Deep recurrent neural networks and LSTM.
result Deep learning algorithm solves complex stochastic control problems efficiently.
Neural GARCH models financial time series with time-varying coefficients.
problem Modeling conditional heteroskedasticity in financial time series.
method Neural network adaptation of GARCH and BEKK models with time-varying coefficients parameterized by a recurrent neural network.
result Neural Students t model consistently outperforms other models on financial time series.
Paper analyzes convergence of SGD in RNNs, proving linear convergence rate.
problem Analyzing convergence of SGD in multi-layer recurrent neural networks.
method Developed tools to analyze multi-layer networks with ReLU activations.
result SGD achieves linear convergence rate in training RNNs with sufficient neurons.
Stochastic gradient descent learns weights of state equations with nonlinear activations.
problem Learning weights of state equations with nonlinear activations using SGD.
method Utilizes stochastic gradient descent to learn weight matrices from input/state trajectories.
result SGD converges to ground truth weights with near-optimal sample size and linear convergence.
PAC-Bayesian bounds for stochastic LTI systems derived.
problem Error bounds for stochastic LTI systems.
method PAC-Bayesian theory applied to autonomous stochastic LTI models.
result Error bounds for stochastic LTI systems derived.
POLA adapts learning rates for online time series prediction.
problem Adapting to changing data distributions in dynamic environments.
method Adaptive learning rate regulation for recurrent neural networks.
result POLA outperforms other online prediction methods in real-world datasets.
New algorithm guarantees optimal convergence rate for stochastic optimization.
problem Optimal convergence rate for stochastic optimization algorithms.
method Regularized versions of Minimization by Incremental Surrogate Optimization (MISO) with arbitrary recurrent data sampling.
result Expected optimality gap converges at under general recurrent sampling schemes.
GAN model simulates 3D particle trajectories in flame zones.
problem Simulating 3D Lagrangian particle trajectories in flame zones.
method Generative adversarial network (GAN) model with stochastic and convoluted neural networks.
result Best-trained GAN model produced trajectories indistinguishable from ground truth.
We propose a method to infer stochastic low-rank RNNs from neural data.
problem Fitting low-rank RNNs to noisy, stochastic neural data.
method Variational sequential Monte Carlo methods for stochastic low-rank RNNs.
result Lower dimensional latent dynamics compared to state-of-the-art methods.
Sleep-based regularization stabilizes STDP in recurrent neural networks.
problem Pathological weight dynamics in recurrent SNNs.
method Periodic offline phases with stochastic decay and spontaneous activity.
result Sleep-based renormalization prevents weight saturation and preserves learned structure.
Introduces -Cell for improved financial volatility forecasting.
problem Improving volatility forecasting in financial markets.
method Combines GARCH and deep learning, incorporating stochastic layers and time-varying parameters.
result Demonstrates superior forecasting accuracy compared to traditional models.