Recurrent neural networks show state-of-the-art results in many text analysis tasks but often require a lot of memory to store their weights. Recently proposed Sparse Variational Dropout eliminates the majority of the weights in a feed-forward neural network without significant loss of quality. We apply this technique …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Variational Recurrent Neural Network”
GraphVRNN generates graphs with latent variables and node attributes.
problem Generating diverse and complex graph structures.
method Probabilistic autoregressive model for graph generation.
result GraphVRNN can model complicated distributions and generate plausible structures and node attributes.
SIS-RNN improves model flexibility for sequential data.
problem Limited expressive power of existing stochastic RNNs.
method Semi-implicit variational inference for implicit latent representations.
result SIS-RNN outperforms existing methods in various tasks.
Leveraging advances in variational inference, we propose to enhance recurrent neural networks with latent variables, resulting in Stochastic Recurrent Networks (STORNs). The model i) can be trained with stochastic gradient methods, ii) allows structured and multi-modal conditionals at each time step, iii) features a re…
How can we efficiently propagate uncertainty in a latent state representation with recurrent neural networks? This paper introduces stochastic recurrent neural networks which glue a deterministic recurrent neural network and a state space model together to form a stochastic and sequential neural generative model. The c…
VHVM models financial time series with varying volatility.
problem Modeling heteroscedastic behavior in multivariate financial time series.
method Variational autoencoder and recurrent neural network for capturing relationships and temporal dynamics.
result VHVM outperforms GARCH and SV models on FX datasets.
A new approach to learning in brain-like networks using adversarial algorithms.
problem Complex inter-dependencies in brain-like networks not compatible with conditional independence assumptions.
method Adversarial algorithm for learning models of perceptual processing.
result The approach can mimic known neural phenomena and yields testable hypotheses.
Variational Bayesian Sequence-to-Sequence Networks for Memory-Efficient Sign Language Translationstat.ML
A new memory-efficient sign language translation model reduces weight usage.
problem Memory constraints in real-time sign language translation.
method Variational Bayesian sequence-to-sequence network with Gaussian posterior and Indian Buffet Process prior.
result The proposed model achieves substantial weight compression without compromising performance.
In this paper, we use variational recurrent neural network to investigate the anomaly detection problem on graph time series. The temporal correlation is modeled by the combination of recurrent neural network (RNN) and variational inference (VI), while the spatial information is captured by the graph convolutional netw…
Proposes a new RNN for language generation capturing long-range dependencies.
problem Capturing long-range word dependencies and sentence order in text corpora.
method Recurrent Hierarchical Topic-Guided RNN with dynamic deep topic model.
result Outperforms larger-context RNN-based language models and learns interpretable topics.
Neural GARCH models financial time series with time-varying coefficients.
problem Modeling conditional heteroskedasticity in financial time series.
method Neural network adaptation of GARCH and BEKK models with time-varying coefficients parameterized by a recurrent neural network.
result Neural Students t model consistently outperforms other models on financial time series.
We focus on the problem of streaming recommender system and explore novel collaborative filtering algorithms to handle the data dynamicity and complexity in a streaming manner. Although deep neural networks have demonstrated the effectiveness of recommendation tasks, it is lack of explorations on integrating probabilis…
Bayesian meta-reinforcement learning improves over point estimates with Laplace approximation.
problem Improving meta-reinforcement learning by providing full posterior distributions.
method Augmenting point estimates with Laplace approximation for full posterior distributions.
result Our method performs similarly to variational baselines with fewer parameters.
Bayesian Recurrent Neural Networks improve fault detection and identification in manufacturing.
problem Detect and identify faults in chemical processes to ensure optimal operations.
method Bayesian Recurrent Neural Networks (BRNNs) with variational dropout.
result BRNNs provide uncertainty estimates for fault detection and identification.
Graph processes model a number of important problems such as identifying the epicenter of an earthquake or predicting weather. In this paper, we propose a Graph Convolutional Recurrent Neural Network (GCRNN) architecture specifically tailored to deal with these problems. GCRNNs use convolutional filter banks to keep th…
Recurrent neural networks with various types of hidden units have been used to solve a diverse range of problems involving sequence data. Two of the most recent proposals, gated recurrent units (GRU) and minimal gated units (MGU), have shown comparable promising results on example public datasets. In this paper, we int…
In this paper, we show that the recent integration of statistical models with deep recurrent neural networks provides a new way of formulating volatility (the degree of variation of time series) models that have been widely used in time series analysis and prediction in finance. The model comprises a pair of complement…
Recurrent Neural Networks with more flexible memory: better predictions than rough volatilityq-fin.ST
Extended LSTMs improve volatility prediction by 20%.
problem Predicting asset price volatility with long memory.
method Extended LSTMs with multiple flexible timescales.
result Extended LSTMs outperform rough volatility predictions by 20%.
Recurrent neural networks (RNNs) stand at the forefront of many recent developments in deep learning. Yet a major difficulty with these models is their tendency to overfit, with dropout shown to fail when applied to recurrent layers. Recent results at the intersection of Bayesian modelling and deep learning offer a Bay…
Representation learning over graph structured data has been mostly studied in static graph settings while efforts for modeling dynamic graphs are still scant. In this paper, we develop a novel hierarchical variational model that introduces additional latent random variables to jointly model the hidden states of a graph…
In this work we explore a straightforward variational Bayes scheme for Recurrent Neural Networks. Firstly, we show that a simple adaptation of truncated backpropagation through time can yield good quality uncertainty estimates and superior regularisation at only a small extra computational cost during training, also re…
Recurrent neural network grammars (RNNG) are generative models of language which jointly model syntax and surface structure by incrementally generating a syntax tree and sentence in a top-down, left-to-right order. Supervised RNNGs achieve strong language modeling and parsing performance, but require an annotated corpu…
Many works have been proposed in the literature to capture the dynamics of diffusion in networks. While some of them define graphical markovian models to extract temporal relationships between node infections in networks, others consider diffusion episodes as sequences of infections via recurrent neural models. In this…
Many efforts have been devoted to training generative latent variable models with autoregressive decoders, such as recurrent neural networks (RNN). Stochastic recurrent models have been successful in capturing the variability observed in natural sequential data such as speech. We unify successful ideas from recently pr…
A new model integrates LSTM and copulas for high-dimensional financial data.
problem Modeling high-dimensional dependencies across financial markets.
method Variational LSTM with regular vine copulas.
result Outperforms benchmarks in cross-market portfolio forecasting.
VSE estimates complex processes from noisy measurements without a model.
problem Estimating states of complex, model-free processes from noisy data.
method Variational state estimation using recurrent neural networks (RNNs) in both learning and inference phases.
result VSE provides a competitive state estimate for a benchmark process (Lorenz system) compared to known and data-driven methods.
Model integrates multi-view temporal data for better understanding of latent dynamics.
problem Understanding time-dependent heterogeneous properties from multi-view data.
method Generative model using variational autoencoder and recurrent neural network.
result Identifies disentangled latent embeddings across views while accounting for time factor.
This paper develops a novel deep recurrent neural network for sequential signal reconstruction.
problem Sequential signal reconstruction from low-dimensional measurements.
method Unfolding a reweighted - minimization algorithm to design a deep recurrent neural network.
result The proposed reweighted-RNN significantly outperforms existing RNN models in sequential frame reconstruction.
QRPNNs use quaternion-valued recurrent correlation neural networks to solve cross-talk issues.
problem Cross-talk problem in QRCNNs.
method Combining non-local projection learning with QRCNNs.
result QRPNNs exhibit greater storage capacity and noise tolerance.
Interneurons accelerate learning dynamics in recurrent neural networks for statistical adaptationq-bio.NC
Interneurons improve learning in neural networks by accelerating convergence.
problem Rapid adaptation to changing input statistics in neural networks.
method Two mathematically tractable recurrent linear neural networks were compared: one with direct recurrent connections and the other with interneurons that mediate recurrent communication.
result The network with interneurons converges more quickly than the network with direct recurrent connections, scaling logarithmically with initialization spectrum.
VOWEL trains WTA-SNNs for multi-valued events, overcoming resource limitations.
problem Training WTA-SNNs for multi-valued events is challenging due to non-differentiability and recurrent behavior.
method Develops a variational online local training rule (VOWEL) for WTA-SNNs using local pre- and post-synaptic information and a common reward signal.
result VOWEL outperforms conventional binary SNNs in real-world neuromorphic datasets with multi-valued events.
QRNN uses quantum neurons to learn sequences efficiently.
problem Efficiently learning sequences with quantum computing.
method Parametrized quantum neurons and amplitude amplification.
result QRNN outperforms classical RNNs on sequence learning tasks.
DVE models dynamic changes in feature embeddings for better sequence-aware applications.
problem Dynamic characterization of feature variations in time-varying data.
method Dynamic Variational Embedding (DVE) using recurrent neural networks.
result DVE models intrinsic nature and temporal variation of nodes effectively.
We define Recurrent Gaussian Processes (RGP) models, a general family of Bayesian nonparametric models with recurrent GP priors which are able to learn dynamical patterns from sequential data. Similar to Recurrent Neural Networks (RNNs), RGPs can have different formulations for their internal states, distinct inference…
We propose a method to infer stochastic low-rank RNNs from neural data.
problem Fitting low-rank RNNs to noisy, stochastic neural data.
method Variational sequential Monte Carlo methods for stochastic low-rank RNNs.
result Lower dimensional latent dynamics compared to state-of-the-art methods.
Variational autoencoders were proven successful in domains such as computer vision and speech processing. Their adoption for modeling user preferences is still unexplored, although recently it is starting to gain attention in the current literature. In this work, we propose a model which extends variational autoencoder…
VSDN models sporadic time series with neural SDEs.
problem Modeling irregular and sparse time series data.
method Variational Bayesian method and neural SDEs.
result VSDNs outperform state-of-the-art models in prediction and interpolation.
New clustering algorithm for time series data using RNN and variational Bayes.
problem Lack of generative model-based clustering methods for time series data.
method Recurrent Neural Network (RNN) with variational Bayes method.
result Robustness against phase shift, amplitude, and signal length variations.
Ultra-low latency recurrent neural network inference on FPGAs for physics applications with hls4mlcs.LG
Paper presents FPGA implementation for efficient recurrent neural networks.
problem Implementing recurrent neural networks on FPGAs for low latency.
method Developed hls4ml framework to implement LSTM and GRU layers.
result Demonstrated effective designs for both small and large models.
Natural spatiotemporal processes can be highly non-stationary in many ways, e.g. the low-level non-stationarity such as spatial correlations or temporal dependencies of local pixel values; and the high-level variations such as the accumulation, deformation or dissipation of radar echoes in precipitation forecasting. Fr…
This study proposes methods for multi-step-ahead stock price prediction using decomposition and neural networks.
problem Inaccurate one-step-ahead forecasting limits stock market decision-making.
method Two novel methods: DCT-MFRFNN and VMD-MFRFNN.
result VMD-MFRFNN outperforms other methods in multi-step-ahead stock price prediction.
Scalable verifier for recurrent neural networks using polyhedral abstractions.
problem Certifying the correctness of recurrent neural networks.
method Combining sampling, optimization, and Fermat's theorem for polyhedral abstractions; gradient descent for refinement.
result Successfully verified challenging recurrent models in various domains.
Knowledge extraction from the learning of sequences in a long short term memory (LSTM) architecturecs.LG
Method extracts knowledge from LSTM for sequence validation.
problem Validating sequences generated by unknown automata.
method Clustering hidden states to build a corresponding automaton.
result Automaton accurately predicts sequence validity.
Bayesian approach improves neural network recurrence.
problem Improving neural network recurrence mechanisms.
method Introducing Bayesian recurrence relations and gates.
result Bayesian approach can perform as well as or better than conventional recurrent networks.
In this work, we attempt to ameliorate the impact of data sparsity in the context of session-based recommendation. Specifically, we seek to devise a machine learning mechanism capable of extracting subtle and complex underlying temporal dynamics in the observed session data, so as to inform the recommendation algorithm…
Parameter inference for stochastic differential equations is challenging due to the presence of a latent diffusion process. Working with an Euler-Maruyama discretisation for the diffusion, we use variational inference to jointly learn the parameters and the diffusion paths. We use a standard mean-field variational appr…
Time series forecasting is difficult. It is difficult even for recurrent neural networks with their inherent ability to learn sequentiality. This article presents a recurrent neural network based time series forecasting framework covering feature engineering, feature importances, point and interval predictions, and for…
Proposes a model to estimate treatment effects in complex multiagent systems over time.
problem Challenges in evaluating interventions in multiagent systems, especially with time-varying relationships and covariates.
method Interpretable counterfactual recurrent network leveraging graph variational recurrent neural networks and domain knowledge.
result Achieved lower estimation errors and more effective treatment timing than baselines in simulated and real-world scenarios.