Recurrent neural networks show state-of-the-art results in many text analysis tasks but often require a lot of memory to store their weights. Recently proposed Sparse Variational Dropout eliminates the majority of the weights in a feed-forward neural network without significant loss of quality. We apply this technique …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,042 papers · 148 categories
Trend · papers per month
48 results for “variational recurrent neural network”
GraphVRNN generates graphs with latent variables and node attributes.
problem Generating diverse and complex graph structures.
method Probabilistic autoregressive model for graph generation.
result GraphVRNN can model complicated distributions and generate plausible structures and node attributes.
SIS-RNN improves model flexibility for sequential data.
problem Limited expressive power of existing stochastic RNNs.
method Semi-implicit variational inference for implicit latent representations.
result SIS-RNN outperforms existing methods in various tasks.
Leveraging advances in variational inference, we propose to enhance recurrent neural networks with latent variables, resulting in Stochastic Recurrent Networks (STORNs). The model i) can be trained with stochastic gradient methods, ii) allows structured and multi-modal conditionals at each time step, iii) features a re…
GCRNNs improve graph problem solving with fewer parameters.
problem Graph process problems like earthquake epicenter identification and weather prediction.
method GCRNNs use convolutional filter banks and time-gated variations of GCRNNs (Gated GCRNNs) to improve performance.
result GCRNNs significantly improve performance over GNNs and another graph recurrent architecture.
How can we efficiently propagate uncertainty in a latent state representation with recurrent neural networks? This paper introduces stochastic recurrent neural networks which glue a deterministic recurrent neural network and a state space model together to form a stochastic and sequential neural generative model. The c…
Paper develops a new model for dynamic graph representation learning.
problem Learning over dynamic graphs with changing topology and node attributes.
method Hierarchical variational model with latent random variables and semi-implicit variational inference.
result SI-VGRNN and VGRNN outperform existing methods in dynamic link prediction.
Unsupervised RNNGs perform similarly to supervised ones in language modeling and grammar induction.
problem Training RNNGs requires annotated data, which is costly.
method Amortized variational inference with a neural CRF parser.
result Unsupervised RNNGs achieve comparable performance to supervised ones.
Proposes CVRCF for streaming recommender systems combining deep learning and probabilistic models.
problem Streaming recommendation problem with dynamic data and complexity.
method Coupled Variational Recurrent Collaborative Filtering (CVRCF) framework integrating stochastic processes and deep factorization models.
result Favorable performance in temporal dependency modeling and predictive accuracy compared to state-of-the-art methods.
VHVM models financial time series with varying volatility.
problem Modeling heteroscedastic behavior in multivariate financial time series.
method Variational autoencoder and recurrent neural network for capturing relationships and temporal dynamics.
result VHVM outperforms GARCH and SV models on FX datasets.
A new approach to learning in brain-like networks using adversarial algorithms.
problem Complex inter-dependencies in brain-like networks not compatible with conditional independence assumptions.
method Adversarial algorithm for learning models of perceptual processing.
result The approach can mimic known neural phenomena and yields testable hypotheses.
Variational Bayesian Sequence-to-Sequence Networks for Memory-Efficient Sign Language Translationstat.ML
A new memory-efficient sign language translation model reduces weight usage.
problem Memory constraints in real-time sign language translation.
method Variational Bayesian sequence-to-sequence network with Gaussian posterior and Indian Buffet Process prior.
result The proposed model achieves substantial weight compression without compromising performance.
In this paper, we use variational recurrent neural network to investigate the anomaly detection problem on graph time series. The temporal correlation is modeled by the combination of recurrent neural network (RNN) and variational inference (VI), while the spatial information is captured by the graph convolutional netw…
Proposes a new RNN for language generation capturing long-range dependencies.
problem Capturing long-range word dependencies and sentence order in text corpora.
method Recurrent Hierarchical Topic-Guided RNN with dynamic deep topic model.
result Outperforms larger-context RNN-based language models and learns interpretable topics.
Neural GARCH models financial time series with time-varying coefficients.
problem Modeling conditional heteroskedasticity in financial time series.
method Neural network adaptation of GARCH and BEKK models with time-varying coefficients parameterized by a recurrent neural network.
result Neural Students t model consistently outperforms other models on financial time series.
Bayesian meta-reinforcement learning improves over point estimates with Laplace approximation.
problem Improving meta-reinforcement learning by providing full posterior distributions.
method Augmenting point estimates with Laplace approximation for full posterior distributions.
result Our method performs similarly to variational baselines with fewer parameters.
Bayesian Recurrent Neural Networks improve fault detection and identification in manufacturing.
problem Detect and identify faults in chemical processes to ensure optimal operations.
method Bayesian Recurrent Neural Networks (BRNNs) with variational dropout.
result BRNNs provide uncertainty estimates for fault detection and identification.
Method learns to predict agent interactions from partial observations.
problem Predicting interactions between multiple agents from incomplete data.
method Graph-Structured Variational Recurrent Neural Network (Graph-VRNN) trained end-to-end.
result Graph-VRNN outperforms baselines on sports datasets.
Recurrent neural networks with various types of hidden units have been used to solve a diverse range of problems involving sequence data. Two of the most recent proposals, gated recurrent units (GRU) and minimal gated units (MGU), have shown comparable promising results on example public datasets. In this paper, we int…
In this paper, we show that the recent integration of statistical models with deep recurrent neural networks provides a new way of formulating volatility (the degree of variation of time series) models that have been widely used in time series analysis and prediction in finance. The model comprises a pair of complement…
Recurrent Neural Networks with more flexible memory: better predictions than rough volatilityq-fin.ST
Extended LSTMs improve volatility prediction by 20%.
problem Predicting asset price volatility with long memory.
method Extended LSTMs with multiple flexible timescales.
result Extended LSTMs outperform rough volatility predictions by 20%.
Recurrent neural networks (RNNs) stand at the forefront of many recent developments in deep learning. Yet a major difficulty with these models is their tendency to overfit, with dropout shown to fail when applied to recurrent layers. Recent results at the intersection of Bayesian modelling and deep learning offer a Bay…
In this work we explore a straightforward variational Bayes scheme for Recurrent Neural Networks. Firstly, we show that a simple adaptation of truncated backpropagation through time can yield good quality uncertainty estimates and superior regularisation at only a small extra computational cost during training, also re…
Many works have been proposed in the literature to capture the dynamics of diffusion in networks. While some of them define graphical markovian models to extract temporal relationships between node infections in networks, others consider diffusion episodes as sequences of infections via recurrent neural models. In this…
Many efforts have been devoted to training generative latent variable models with autoregressive decoders, such as recurrent neural networks (RNN). Stochastic recurrent models have been successful in capturing the variability observed in natural sequential data such as speech. We unify successful ideas from recently pr…
A new model integrates LSTM and copulas for high-dimensional financial data.
problem Modeling high-dimensional dependencies across financial markets.
method Variational LSTM with regular vine copulas.
result Outperforms benchmarks in cross-market portfolio forecasting.
VSE estimates complex processes from noisy measurements without a model.
problem Estimating states of complex, model-free processes from noisy data.
method Variational state estimation using recurrent neural networks (RNNs) in both learning and inference phases.
result VSE provides a competitive state estimate for a benchmark process (Lorenz system) compared to known and data-driven methods.
Model integrates multi-view temporal data for better understanding of latent dynamics.
problem Understanding time-dependent heterogeneous properties from multi-view data.
method Generative model using variational autoencoder and recurrent neural network.
result Identifies disentangled latent embeddings across views while accounting for time factor.
This paper develops a novel deep recurrent neural network for sequential signal reconstruction.
problem Sequential signal reconstruction from low-dimensional measurements.
method Unfolding a reweighted - minimization algorithm to design a deep recurrent neural network.
result The proposed reweighted-RNN significantly outperforms existing RNN models in sequential frame reconstruction.
Interneurons accelerate learning dynamics in recurrent neural networks for statistical adaptationq-bio.NC
Interneurons improve learning in neural networks by accelerating convergence.
problem Rapid adaptation to changing input statistics in neural networks.
method Two mathematically tractable recurrent linear neural networks were compared: one with direct recurrent connections and the other with interneurons that mediate recurrent communication.
result The network with interneurons converges more quickly than the network with direct recurrent connections, scaling logarithmically with initialization spectrum.
QRPNNs use quaternion-valued recurrent correlation neural networks to solve cross-talk issues.
problem Cross-talk problem in QRCNNs.
method Combining non-local projection learning with QRCNNs.
result QRPNNs exhibit greater storage capacity and noise tolerance.
VOWEL trains WTA-SNNs for multi-valued events, overcoming resource limitations.
problem Training WTA-SNNs for multi-valued events is challenging due to non-differentiability and recurrent behavior.
method Develops a variational online local training rule (VOWEL) for WTA-SNNs using local pre- and post-synaptic information and a common reward signal.
result VOWEL outperforms conventional binary SNNs in real-world neuromorphic datasets with multi-valued events.
QRNN uses quantum neurons to learn sequences efficiently.
problem Efficiently learning sequences with quantum computing.
method Parametrized quantum neurons and amplitude amplification.
result QRNN outperforms classical RNNs on sequence learning tasks.
RecNets use RNNs to process image channels in a compact, recurrent way.
problem Creating efficient neural network architectures for computer vision.
method Introducing RecNets with CRC layers that simulate recurrent processing of image channels.
result RecNets achieve superior size-accuracy trade-off compared to other compact models.
DVE models dynamic changes in feature embeddings for better sequence-aware applications.
problem Dynamic characterization of feature variations in time-varying data.
method Dynamic Variational Embedding (DVE) using recurrent neural networks.
result DVE models intrinsic nature and temporal variation of nodes effectively.
We define Recurrent Gaussian Processes (RGP) models, a general family of Bayesian nonparametric models with recurrent GP priors which are able to learn dynamical patterns from sequential data. Similar to Recurrent Neural Networks (RNNs), RGPs can have different formulations for their internal states, distinct inference…
We propose a method to infer stochastic low-rank RNNs from neural data.
problem Fitting low-rank RNNs to noisy, stochastic neural data.
method Variational sequential Monte Carlo methods for stochastic low-rank RNNs.
result Lower dimensional latent dynamics compared to state-of-the-art methods.
Variational autoencoders were proven successful in domains such as computer vision and speech processing. Their adoption for modeling user preferences is still unexplored, although recently it is starting to gain attention in the current literature. In this work, we propose a model which extends variational autoencoder…
VSDN models sporadic time series with neural SDEs.
problem Modeling irregular and sparse time series data.
method Variational Bayesian method and neural SDEs.
result VSDNs outperform state-of-the-art models in prediction and interpolation.
New clustering algorithm for time series data using RNN and variational Bayes.
problem Lack of generative model-based clustering methods for time series data.
method Recurrent Neural Network (RNN) with variational Bayes method.
result Robustness against phase shift, amplitude, and signal length variations.
Algorithm discovers dynamic cell structures for better neural network performance.
problem Finding optimal neural network architectures for diverse data samples and time steps.
method Combines recurrent and recursive neural networks to dynamically search for customized cell structures.
result Achieves better prediction accuracy compared to existing models.
Ultra-low latency recurrent neural network inference on FPGAs for physics applications with hls4mlcs.LG
Paper presents FPGA implementation for efficient recurrent neural networks.
problem Implementing recurrent neural networks on FPGAs for low latency.
method Developed hls4ml framework to implement LSTM and GRU layers.
result Demonstrated effective designs for both small and large models.
Natural spatiotemporal processes can be highly non-stationary in many ways, e.g. the low-level non-stationarity such as spatial correlations or temporal dependencies of local pixel values; and the high-level variations such as the accumulation, deformation or dissipation of radar echoes in precipitation forecasting. Fr…
This study proposes methods for multi-step-ahead stock price prediction using decomposition and neural networks.
problem Inaccurate one-step-ahead forecasting limits stock market decision-making.
method Two novel methods: DCT-MFRFNN and VMD-MFRFNN.
result VMD-MFRFNN outperforms other methods in multi-step-ahead stock price prediction.
Scalable verifier for recurrent neural networks using polyhedral abstractions.
problem Certifying the correctness of recurrent neural networks.
method Combining sampling, optimization, and Fermat's theorem for polyhedral abstractions; gradient descent for refinement.
result Successfully verified challenging recurrent models in various domains.
Knowledge extraction from the learning of sequences in a long short term memory (LSTM) architecturecs.LG
Method extracts knowledge from LSTM for sequence validation.
problem Validating sequences generated by unknown automata.
method Clustering hidden states to build a corresponding automaton.
result Automaton accurately predicts sequence validity.
This paper analyzes the generalization risk of unrolled neural networks using Stein's Unbiased Risk Estimator.
problem Analyzing the generalization risk of unrolled neural networks and its relationship to network design and train sample size.
method Using Stein's Unbiased Risk Estimator (SURE), the paper analyzes the generalization risk with bias and variance components for recurrent unrolled networks, focusing on the degrees-of-freedom (DOF) component and the trace of the end-to-end network Jacobian.
result DOF is well-approximated by the weighted path sparsity of the network under incoherence conditions on the trained weights, and DOF increases with train sample size and converges to the generalization risk for both recurrent and non-recurrent schemes.
Bayesian approach improves neural network recurrence.
problem Improving neural network recurrence mechanisms.
method Introducing Bayesian recurrence relations and gates.
result Bayesian approach can perform as well as or better than conventional recurrent networks.