Bayesian approach improves neural network recurrence.
problem Improving neural network recurrence mechanisms.
method Introducing Bayesian recurrence relations and gates.
result Bayesian approach can perform as well as or better than conventional recurrent networks.
Bayesian units improve speech recognition with minimal parameters.
problem Improving speech recognition models with fewer parameters.
method Derived Bayesian recurrent units integrated into deep learning frameworks.
result Adding Bayesian units improves speech recognition performance.
This study uses neural networks to approximate Bayesian filtering problems.
problem Estimating latent time-series signal statistics from observation sequences.
method Formulated a generic recurrent neural network framework to learn recursive mappings directly.
result Approximation error bounds for filtering in non-compact domains and strong time-uniform bounds.
Despite the recent popularity of deep generative state space models, few comparisons have been made between network architectures and the inference steps of the Bayesian filtering framework -- with most models simultaneously approximating both state transition and update steps with a single recurrent neural network (RN…
Recurrent neural networks show state-of-the-art results in many text analysis tasks but often require a lot of memory to store their weights. Recently proposed Sparse Variational Dropout eliminates the majority of the weights in a feed-forward neural network without significant loss of quality. We apply this technique …
In this work we explore a straightforward variational Bayes scheme for Recurrent Neural Networks. Firstly, we show that a simple adaptation of truncated backpropagation through time can yield good quality uncertainty estimates and superior regularisation at only a small extra computational cost during training, also re…
A new memory-efficient sign language translation model reduces weight usage.
problem Memory constraints in real-time sign language translation.
method Variational Bayesian sequence-to-sequence network with Gaussian posterior and Indian Buffet Process prior.
result The proposed model achieves substantial weight compression without compromising performance.
Adaptive vehicle trajectory prediction for safer autonomous driving.
problem Inability of current methods to guarantee physical feasibility and adapt to human driving policies.
method Bayesian recurrent neural network combining policy and physical models, with gradient-based training and parameter adaptation.
result The proposed method ensures physical feasibility and adaptability to human driving policies.
Bayesian model detects altered neural circuits in MCI patients.
problem Detecting altered neural circuits in Mild Cognitive Impairment patients.
method Hierarchical Bayesian recurrent state space model.
result Model discovers latent states predominantly observed in MCI patients.
Bayesian Recurrent Neural Networks improve fault detection and identification in manufacturing.
problem Detect and identify faults in chemical processes to ensure optimal operations.
method Bayesian Recurrent Neural Networks (BRNNs) with variational dropout.
result BRNNs provide uncertainty estimates for fault detection and identification.
CNN-F uses generative feedback to improve neural networks' robustness to perturbations.
problem Neural networks' vulnerability to input perturbations like noise and attacks.
method Enforces self-consistency in neural networks by incorporating generative recurrent feedback.
result CNN-F shows significantly improved adversarial robustness compared to conventional CNNs.
Bayesian methods have been successfully applied to sparsify weights of neural networks and to remove structure units from the networks, e. g. neurons. We apply and further develop this approach for gated recurrent architectures. Specifically, in addition to sparsification of individual weights and neurons, we propose t…
Bayesian meta-reinforcement learning improves over point estimates with Laplace approximation.
problem Improving meta-reinforcement learning by providing full posterior distributions.
method Augmenting point estimates with Laplace approximation for full posterior distributions.
result Our method performs similarly to variational baselines with fewer parameters.
In this paper we introduce ZhuSuan, a python probabilistic programming library for Bayesian deep learning, which conjoins the complimentary advantages of Bayesian methods and deep learning. ZhuSuan is built upon Tensorflow. Unlike existing deep learning libraries, which are mainly designed for deterministic neural netw…
Recurrent neural networks (RNNs) stand at the forefront of many recent developments in deep learning. Yet a major difficulty with these models is their tendency to overfit, with dropout shown to fail when applied to recurrent layers. Recent results at the intersection of Bayesian modelling and deep learning offer a Bay…
This paper considers the computational power of constant size, dynamic Bayesian networks. Although discrete dynamic Bayesian networks are no more powerful than hidden Markov models, dynamic Bayesian networks with continuous random variables and discrete children of continuous parents are capable of performing Turing-co…
Paper derives PAC-Bayesian bounds for LTI systems learning from empirical data.
problem Characterizing predictive power of LTI systems learned from data.
method PAC-Bayesian bounds for LTI stochastic dynamical systems with inputs.
result Finite-sample error bounds for learning algorithms of LTI systems.
DeepBayes uses neural networks to efficiently estimate parameters in complex dynamical models.
problem Estimating parameters in stochastic, nonlinear dynamical models is challenging.
method DeepBayes leverages deep recurrent neural networks to learn an estimator that minimizes mean-squared error.
result DeepBayes achieves asymptotically equivalent performance to Bayesian estimation methods.
Deep neural network improves cancer mutation calls with confidence.
problem Improving accuracy and confidence in somatic variant calls from cancer sequencing.
method Deep Bayesian Recurrent Neural Network (RNN) with flexible priors.
result Enhanced confidence in mutation calls without performance degradation.
Bayesian RNNs improve safety in RL tasks.
problem Lack of explainability and safety in RL algorithms.
method Fully Bayesian Recurrent Neural Networks (RNNs) with uncertainty estimates.
result Significantly outperforms existing methods in collision avoidance tasks.
Bayesian Neural Networks detect gravitational wave events with high accuracy and real-time potential.
problem Detecting and identifying the full duration of compact binary coalescence events in gravitational wave data.
method Integrating Bayesian approach into a CLDNN classifier that combines CNN and LSTM for event detection and uncertainty estimation.
result Successfully detected all seven BBH events in LIGO Livingston O2 data with high accuracy.
Unified Bayesian framework for missing data imputation and prediction in clinical time series.
problem High prevalence of missing values in clinical time series data.
method Unified Bayesian recurrent framework for imputation and prediction.
result Strong performance gains over state-of-the-art methods on mortality prediction tasks.
Bayesian method reconstructs neural network memories from connectivity.
problem Reconstructing memories from neural network connectivity.
method Bayesian inference using statistical physics principles.
result Algorithm successfully reconstructs stored patterns from synaptic connectivity.
We focus on the problem of streaming recommender system and explore novel collaborative filtering algorithms to handle the data dynamicity and complexity in a streaming manner. Although deep neural networks have demonstrated the effectiveness of recommendation tasks, it is lack of explorations on integrating probabilis…
JANA trains networks to approximate Bayesian models efficiently.
problem Intractable likelihood functions and posterior densities in Bayesian models.
method End-to-end training of three networks: summary, posterior, and likelihood networks.
result JANA provides accurate amortized marginal likelihood and posterior predictive estimation.
Adaptive prediction timing improves healthcare outcomes by predicting patient events at the right frequency.
problem Inconsistent prediction granularity in healthcare models.
method Introduces a novel approach using Bayesian recurrent models and a new aggregation method to adapt prediction frequency based on uncertainty.
result Adaptive prediction timing leads to improved predictive performance, especially in the critical first 12 hours of patient stay.
Study improves stock index prediction accuracy using TPE-GRNN models.
problem Enhancing prediction of stock index prices in volatile markets.
method Gated recurrent neural networks (LSTM, GRU) combined with TPE Bayesian optimization.
result TPE-LSTM method shows lowest MAPE (best accuracy) for NIFTY 50 index prediction.
Bayesian optimization reduces RNN architecture search time.
problem Optimizing RNN architectures for time series prediction.
method Bayesian Optimization with a training-free performance metric.
result BO designs well-performing RNN architectures with reduced optimization time.
Improved neural network predicts tropical storm trajectories and Bayesian intervals.
problem Accurately predicting the trajectories of tropical storms to prevent damage.
method Developed an improved RNN model with dropout to predict Bayesian intervals.
result Neural network dropout values significantly affect prediction accuracy and intervals.
PAC-Bayesian bounds for stochastic LTI systems derived.
problem Error bounds for stochastic LTI systems.
method PAC-Bayesian theory applied to autonomous stochastic LTI models.
result Error bounds for stochastic LTI systems derived.
Stochastic gradient Markov chain Monte Carlo (SG-MCMC) has become increasingly popular for simulating posterior samples in large-scale Bayesian modeling. However, existing SG-MCMC schemes are not tailored to any specific probabilistic model, even a simple modification of the underlying dynamical system requires signifi…
Exponentially smoothed RNNs improve industrial forecasting.
problem Complexity and non-stationarity in industrial time series data.
method Exponential smoothed recurrent neural networks (RNNs) for modeling non-linear dynamics.
result Exponentially smoothed RNNs outperform traditional models in multi-step forecasting.
New method identifies latent variables in cognitive models using neural networks.
problem Inference of latent variables in complex cognitive models is limited.
method Recurrent neural networks and simulation-based inference for latent variable sequences.
result Extends neural Bayes estimation to broader classes of cognitive models.
A new approach to learning in brain-like networks using adversarial algorithms.
problem Complex inter-dependencies in brain-like networks not compatible with conditional independence assumptions.
method Adversarial algorithm for learning models of perceptual processing.
result The approach can mimic known neural phenomena and yields testable hypotheses.
Recurrent neural networks (RNNs) are nonlinear dynamical models commonly used in the machine learning and dynamical systems literature to represent complex dynamical or sequential relationships between variables. More recently, as deep learning models have become more common, RNNs have been used to forecast increasingl…
Probabilistic models predict neural network performance across varying hyperparameters.
problem Predicting neural network performance with different hyperparameters.
method Probabilistic models based on random forests and Bayesian recurrent neural networks.
result Models outperform state-of-the-art hyperparameter optimization methods.
We define Recurrent Gaussian Processes (RGP) models, a general family of Bayesian nonparametric models with recurrent GP priors which are able to learn dynamical patterns from sequential data. Similar to Recurrent Neural Networks (RNNs), RGPs can have different formulations for their internal states, distinct inference…
A new framework for efficient sequence maps using Bayesian filtering and covariance.
problem Designing efficient recurrent sequence maps from explicit memory assumptions.
method Design-model framework, exact Bayesian filtering, query-dependent readout, linear-Gaussian instantiation.
result Improved robustness and retrieval performance across various benchmarks.
This paper improves speech synthesis using a DGP with SRU for naturalness.
problem Improving naturalness in synthetic speech.
method Deep Gaussian process with a recurrent architecture using SRU.
result SRU-DGP outperforms other models in naturalness of synthetic speech.
QRPNNs use quaternion-valued recurrent correlation neural networks to solve cross-talk issues.
problem Cross-talk problem in QRCNNs.
method Combining non-local projection learning with QRCNNs.
result QRPNNs exhibit greater storage capacity and noise tolerance.
Interneurons improve learning in neural networks by accelerating convergence.
problem Rapid adaptation to changing input statistics in neural networks.
method Two mathematically tractable recurrent linear neural networks were compared: one with direct recurrent connections and the other with interneurons that mediate recurrent communication.
result The network with interneurons converges more quickly than the network with direct recurrent connections, scaling logarithmically with initialization spectrum.
The performance of deep neural networks crucially depends on good hyperparameter configurations. Bayesian optimization is a powerful framework for optimizing the hyperparameters of DNNs. These methods need sufficient evaluation data to approximate and minimize the validation error function of hyperparameters. However, …
Deep Bayesian neural networks improve somatic variant calling accuracy.
problem Improving accuracy in pinpointing somatic variants from next-gen sequencing data.
method Deep Bayesian Recurrent Neural Networks (RNNs) for somatic variant calling.
result Deep Bayesian RNNs provide more reliable confidence intervals for variant calls.
Paper presents FPGA implementation for efficient recurrent neural networks.
problem Implementing recurrent neural networks on FPGAs for low latency.
method Developed hls4ml framework to implement LSTM and GRU layers.
result Demonstrated effective designs for both small and large models.
We present Ordinary Differential Equation Variational Auto-Encoder (ODE2VAE), a latent second order ODE model for high-dimensional sequential data. Leveraging the advances in deep generative models, ODE2VAE can simultaneously learn the embedding of high dimensional trajectories and infer arbitrarily complex conti…
Scalable verifier for recurrent neural networks using polyhedral abstractions.
problem Certifying the correctness of recurrent neural networks.
method Combining sampling, optimization, and Fermat's theorem for polyhedral abstractions; gradient descent for refinement.
result Successfully verified challenging recurrent models in various domains.
Sophisticated gated recurrent neural network architectures like LSTMs and GRUs have been shown to be highly effective in a myriad of applications. We develop an un-gated unit, the statistical recurrent unit (SRU), that is able to learn long term dependencies in data by only keeping moving averages of statistics. The SR…
Objective: To compare different deep learning architectures for predicting the risk of readmission within 30 days of discharge from the intensive care unit (ICU). The interpretability of attention-based models is leveraged to describe patients-at-risk. Methods: Several deep learning architectures making use of attentio…