The medical field stands to see significant benefits from the recent advances in deep learning. Knowing the uncertainty in the decision made by any machine learning algorithm is of utmost importance for medical practitioners. This study demonstrates the utility of using Bayesian LSTMs for classification of medical time…
Bayesian LSTM model improves VaR and ES forecasting accuracy.
problem Joint forecasting of Value at Risk (VaR) and Expected Shortfall (ES).
method Hybrid model combining LSTM for time series dynamics and Asymmetric Laplace quasi-likelihood for joint likelihood.
result The LSTM-AL model outperforms existing models in VaR and ES forecasting accuracy.
Paper proposes CNN-LSTM model for multi-speaker speech separation.
problem Multi-speaker source separation using deep learning.
method Parallel CNN-LSTM architecture with Bayesian hyperparameter optimization.
result Parallel CNN-LSTM model outperforms LSTM-only and CNN-only models.
Bayesian model predicts sequences better than LSTMs by identifying underlying rules.
problem Current RNNs struggle to generalize from limited training data and identify underlying rules in sequences.
method Bayesian model that learns underlying concepts from sequences and generalizes to new data.
result Bayesian model predicts sequences better than traditional LSTMs.
New LSTM scheme incorporates prior knowledge and measurement uncertainties.
problem Overfitting and insufficient data for accurate time-dependent solutions.
method Sparse Bayesian training algorithm for automatic connection determination.
result Less prone to overfitting, smaller data set required for satisfying accuracy.
Bayesian method sparsifies gated RNNs, improving speed and interpretability.
problem Sparsifying neural networks to reduce complexity and improve performance.
method Bayesian approach to sparsify weights, neurons, and gates in LSTM architectures.
result Sparsified gated RNNs speed up forward pass and improve compression.
Enhanced volatility model using LSTM and realized volatility.
problem Volatility modeling in financial markets.
method Combining deep learning (LSTM) and realized volatility measures in a Bayesian framework.
result Superior predictive performance compared to benchmark models.
Proposes ECLSTM for more accurate RUL estimation from time series data.
problem Predicting Remaining Useful Life (RUL) from multivariate time series data.
method Embedded Convolutional LSTM (ECLSTM) with automated hyperparameter optimization.
result ECLSTM outperforms state-of-the-art approaches on benchmark data sets.
Bayesian learning from variable-length sequences using Gaussian processes with signature covariances.
problem Learning from sequences of varying lengths and complex sequential structures.
method Gaussian processes with signature kernels, sparse variational approach, combining with LSTM/GRU models.
result Effective learning from sequences of different lengths and complex structures.
Bayesian LSTM for outlier detection reduces overfitting.
problem Overfitting and lack of uncertainty in LSTM networks.
method Approximate Bayesian estimation with Ensemble Kalman Filter and maximum likelihood.
result The method reduces overfitting and provides uncertainty estimates.
Sophisticated gated recurrent neural network architectures like LSTMs and GRUs have been shown to be highly effective in a myriad of applications. We develop an un-gated unit, the statistical recurrent unit (SRU), that is able to learn long term dependencies in data by only keeping moving averages of statistics. The SR…
SVOCD detects changepoints in complex systems efficiently.
problem Detecting changepoints in complex systems efficiently.
method Stein variational online changepoint detection integrating SVN method and BOCPD.
result Successfully applied to Hawkes processes and LSTM neural networks.
Many applications in speech, robotics, finance, and biology deal with sequential data, where ordering matters and recurrent structures are common. However, this structure cannot be easily captured by standard kernel functions. To model such structure, we propose expressive closed-form kernel functions for Gaussian proc…
Study improves stock index prediction accuracy using TPE-GRNN models.
problem Enhancing prediction of stock index prices in volatile markets.
method Gated recurrent neural networks (LSTM, GRU) combined with TPE Bayesian optimization.
result TPE-LSTM method shows lowest MAPE (best accuracy) for NIFTY 50 index prediction.
Bayesian deep learning predicts satellite collisions.
problem Space debris poses planetary risk.
method Bayesian deep learning with LSTM networks.
result Predicts conjunction event evolution with uncertainties.
RegPred Net forecasts foreign exchange rates with improved accuracy and interpretability.
problem Multi-step forecasting of Foreign Exchange (FX) rates.
method Bayesian optimization for hyperparameter tuning of a multi-layered regression network.
result RegPred Net significantly outperforms other models in terms of RMSE and correlation metrics.
Bayesian Layers adds uncertainty to neural networks, enabling faster experimentation and scalability.
problem Enabling neural networks to quantify uncertainty in predictions.
method Drop-in replacements for common layers, capturing uncertainty over weights, activations, etc.
result Bayesian Layers can fit large models like 5-billion parameter Bayesian Transformers.
Bayesian Transformer improves probabilistic load forecasting with calibrated uncertainty estimates.
problem Overconfident point predictions from deep learning models fail under extreme weather distributional shifts.
method Integrates three uncertainty mechanisms: MC Dropout, variational layers, and stochastic attention.
result Achieves state-of-the-art performance with CRPS of 0.0289 and 90% PICP across various horizons.
Deep learning models predict mutual funds' performance better than traditional methods.
problem Predicting mutual funds' performance accurately.
method Deep learning models (LSTM, GRUs) trained with Bayesian optimization and ensemble methods.
result Ensemble method of LSTM and GRUs achieves the highest accuracy in forecasting mutual funds' Sharpe ratios.
A-LSTM improves emotion recognition by better modeling time dependencies.
problem Conventional LSTM's time dependency modeling is limited.
method Proposed A-LSTM for better temporal context modeling in RNNs.
result A-LSTM outperforms conventional LSTM by 5.5% in emotion recognition.
Stacked LSTM improves weather forecasting accuracy by incorporating spatial information.
problem Improving temperature prediction accuracy in weather forecasting.
method 2-layer spatio-temporal stacked LSTM model with independent LSTM models per location in the first layer and combined hidden states in the second layer.
result The stacked LSTM model outperforms single LSTM models in most cases by utilizing spatial information.
Bayesian model predicts oncology demand trends with high accuracy.
problem Accurate forecasting of oncology demand for resource planning.
method Boosting-based Bayesian conjugate models for Poisson process.
result Model outperforms other methods in trend detection accuracy.
eSPA breaches overfitting barriers in ML with 10^-12 cost.
problem Overfitting in machine learning with small data.
method eSPA, entropy-optimal Scalable Probabilistic Approximations algorithm.
result 30-fold boost in classification performance with eSPA.
This paper compares LSTMs and attention mechanisms for financial time series forecasting.
problem Improving financial time series forecasting accuracy.
method Implemented an LSTM with attention mechanism and compared it to a standard LSTM.
result An LSTM with attention can outperform standalone LSTMs, but further investigation is needed.
Novel unsupervised LSTM method for video frame prediction.
problem Video frame prediction with limited labeled data.
method Inception-based LSTM architecture.
result Inception-based LSTM outperforms convolutional LSTM and has better performance than Inception v2 LSTM.
Recurrent neural networks (RNNs) stand at the forefront of many recent developments in deep learning. Yet a major difficulty with these models is their tendency to overfit, with dropout shown to fail when applied to recurrent layers. Recent results at the intersection of Bayesian modelling and deep learning offer a Bay…
LSTM-FCN and ALSTM-FCN improve time series classification performance.
problem Improving time series classification performance.
method Ablation tests on LSTM-FCN and ALSTM-FCN, comparing z-normalizing techniques, dimension shuffle impact, and GRU replacement.
result LSTM and FCN blocks perform better together, and z-normalizing the whole dataset is more effective.
MC-LSTM extends LSTM to conserve mass in neural networks.
problem Conservation laws in real-world systems.
method Extending LSTM's inductive bias to conserve mass.
result MC-LSTM sets new state-of-the-art for predicting peak flows.
Grow and prune LSTM to reduce parameters and latency.
problem Model redundancy, increased run-time delay, and overfitting in deep LSTM models.
method Hidden-layer LSTM (H-LSTM) with grow-and-prune (GP) training.
result Significant reduction in parameters, latency, and improvement in accuracy.
Proposes a model combining difference-attention and error-correction LSTMs for improved time series prediction.
problem Improving accuracy in time series prediction.
method Combines difference-attention LSTM and error-correction LSTM in a cascade approach.
result Improves prediction accuracy in time series.
Simplified LSTM models improve sentiment analysis on Twitter debate data.
problem Performing sentiment analysis on long sequence data from Twitter debates.
method Developed six parameter-reduced LSTM models (slim LSTM) for faster training and reduced computational cost.
result Slim LSTM models outperform standard LSTM model in sentiment analysis of GOP Debate Twitter dataset.
Enhanced LSTM model learns complex temporal dependencies.
problem Modeling long-term dependencies in sequential data.
method Gamma-LSTM with hierarchical memory units and gates.
result Gamma-LSTM outperforms regular and stacked LSTMs in sequence prediction.
CD interprets LSTM predictions by identifying word interactions.
problem LSTMs are black boxes; understanding their internal workings is difficult.
method Contextual decomposition (CD) to interpret LSTM predictions.
result CD reliably identifies word interactions and sentiment combinations.
A forget-gate-only LSTM outperforms standard LSTM on benchmark datasets.
problem The necessity of all gates in LSTM networks.
method A forget-gate-only LSTM with chrono-initialized biases.
result The forget-gate-only LSTM outperforms standard LSTM on MNIST and pMNIST datasets.
Study benchmarks LSTM units across PyTorch, TensorFlow, Lasagne, and Keras.
problem Comparing LSTM implementations in different deep learning frameworks.
method Benchmarked LSTM units in PyTorch, TensorFlow, Lasagne, and Keras across two speech recognition scenarios.
result Performance differences between LSTM implementations in various frameworks.
Variational Bi-LSTM improves Bi-LSTM performance by jointly optimizing forward and backward paths.
problem Improving performance of Bi-LSTMs by capturing richer data representation.
method Proposes a Variational Bi-LSTM architecture that connects forward and backward paths during training.
result Shows state-of-the-art performance on various benchmarks.
ARIMA-LSTM hybrid model predicts stock price correlation coefficients.
problem Predicting future stock price correlation coefficients for portfolio optimization.
method ARIMA-LSTM hybrid model combining ARIMA for linear tendencies and LSTM for non-linear temporal dependencies.
result ARIMA-LSTM model outperforms other models in predicting stock price correlation coefficients.
State Space LSTM models improve interpretability of LSTM with efficient SMC inference.
problem Combining interpretability of state space models with LSTM's performance.
method Introducing State Space LSTM models and an efficient SMC sampler for direct posterior sampling.
result Efficient SMC inference confirms superior and stable performance on various domains.
Paper predicts house prices in major Chinese cities using LSTM networks.
problem Predicting house prices in major Chinese cities.
method Used LSTM networks compared to ARIMA for prediction accuracy.
result LSTM networks showed superior accuracy in predicting house prices.
Solves exploding and vanishing gradient problem in LSTMs.
problem Exploding and vanishing gradient problem in LSTM optimization.
method Introduces a simple stochastic algorithm (h-detach) to prevent suppression of gradient components through the cell state path in LSTM.
result Significant improvements in convergence speed, robustness, and generalization over vanilla LSTM training.
Explains RNN and LSTM fundamentals, derives formulas, and addresses training issues.
problem Lack of detailed formulas and unrolling techniques in LSTM and RNN literature.
method Derives canonical RNN and LSTM formulas from differential equations, proposes unrolling technique, addresses training difficulties.
result Provides a comprehensive understanding of RNN and LSTM, including detailed formulas and unrolling techniques.
DA-LSTM adapts LSTM depth to non-uniform data, improving efficiency.
problem Non-uniform information distribution in sequential data cannot be accurately modeled by traditional LSTM.
method Developed DA-LSTM architecture that dynamically adjusts LSTM depth based on information distribution.
result DA-LSTM reduces computation resource usage and convergence time by 41.78% and 46.01% respectively.
BiLSTM models outperform LSTM and ARIMA in financial time series forecasting.
problem Comparing forecasting accuracy of ARIMA, LSTM, and BiLSTM models.
method Behavioral analysis and comparison of BiLSTM and LSTM models.
result BiLSTM models provide better predictions than ARIMA and LSTM models.
Enhanced LSTM with multiple kernels and attention improves video action recognition.
problem Improving motion understanding in video analysis.
method Proposed a Network-in-LSTM approach with multiple convolutional kernels and layers, and an attention-based mechanism.
result Improves accuracy in supervised classification on UCF-101 and Sports-1M datasets.
EA-LSTM improves LSTM for time series prediction by evolving attention.
problem LSTMs struggle with assigning varying attention to sub-windows in time series data.
method Evolutionary attention-based LSTM with competitive random search.
result EA-LSTM achieves competitive performance in multivariate time series prediction.
LSTMs show surprising few-shot learning ability, improving on MAML.
problem Few-shot learning with limited data.
method Revisited LSTM approach with Outer Product LSTM (OP-LSTM).
result OP-LSTM outperforms MAML on simple few-shot tasks, but not complex ones.
Proposes an interpretable LSTM for time series with exogenous variables.
problem Lack of variable importance characterization in recurrent neural networks.
method Develops a multi-variable LSTM with tensorized hidden states for learning variable-specific representations.
result Variable attention in real datasets is highly aligned with statistical causality.
Deep neural network predicts blood glucose levels for diabetes patients.
problem Diabetes patients need to anticipate hyperglycaemia and hypoglycaemia.
method Used a sequential model with LSTM and Bi-LSTM layers to predict blood glucose levels.
result The proposed network outperforms baseline methods in predicting blood glucose levels.