This paper reviews methods for interpreting deep learning models with sequential data.
problem Limited interpretability of deep learning models in sequential data domains.
method Reviews and compares techniques for sequential interpretability.
result Current techniques have limitations and future research is needed.
Paper tackles uncertainty prediction for deep sequential regression.
problem Challenges in generating accurate uncertainty estimates for deep recurrent networks.
method Flexible method that generates symmetric and asymmetric uncertainty estimates without stationarity assumptions.
result Outperforms competitive baselines on both drift and non-drift scenarios.
Unified deep sequential and state-space models for robust option pricing with uncertainty.
problem Combining robustness to noise and uncertainty measurement in option pricing models.
method Unscattered reservoir smoother (URS) integrating deep sequential and state-space models.
result URS achieves competitive forecasting accuracy and uncertainty measurement in noisy datasets.
Sparse Meta Networks adapt deep neural networks incrementally for fast learning.
problem Training deep neural networks is slow and impractical for complex, changing environments.
method Sparse Meta Networks use a memory layer to learn online sequential adaptation, accumulating fast-weights incrementally.
result Sparse Meta Networks achieve strong performance in various sequential adaptation scenarios.
Detects data drift in deep learning models using neural embeddings.
problem Detecting changes in data distribution in deep learning models.
method Formulates drift detection in a sequential decision framework and introduces a loss function to balance false alarms and quick detection.
result Demonstrates improved ability to balance false alarms and quick detection in change detection.
We build on auto-encoding sequential Monte Carlo (AESMC): a method for model and proposal learning based on maximizing the lower bound to the log marginal likelihood in a broad family of structured probabilistic models. Our approach relies on the efficiency of sequential Monte Carlo (SMC) for performing inference in st…
This paper develops a novel deep recurrent neural network for sequential signal reconstruction.
problem Sequential signal reconstruction from low-dimensional measurements.
method Unfolding a reweighted ℓ1-ℓ1 minimization algorithm to design a deep recurrent neural network. result The proposed reweighted-RNN significantly outperforms existing RNN models in sequential frame reconstruction.
New insights into continual learning for deep models, showing convergence issues but local linear solutions.
problem Challenges in continual learning for homogeneous deep models.
method Sequential projections onto task margin sets, leveraging nonconvex projection theory.
result Local linear convergence under certain conditions for homogeneous deep networks.
Deep learning improves credit risk assessment without new data.
problem Improving credit risk assessment in banking without new data.
method Sequential deep learning using temporal convolutional networks.
result Sequential deep learning outperformed tree-based models in credit risk assessment.
The paper provides a theoretical justification for using stable SSM blocks in deep sequential models.
problem Developing generalization bounds for deep sequential models with varying sequence lengths.
method Using Rademacher contraction and stability constraints, the paper derives a PAC bound that is independent of sequence length.
result The derived PAC bound decreases as the stability of SSM blocks increases, providing theoretical justification for their use.
This review assesses deep-learning methods for complex sequential data.
problem Lack of robustness and transparency in deep-learning frameworks for irregular sequential data.
method Systematic literature review of existing algorithms.
result Recurrent neural networks dominate in performance evaluation of deep-learning frameworks.
Deep model learns protein interfaces from high-order interactions.
problem Predicting protein interfaces from amino acid pairs.
method Graph neural networks and convolutional neural networks for 2D dense predictions.
result Our method consistently improves interface prediction performance.
We use deep reinforcement learning to optimize experimental designs efficiently.
problem Optimizing sequential experimental designs with limited exploration and black-box models.
method Reduced the optimal design problem to an MDP and solved it with deep reinforcement learning.
result Our approach achieves state-of-the-art performance on both continuous and discrete design spaces.
Deep learning complements OR/MS for decision-making under uncertainty.
problem Sequential decision-making in uncertain environments.
method Integration of deep learning and OR/MS frameworks.
result Deep learning enhances adaptability and scalability in decision systems.
Deep learning predicts customer churn in retail.
problem Accurately predicting which customers are likely to stop purchasing.
method Survival model parameters learned by recurrent neural networks.
result Individual level survival models for purchasing behavior.
G-Net uses deep learning for complex counterfactual outcome prediction.
problem Estimating counterfactual outcomes under dynamic treatment strategies.
method G-Net is a sequential deep learning framework for G-computation.
result G-Net can handle complex temporal data and provide accurate treatment effects.
Bayesian RL tackles uncertainty with deep generative models and sequential samplers.
problem Optimal decision-making in uncertain environments with limited data.
method Bayesian approach using deep generative models and prequential scoring rule for posterior inference. Policy learning via expected Thompson sampling.
result Improves policy learning in high-dimensional parameter spaces and continuous action spaces.
DeepICMGP surrogate models multiple outputs efficiently.
problem Challenges in modeling dependencies between multiple outputs using traditional multi-output GPs.
method Introduces hierarchical coregionalization structures across layers in DGPs.
result Demonstrates competitive performance and active learning strategies.
A new method for estimating uncertainty in deep neural networks.
problem Challenges in uncertainty estimation in deep neural networks, especially with increased complexity.
method Decompose tasks into representation learning and state space model for uncertainty estimation.
result The proposed method can estimate predictive distributions on top of existing neural networks.
Twin-to-twin transfusion syndrome treatment requires fetoscopic laser photocoagulation of placental vascular anastomoses to regulate blood flow to both fetuses. Limited field-of-view (FoV) and low visual quality during fetoscopy make it challenging to identify all vascular connections. Mosaicking can align multiple ove…
The usage of deep generative models for image compression has led to impressive performance gains over classical codecs while neural video compression is still in its infancy. Here, we propose an end-to-end, deep generative modeling approach to compress temporal sequences with a focus on video. Our approach builds upon…
Can machines trace human knowledge like humans? Knowledge tracing (KT) is a fundamental task in a wide range of applications in education, such as massive open online courses (MOOCs), intelligent tutoring systems, educational games, and learning management systems. It models dynamics in a student's knowledge states in …
In order to build efficient deep recurrent neural architectures, it is essential to analyze the complexityof long distance dependencies (LDDs) of the dataset being modeled. In this paper, we presentdetailed analysis of the dependency decay curve exhibited by various datasets. The datasets sampledfrom a similar process …
Bayesian framework for sequential learning tasks with low-rank approximations.
problem Balancing knowledge retention and adaptability in sequential neural networks.
method Bayesian framework with diagonal plus low-rank approximations of the precision matrix.
result Unlocking capabilities to encode task relationships and incorporate prior knowledge from later tasks.
Developments in deep generative models have allowed for tractable learning of high-dimensional data distributions. While the employed learning procedures typically assume that training data is drawn i.i.d. from the distribution of interest, it may be desirable to model distinct distributions which are observed sequenti…
Volatility is a quantity of measurement for the price movements of stocks or options which indicates the uncertainty within financial markets. As an indicator of the level of risk or the degree of variation, volatility is important to analyse the financial market, and it is taken into consideration in various decision-…
New algorithm reduces overfitting in neural networks.
problem Overfitting in neural networks.
method Integrates SMC with SGHMC for mini-batch sampling.
result SMCSGHMC outperforms SGD and deep ensembles.
Combines deep state space models with diffusion models for better forecasting and capturing latent dynamics
problem Forecasting and capturing latent dynamics in time series
method DDSSM: Diffusion-driven state space model
result Empirically outperforms state-of-the-art deep SSM
AI learns to learn sequentially without forgetting.
problem Preventing catastrophic forgetting in machine learning models.
method Meta-learning a neuromodulatory activation-gating function to control selective activation in deep neural networks.
result State-of-the-art continual learning performance with 600 classes (9,000 updates).
Context-aware recommender systems (CARSs) apply sensing and analysis of user context in order to provide personalized services. Adding context to a recommendation model is challenging, since the addition of context may increases both the dimensionality and sparsity of the model. Recent research has shown that modeling …
Prequential posteriors tackle data assimilation for deep generative forecasting models.
problem Challenges in assimilating data into deep generative forecasting models due to intractable likelihood functions.
method Introduces prequential posteriors based on a predictive-sequential loss function, proving consistency under mild conditions, and using parallelizable SMC samplers for scalable inference.
result Prequential posteriors concentrate around parameters with optimal predictive performance, validating method on synthetic and real-world datasets.
A new stopping rule based on E-values helps efficiently use sampling in Bayesian Deep Ensembles.
problem How long should sampling continue in Bayesian Deep Ensembles to yield significant improvements?
method Formulated as a sequential anytime-valid hypothesis test, using E-values to decide when to stop sampling.
result Only a fraction of the full-chain budget is often required for significant improvements.
Constructs classifiers for neural networks with specific data configurations.
problem Finding global minima of deep ReLU neural networks on sequentially separable data.
method Explicitly constructs zero loss neural network classifiers using cumulative parameters and truncation maps.
result Global minimizers can be described with a limited number of parameters based on the data structure.
Adapts MBDOE for real-time parameter estimation in complex systems.
problem Costly posterior inference and design optimization in nonlinear systems.
method Combines DAD with differentiable mechanistic models for real-time parameter estimation.
result Demonstrated on four systems, including a DC motor.
Deep dynamic generative models are developed to learn sequential dependencies in time-series data. The multi-layered model is designed by constructing a hierarchy of temporal sigmoid belief networks (TSBNs), defined as a sequential stack of sigmoid belief networks (SBNs). Each SBN has a contextual hidden state, inherit…
This thesis investigates unsupervised time series representation learning for sequence prediction problems, i.e. generating nice-looking input samples given a previous history, for high dimensional input sequences by decoupling the static input representation from the recurrent sequence representation. We introduce thr…
Develops Bayesian filtering for online learning and related problems.
problem Sequential machine learning challenges, especially non-stationarity, model misspecification, and high dimensionality.
method Modular adaptive framework, provably robust filter, and sequential parameter updates.
result Improved performance in dynamic, high-dimensional, and misspecified models.
New techniques extend certified unlearning to deep neural networks.
problem Applying certified unlearning to deep neural networks (DNNs) is challenging due to their nonconvex nature.
method Developed simple techniques and an efficient computation method for nonconvex objectives, considering nonconvergence training and sequential unlearning.
result Demonstrated the efficacy of the method on real-world datasets, showing advantages of certified unlearning in DNNs.
Deep RL optimizes goal-based investing strategies.
problem Optimizing investment strategies for achieving financial goals.
method Novel deep reinforcement learning approach for goal-based investing.
result Superior performance compared to benchmarks.
Continual lifelong learning is essential to many applications. In this paper, we propose a simple but effective approach to continual deep learning. Our approach leverages the principles of deep model compression, critical weights selection, and progressive networks expansion. By enforcing their integration in an itera…
Bayesian approach helps update deep models without forgetting past data.
problem Updating deep models sequentially without forgetting past data.
method Bayesian inference for continual learning.
result Bayesian approach enables updating model beliefs with new data.
Many recent advances in large scale probabilistic inference rely on variational methods. The success of variational approaches depends on (i) formulating a flexible parametric family of distributions, and (ii) optimizing the parameters to find the member of this family that most closely approximates the exact posterior…
Generative adversarial network for probabilistic forecasting of random systems.
problem Forecasting random dynamical systems without distributional assumptions.
method Recurrent neural network and generative adversarial network (GAN) with regularization based on maximum mean discrepancy (MMD).
result The proposed model successfully forecasts complex stochastic processes with multiple-step predictions.
Dynamic SBI improves SBI efficiency without rounds, reducing simulation and training costs.
problem Efficiently perform complex scientific inference with high-dimensional data.
method Adaptive dataset transformation, parallel simulation and training.
result Significant improvements in simulation and training efficiency.
This paper applies deep learning to ordinal regression, modeling it as a binary search.
problem Ordinal regression with deep learning models.
method Formulated ordinal regression as a binary search problem, using recurrent neural networks.
result Deep learning model shows comparable or better predictive power compared to traditional methods.
Proposes DSW for unbiased ITE estimation with dynamic confounders.
problem Estimating ITE from dynamic observational data with time-varying confounders.
method Deep Sequential Weighting (DSW) infers hidden confounders using current treatment assignments and historical information.
result DSW generates unbiased and accurate treatment effects.
Learning and adapting to new distributions or learning new tasks sequentially without forgetting the previously learned knowledge is a challenging phenomenon in continual learning models. Most of the conventional deep learning models are not capable of learning new tasks sequentially in one model without forgetting the…
Deep Bayesian models estimate causal effects for dynamic treatment regimes over long follow-up times.
problem Challenges in causal effect estimation for dynamic treatment regimes with long follow-up times.
method Combining outcome regression models with deep Bayesian models for high-dimensional features.
result Stable and accurate dynamic causal effect estimation from observational data, especially with long-term follow-up.