New analysis explains pathology of deep Gaussian processes.
problem Pathology of deep Gaussian processes reduces learning capacities with increased layers.
method Study nonlinear dynamic systems corresponding to DGPs, derive recurrence relations.
result Provide tighter bounds and rate of convergence for dynamic systems.
New DRGP models improve prediction accuracy for sequential data.
problem Modeling sequential data for applications like autonomous driving.
method Introduces Deep recurrent Gaussian process (DRGP) models based on Sparse Spectrum Gaussian process (SSGP) and variational Sparse Spectrum Gaussian process (VSSGP).
result Improves prediction accuracy compared to current state of the art methods.
New DRGP models improve prediction accuracy for sequential data.
problem Modeling sequential data for applications like autonomous driving.
method Introduces two new DRGP models based on SSGP and VSSGP.
result Optimal variational distribution exists for the lower bound.
This paper improves speech synthesis using a DGP with SRU for naturalness.
problem Improving naturalness in synthetic speech.
method Deep Gaussian process with a recurrent architecture using SRU.
result SRU-DGP outperforms other models in naturalness of synthetic speech.
Unified theory for deep and recurrent networks using Gaussian processes.
problem Understanding capabilities and limitations of different network architectures.
method Unified derivation of mean-field theory from statistical physics of disordered systems.
result Gaussian processes yield identical Gaussian kernels for both architectures at a single time point or layer.
We define Recurrent Gaussian Processes (RGP) models, a general family of Bayesian nonparametric models with recurrent GP priors which are able to learn dynamical patterns from sequential data. Similar to Recurrent Neural Networks (RNNs), RGPs can have different formulations for their internal states, distinct inference…
GP-LSTM models sequential data with LSTM inductive biases and scalable training.
problem Capturing recurrent structures in sequential data with standard kernel functions.
method Expressive closed-form kernel functions for Gaussian processes, optimized with semi-stochastic gradient procedure.
result State-of-the-art performance on benchmarks and autonomous driving application.
A new memory-efficient sign language translation model reduces weight usage.
problem Memory constraints in real-time sign language translation.
method Variational Bayesian sequence-to-sequence network with Gaussian posterior and Indian Buffet Process prior.
result The proposed model achieves substantial weight compression without compromising performance.
Proposes a new model for time-to-event prediction with uncertainty quantification.
problem Lack of uncertainty in time-to-event predictions using recurrent neural networks.
method Deep Kernel Accelerated Failure Time models combining RNN and sparse Gaussian Process.
result Model delivers better uncertainty estimates compared to related methods.
Personalized healthcare predictions using deep mixed effect model with Gaussian Processes.
problem Making personalized and reliable predictions from time-series data in healthcare.
method A composite model combining a deep neural network for global trends and Gaussian Processes for individual variability.
result Practical advantages over standard time-series deep models, demonstrated on diverse EHR datasets.
A novel online GP model captures long-term memory in sequential data.
problem Capturing long-term memory in sequential data online.
method Integrates HiPPO framework into interdomain GP, leveraging time-varying orthogonal projections as inducing variables.
result OHSVGP outperforms existing online GP methods in predictive performance, long-term memory preservation, and computational efficiency.
Deep Random Splines model neural activity data with better dimensionality.
problem Modeling neural population data with shape constraints.
method Deep neural network transforming Gaussian noise into spline parameters.
result Better dimensionality reduction of neural spiking activity.
Deep neural nets predict aircraft flight paths from weather data.
problem Accurate prediction of aircraft trajectories for aviation efficiency.
method Deep generative convolutional recurrent neural network with tree-based matching.
result Model accurately predicts aircraft flight paths from weather data.
Time series analysis is a key component of machine learning, with applications in various fields.
problem Time series analysis in machine learning
method Basic concepts, classical statistical models, modern machine learning approaches
result Machine learning techniques for time series analysis
Deep learning predicts business process events with high precision.
problem Predicting next events in business processes.
method Recurrent neural networks applied to deep learning.
result Deep learning surpasses state-of-the-art in prediction precision.
Deep learning predicts AF recurrence from MRI images without pre-processing.
problem Estimating AF recurrence from MRI images efficiently and accurately.
method Deep learning approach to predict shape descriptors directly from MRI images.
result Deep learning method produces similar outcomes to state-of-the-art methods.
Recursive KalmanNet combines neural networks with Kalman filters for precise state estimation.
problem State estimation in systems with noisy measurements and non-Gaussian noise.
method Recursive KalmanNet uses a recurrent neural network to estimate states with consistent error covariance, optimizing for Gaussian negative log-likelihood.
result Recursive KalmanNet outperforms conventional Kalman filters and deep learning-based estimators in non-Gaussian noise conditions.
This work introduces a new model for complex stochastic processes.
problem Difficulties in representing non-stationary distributions with conventional models.
method Recurrent Autoregressive Flows using normalizing flows with recurrent neural connections.
result Demonstrates the effectiveness of the proposed model through experiments.
Model captures complex neural dynamics with Gaussian process and RNN.
problem Challenges in extracting latent dynamics from noisy neural data.
method Gaussian process recurrent neural networks (GP-RNN) for nonlinear, non-Markovian dynamics.
result Outperforms state-of-the-art methods in reconstructing latent dynamics.
Deep RNNs reveal interesting insights about short-term memory dynamics.
problem Understanding the short-term memory capabilities of deep RNNs.
method Investigation of state dynamics in successive levels of deep RNNs.
result Higher layers in a hierarchically organized RNN architecture exhibit longer memory spans.
Study Gaussian-process limits of neural networks using tensor programs.
problem Understanding the behavior of neural networks as they approach infinite width.
method Quantitative analysis through tensor programs and Wasserstein distance.
result Explicit finite-width error bounds, showing convergence to Gaussian-process limits.
Recurrent Neural Processes model time series with conditional independence to capture slow variabilities efficiently.
problem Modeling time series data with slow long-term variabilities efficiently.
method Recurrent Neural Processes (RNP) model state space with conditional independence among subsequences.
result RNP state spaces improve predictive performance on real-world time-series data and nonlinear system identification.
Neural model improves volatility estimation and prediction in finance.
problem Improving volatility estimation and prediction in finance.
method Integrates deep neural networks with stochastic volatility models.
result Proposed model outperforms existing methods on average negative log-likelihood.
SIS-RNN improves model flexibility for sequential data.
problem Limited expressive power of existing stochastic RNNs.
method Semi-implicit variational inference for implicit latent representations.
result SIS-RNN outperforms existing methods in various tasks.
TransformerLSR models longitudinal, recurrent, and survival data jointly.
problem Joint modeling of longitudinal measurements, recurrent events, and survival data with dependencies.
method Transformer-based deep learning framework integrating deep temporal point processes and latent structure representation.
result TransformerLSR effectively models all three components simultaneously, demonstrating necessity and effectiveness through simulations and real-world data.
Proposes learnable, probabilistic activation functions for neural networks.
problem Handling uncertainties and overfitting in neural network activation functions.
method Gaussian process prior over stochastic activation functions, variational Bayesian inference.
result Gaussian process neurons can efficiently handle uncertainties and self-estimate confidence.
A new method for identifying Gaussian process state space models.
problem Challenges in learning the model of Gaussian process state space models.
method Structured Gaussian variational posterior distribution over latent states parameterized by a recognition model.
result The method allows for efficient computation of a lower bound on the marginal likelihood and generation of plausible future trajectories.
Deep convolutional Gaussian processes boost image classification accuracy.
problem Image classification with hierarchical feature combinations.
method Deep Gaussian process architecture with convolutional structure.
result Significantly improved image classification performance on MNIST and CIFAR-10 datasets.
Improved model detects sepsis early using clinical data.
problem Early detection of sepsis improves patient outcomes.
method Multi-output Gaussian process RNN with real-time validation.
result Substantially outperforms clinical baselines.
Novel complex RNN improves stability and performance in sequence tasks.
problem Lack of complex representations in deep learning for sequence tasks.
method Developed a complex gated recurrent cell combining complex-valued and norm-preserving state transitions with a gating mechanism.
result Improves stability and convergence properties, performs competitively on various tasks.
End-to-end classifier predicts sepsis from clinical data.
problem Predicting sepsis onset from clinical data with high accuracy.
method Multitask Gaussian process RNN classifier for multivariate time series.
result Outperforms baselines, improving AUC and PRC.
Deep Gaussian Processes with polynomial kernels can collapse rapidly without proper hyperparameter tuning.
problem The collapse of Deep Gaussian Processes with polynomial kernels without careful hyperparameter tuning.
method Analysis using the Berry-Esseen Theorem and observation of prior behavior.
result The prior of a Deep Gaussian Process collapses rapidly towards zero or places negligible mass on low norm functions without proper hyperparameter tuning.
A new deep learning method using differential flows and Gaussian processes.
problem Improving deep learning performance through more flexible transformations.
method Differential flows and Gaussian processes to learn stochastic differential equations.
result Exceeds performance of deep Gaussian processes and neural networks.
A new model uses attention and Gaussian processes for efficient time-series generation.
problem Computational inefficiency and uncertainty underestimation in sequence transduction.
method Attention-based Gaussian process network for real-valued sequence generation.
result The model improves training efficiency and learns factorized generative distribution.
Deep models predict intraday electricity prices accurately.
problem Accurately forecasting intraday electricity prices.
method Two deep time series probabilistic models using ESNs with stochastic disturbances and copulas.
result Deep distributional models provide accurate short-term probabilistic price forecasts.
Deep LSTMs predict business process completion times.
problem Predicting accurate completion times for business processes under SLA constraints.
method Deep Recurrent Neural Networks (LSTMs) to analyze process instance data.
result LSTMs produce accurate predictions of process completion times.
Deep ESN models forecast spatio-temporal data with uncertainty quantification.
problem Complex nonlinear dynamics in spatio-temporal systems are hard to model.
method Deep ensemble ESN models using bootstrap and hierarchical Bayesian frameworks.
result Models produce forecasts and uncertainty measures for spatio-temporal data.
ARU adapts deep forecasting models in streaming data with efficient updates.
problem Adapting deep globally trained models for streaming data efficiently.
method ARU combines deep global models with closed-form linear models for per-series adaptation.
result ARU outperforms local adaptation methods on various datasets.
This paper reviews deep learning's latest progress and applications.
problem Challenges in deep learning models and applications.
method Analysis of existing models and new emerging models.
result Summarizes deep learning's applications in various AI fields.
The paper links deep neural networks to Gaussian processes, showing convergence under certain conditions.
problem Understanding theoretical properties of deep neural networks.
method Study of random, wide, fully connected feedforward networks and Gaussian processes with recursive kernels.
result As network width increases, random functions converge to Gaussian processes under broad conditions.
Develops a deep non-stationary kernel for non-stationary spatio-temporal point processes.
problem Capturing non-stationary dependencies in point process data.
method Approximates the influence kernel with a novel low-rank decomposition and introduces a log-barrier penalty to maintain non-negativity.
result Demonstrates superior performance and computational efficiency compared to state-of-the-art methods.
Bayesian Optimization uses Deep Gaussian Processes for non-stationary functions.
problem Optimizing expensive non-stationary functions with classic Gaussian Processes.
method Deep Gaussian Processes as surrogate models for capturing non-stationarity.
result The proposed algorithm outperforms state-of-the-art methods on analytical and aerospace design problems.
Deep learning automates decoding of sequential codes.
problem Automating the discovery of decoding algorithms in coding theory.
method Study of recurrent neural network (RNN) architectures for decoding well-known sequential codes.
result RNN architectures can decode well-known sequential codes with close to optimal performance.
VSE estimates complex processes from noisy measurements without a model.
problem Estimating states of complex, model-free processes from noisy data.
method Variational state estimation using recurrent neural networks (RNNs) in both learning and inference phases.
result VSE provides a competitive state estimate for a benchmark process (Lorenz system) compared to known and data-driven methods.
Deep neural networks and Gaussian processes are shown to be equivalent through activation functions.
problem Understanding the relationship between neural networks and Gaussian processes.
method Developing an equivalence theory based on activation functions and kernels.
result Models can be seen as neural networks with improved uncertainty prediction or deep Gaussian processes with increased accuracy.
Bayesian deep learning on quantum computers using Gaussian process connections.
problem Training deep neural networks with Bayesian uncertainty estimates on quantum computers.
method Connecting deep neural networks to Gaussian processes, leveraging quantum algorithms for inversion.
result At least polynomial speedup over classical algorithms for Bayesian deep learning on quantum computers.
Deep-HGP uses Bayesian nonparametric approach for complex data regression.
problem Complex data regression with compositional structures.
method Deep Gaussian processes with a squared-exponential kernel, data-driven lengthscale parameters.
result Posterior distribution optimally recovers unknown true regression curve in terms of quadratic loss.
Proposes a neural network solution for queuing dynamics.
problem Modeling service system dynamics with limited parametric assumptions.
method Combines recurrent neural networks and generative adversarial networks.
result Evaluates solution on various datasets, demonstrating effectiveness.