The paper bounds generalization errors for deep neural networks with Markov datasets.
problem Bounding generalization errors for deep learning with Markov datasets.
method Developed new symmetrization inequalities for Markov chains, using spectral gap of the infinitesimal generator.
result Derived upper bounds on generalization errors for deep neural networks with Markov datasets.
Deep learning estimates time-varying Markov model parameters.
problem Estimating time-dependent parameters in Markov models.
method Reframes parameter estimation as an optimization problem using maximum likelihood.
result Real solution close to SDE with neural network-derived parameters under specific conditions.
The article proposes a deep learning method to test and infer the Markov property in time series data.
problem Testing and inferring the Markov property in high-dimensional time series data.
method Deep conditional generative learning to estimate conditional density functions and derive a doubly robust test statistic.
result The test controls the type-I error asymptotically and has power approaching one.
Physics-guided model improves deep learning for nonlinear systems.
problem Intractable inference of nonlinear dynamical systems from data.
method Physics-guided Deep Markov Model (PgDMM) using neural networks.
result Improved performance on nonlinear systems with structured latent space.
Deep neural network learns discrete state abstractions for efficient planning.
problem Efficient sequential decision making in large state spaces.
method Information bottleneck method for learning approximate bisimulations using deep neural encoders and action-conditioned HMM.
result Trained method efficiently plans for unseen goals in multi-goal reinforcement learning.
New method improves policies by combining Markov and non-Markov strategies.
problem Improving policies in reinforcement learning.
method Geometric Policy Composition (GPI) using a geometric horizon model (GHM).
result GPI can improve non-Markov policies by composing GHMs of base Markov policies.
Neural Markov models improve time series analysis by balancing deep learning and classical models.
problem Modeling non-stationary time series with high data sparsity.
method Hybrid approach using neural networks to parameterize stochastic matrices, estimating time-inhomogeneous Markov chains.
result Reduction of Chapman-Kolmogorov discrepancy and superior likelihood in financial markets.
Deep GMRFs improve spatial data modeling and prediction.
problem Modeling spatial dependencies in data.
method Established connection between GMRFs and CNNs, allowing for multi-layer architectures.
result Deep GMRFs outperform state-of-the-art models in satellite temperature prediction.
Cyber threat intelligence is one of the emerging areas of focus in information security. Much of the recent work has focused on rule-based methods and detection of network attacks using Intrusion Detection algorithms. In this paper we propose a framework for inspecting and modelling the behavioural aspect of an attacke…
Paper revisits Deep Variational Information Bottleneck and proposes a new optimization approach.
problem Limitations of Deep Variational Information Bottleneck in optimizing mutual information.
method Proposes a new optimization approach by circumventing the limitation of requiring both Markov chains during optimisation.
result Shows how to optimise a lower bound for mutual information, circumventing the limitation of requiring both Markov chains.
A scalable deep GMRF model for general graphs improves predictions and uncertainty estimates.
problem Handling generally structured data on graphs efficiently.
method A new multi-layer structure of Deep GMRFs designed for general graphs, enabling efficient training and close-to-exact Bayesian inference.
result Close-to-exact Bayesian inference for latent field predictions with uncertainty estimates.
Hybrid model improves traffic flow prediction accuracy.
problem Predicting traffic flow with high accuracy in short-term future.
method A hybrid model combining hidden Markov model and LSTM.
result Significant performance gains over conventional methods.
New neural processes use stacked Markov operators to improve flexibility.
problem Improving flexibility in neural processes.
method Stacking neural parameterized Markov transition operators in function space.
result MNPs outperform baseline models on various tasks.
There is an increasing demand for computing the relevant structures, equilibria and long-timescale kinetics of biomolecular processes, such as protein-drug binding, from high-throughput molecular dynamics simulations. Current methods employ transformation of simulated coordinates into structural features, dimension red…
We introduce the Contextual Graph Markov Model, an approach combining ideas from generative models and neural networks for the processing of graph data. It founds on a constructive methodology to build a deep architecture comprising layers of probabilistic models that learn to encode the structured information in an in…
MLDL preserves manifold geometry in vector transformations.
problem Geometric deterioration in neural network transformations.
method Locally isometric smoothness (LIS) and Markov random field (MRF) encoding.
result Enhanced vector transformations into well-behaved metric homeomorphisms.
RDMM integrates RL and deep learning for better trading performance.
problem Improving financial trading performance with complex market dynamics.
method Reinforced Deep Markov Model (RDMM) integrating RL and deep learning.
result RDMM outperforms benchmarks in optimal execution problems.
The paper introduces a method to learn Markov state abstractions for reinforcement learning.
problem Learning Markov state representations in complex environments.
method The paper introduces a novel set of conditions and a training procedure combining inverse model estimation and temporal contrastive learning.
result The approach learns representations that capture the underlying structure of the domain and improve sample efficiency.
Algorithm finds ε-equilibrium policies for multi-agent Markov games with hidden low-rank structure.
problem Designing efficient algorithms for multi-agent Markov games with unknown representation and hidden low-rank structure.
method Model-based and model-free approaches using representation learning to construct an effective representation from data.
result Achieves poly(H,d,A,1/ε) sample complexity for both model-based and model-free approaches. DCDC calculates convergence rates for Markov chains using neural networks.
problem Computing precise convergence rates for Markov chains is hard.
method Developed a neural network-based algorithm (DCDC) to bound convergence rates in Wasserstein distance.
result Demonstrated effective convergence bounds for real-world Markov chains.
Deeptime simplifies learning dynamical models from time series data.
problem Understanding complex systems through dynamical models from time series data.
method Various tools for estimating dynamical models including conventional and kernel/deep learning methods.
result Estimates dynamical models from time series data efficiently and with rich analysis methods.
Deep unfolding accelerates MCMC-based COP solvers.
problem Optimizing combinatorial problems with MCMC and gradient descent.
method Combines MCMC and gradient descent, trains step sizes, uses variance estimation for non-differentiable MCMC.
result Significantly accelerates convergence speed for COPs.
DMSTF models spatio-temporal data with deep Markov priors.
problem Analyzing nonlinear multimodal spatio-temporal dynamics.
method Deep Markov spatio-temporal factorization with stochastic variational inference.
result DMSTF outperforms other methods in predictive performance and clustering.
We introduce LAMP: the Linear Additive Markov Process. Transitions in LAMP may be influenced by states visited in the distant history of the process, but unlike higher-order Markov processes, LAMP retains an efficient parametrization. LAMP also allows the specific dependence on history to be learned efficiently from da…
Deep networks can approximate score functions in high-dimensional graphical models efficiently.
problem Approximation efficiency of score functions by deep neural networks in high-dimensional graphical models like Markov random fields.
method Variational inference denoising algorithms and efficient neural network representation.
result Efficient sample complexity bound for diffusion-based generative modeling when score functions are learned by deep neural networks.
This paper provides a tutorial on Boltzmann Machines and Deep Belief Networks.
problem Understanding and applying Boltzmann Machines and Deep Belief Networks.
method Explains the structures, conditional distributions, Gibbs sampling, training methods, and deep belief networks of RBMs.
result Comprehensive overview of RBMs and DBNs, useful in various fields.
Survey on Evidential Deep Learning for uncertainty estimation in deep neural networks.
problem Uncertainty estimation in deep neural networks with overhead or limited diversity.
method Evidential Deep Learning, parameterizing distributions over distributions.
result Single model and forward pass uncertainty estimation with a unified notation.
Advances in mobile computing technologies have made it possible to monitor and apply data-driven interventions across complex systems in real time. Markov decision processes (MDPs) are the primary model for sequential decision problems with a large or indefinite time horizon. Choosing a representation of the underlying…
Develops RL for optimal market-making in non-Markov processes.
problem Optimal market-making in non-Markov price processes.
method Deep reinforcement learning with Soft Actor-Critic (SAC) algorithm.
result Optimal strategy for market-making in semi-Markov and Hawkes Jump-Diffusion dynamics.
Unified framework DDNs for multi-label classification, improving inference efficiency.
problem Efficient inference for multi-label classification with dependency networks.
method Combining dependency networks and deep learning, proposing novel inference schemes.
result Novel inference schemes outperform basic neural architectures and Markov networks.
JaxSGMC simplifies SG-MCMC for Bayesian deep learning.
problem Uncertainty quantification in deep learning models.
method Modular stochastic gradient MCMC in JAX.
result Facilitates trustworthy neural network predictions.
We propose a deep generative Markov State Model (DeepGenMSM) learning framework for inference of metastable dynamical systems and prediction of trajectories. After unsupervised training on time series data, the model contains (i) a probabilistic encoder that maps from high-dimensional configuration space to a small-siz…
Integrating deep learning with latent state space models has the potential to yield temporal models that are powerful, yet tractable and interpretable. Unfortunately, current models are not designed to handle missing data or multiple data modalities, which are both prevalent in real-world data. In this work, we introdu…
Deep reinforcement learning finds optimal learning policies for adaptive systems.
problem Finding individualized learning plans for learners with unknown latent traits.
method Formulated as a Markov decision process, applied deep Q-learning with a transition model estimator.
result The algorithm efficiently discovers optimal learning policies with small data sets.
Deep neural networks can solve optimal stopping problems without dimensionality issues.
problem Optimal stopping problems in high-dimensional state spaces.
method Established a general framework for deep ReLU neural networks to approximate value functions and continuation values.
result Deep neural networks can approximate value functions and continuation values with error at most ε of size κd^q ε^(-r).
We use deep reinforcement learning to optimize experimental designs efficiently.
problem Optimizing sequential experimental designs with limited exploration and black-box models.
method Reduced the optimal design problem to an MDP and solved it with deep reinforcement learning.
result Our approach achieves state-of-the-art performance on both continuous and discrete design spaces.
Efficiently trains deep Gaussian processes with sparse approximations.
problem High computational complexity in training and inference for DGP models.
method Tensor Markov Gaussian Processes (TMGP) and hierarchical expansion to create DTMGP model.
result DTMGP model achieves superior computational efficiency compared to existing DGP models.
A new algorithm for deep Q-learning with robustness to state transition uncertainty.
problem Model uncertainty in state transitions for non-tabular, continuous state spaces.
method Distributionally robust approach using worst-case transition ball and dualized Bellman operator with Sinkhorn distance.
result Optimal policy found through solving non-linear Bellman equation with neural network parameterization.
Deep nets learn structured densities without dimensionality issues.
problem Learning structured densities in high dimensions.
method Simple L2-minimizing loss for neural networks. result Dimension-independent convergence rates for neural networks.
New algorithm speeds up MCMC for deep learning models.
problem Large biases in SGMCMC for big data.
method Adaptive replica exchange SGMCMC (reSGMCMC).
result Achieves state-of-the-art results on various datasets.
Bayesian RL tackles uncertainty with deep generative models and sequential samplers.
problem Optimal decision-making in uncertain environments with limited data.
method Bayesian approach using deep generative models and prequential scoring rule for posterior inference. Policy learning via expected Thompson sampling.
result Improves policy learning in high-dimensional parameter spaces and continuous action spaces.
Complex high dimensional stochastic dynamic systems arise in many applications in the natural sciences and especially biology. However, while these systems are difficult to describe analytically, "snapshot" measurements that sample the output of the system are often available. In order to model the dynamics of such sys…
Proposes a new model for joint probability distributions in computer vision.
problem Limitation of existing models in meeting diverse downstream tasks.
method Uses parametric conditional probability distributions for each group of variables conditioned on the rest.
result Models can be used for any downstream task without task-specific design.
Unified approach to denoising Markov models for efficient sampling.
problem Designing efficient sampling algorithms for complex distributions.
method Mathematical foundation using measure transport and nonequilibrium statistical mechanics.
result Unified variational objective and backward generator construction.
Training energy-based probabilistic models is confronted with apparently intractable sums, whose Monte Carlo estimation requires sampling from the estimated probability distribution in the inner loop of training. This can be approximately achieved by Markov chain Monte Carlo methods, but may still face a formidable obs…
Langevin autoencoders improve deep latent variable models with efficient posterior sampling.
problem Efficient posterior sampling in deep latent variable models using MCMC.
method Amortized Langevin dynamics (ALD) replaces datapoint-wise sampling with encoder updates.
result ALD is valid as an MCMC algorithm with the target posterior as a stationary distribution.
Deep nets solve MDPs without high dimensions.
problem Solving Bellman equations for MDPs in high dimensions.
method Deep neural networks with ReLU activation approximating payoff and transition functions.
result Deep nets can approximate Q-functions in polynomially bounded parameters. VAE improves MCMC efficiency by generating diverse prior proposals.
problem Inefficient MCMC methods in Bayesian inverse problems, especially subsurface flow modeling.
method Uses Variational Autoencoder (VAE) to generate broader-spectrum prior proposals.
result VAE achieves comparable accuracy to Karhunen-Loève Expansion (KLE) and outperforms it when correlation length is unknown.