arXiv research

DenseHMM improves HMMs by learning dense representations that enable gradient-based optimization.

New method tunes SMC samplers efficiently without high costs.

New SMC method for pBNNs improves scalability and predictive performance.

We present a probabilistic framework for both (i) determining the initial settings of kernel adaptive filters (KAFs) and (ii) constructing fully-adaptive KAFs whereby in addition to weights and dictionaries, kernel parameters are learnt sequentially. This is achieved by formulating the estimator as a probabilistic mode…

New algorithm detects changes in Markov kernels with unknown post-change kernel.

New method improves parameter estimation in complex stochastic models.

RLMH improves adaptive MCMC by optimizing contrastive divergence reward.

We propose a technique for increasing the efficiency of gradient-based inference and learning in Bayesian networks with multiple layers of continuous latent vari- ables. We show that, in many cases, it is possible to express such models in an auxiliary form, where continuous latent variables are conditionally determini…

Proposes a gradient-based variable selection method for binary classification in RKHS.

Stochastic kernel based dimensionality reduction approaches have become popular in the last decade. The central component of many of these methods is a symmetric kernel that quantifies the vicinity between pairs of data points and a kernel-induced Markov chain on the data. Typically, the Markov chain is fully specified…

Bayesian RL tackles uncertainty with deep generative models and sequential samplers.

Paper extends KPCA using dualization for faster, more robust algorithms.

Generalization in deep neural networks can be analyzed using minimax rates for gradient methods.

This work investigates how gradient-based learning performs with structured data, revealing issues and improvements.

Identifies conditions for multiple invariant probabilities in Markov kernels.

Differentiable structure learning addresses DAGs with multiple global minimizers.

New method trains Markov kernels for efficient sampling.

We introduce a gradient-based learning method to automatically adapt Markov chain Monte Carlo (MCMC) proposal distributions to intractable targets. We define a maximum entropy regularised objective function, referred to as generalised speed measure, which can be robustly optimised over the parameters of the proposal di…

A new method for efficient inference in probabilistic programs with mixed support.

A new HMM model captures kernel dependencies using context-specific Bayesian networks.

We consider a heat kernel approach for the development of stochastic pricing kernels. The kernels are constructed by positive propagators, which are driven by time-inhomogeneous Markov processes. We multiply such a propagator with a positive, time-dependent and decreasing weight function, and integrate the product over…

New method detects changes in high-dimensional Markov processes without explicit likelihood evaluation.

The paper improves Stein importance sampling for Markov chain samples.

Proposes an automatic cyclical scheduling for gradient-based discrete sampling.

New method optimizes hyperparameters in deep learning models efficiently.

We develop algorithms with low regret for learning episodic Markov decision processes based on kernel approximation techniques. The algorithms are based on both the Upper Confidence Bound (UCB) as well as Posterior or Thompson Sampling (PSRL) philosophies, and work in the general setting of continuous state and action …

New concentration inequality for U-statistics of Markov chains.

Paper improves generalization bounds for multi-kernel learning with mixed datasets.

This paper develops tools for nonreversible MCMC with convergence guarantees.

Kernel-based methods improve policy evaluation in MRP models.

The generalization properties of Gaussian processes depend heavily on the choice of kernel, and this choice remains a dark art. We present the Neural Kernel Network (NKN), a flexible family of kernels represented by a neural network. The NKN architecture is based on the composition rules for kernels, so that each unit …

The paper analyzes covariate shift in nonparametric regression with Markovian data.

We address the problem of inverse reinforcement learning in Markov decision processes where the agent is risk-sensitive. In particular, we model risk-sensitivity in a reinforcement learning framework by making use of models of human decision-making having their origins in behavioral psychology, behavioral economics, an…

A new method calculates intrinsic effective sample size for manifold-valued data.

We consider Markov models of stochastic processes where the next-step conditional distribution is defined by a kernel density estimator (KDE), similar to Markov forecast densities and certain time-series bootstrap schemes. The KDE Markov models (KDE-MMs) we discuss are nonlinear, nonparametric, fully probabilistic repr…

This paper proposes a novel kernel approach to linear dimension reduction for supervised learning. The purpose of the dimension reduction is to find directions in the input space to explain the output as effectively as possible. The proposed method uses an estimator for the gradient of regression function, based on the…

We consider online learning for minimizing regret in unknown, episodic Markov decision processes (MDPs) with continuous states and actions. We develop variants of the UCRL and posterior sampling algorithms that employ nonparametric Gaussian process priors to generalize across the state and action spaces. When the trans…

Banded matrices can be used as precision matrices in several models including linear state-space models, some Gaussian processes, and Gaussian Markov random fields. The aim of the paper is to make modern inference methods (such as variational inference or gradient-based sampling) available for Gaussian models with band…

We introduce a new geometric approach that constructs a transition kernel of Markov chain. Our method always minimizes the average rejection rate and even reduce it to zero in many relevant cases, which cannot be achieved by conventional methods, such as the Metropolis-Hastings algorithm or the heat bath algorithm (Gib…

Proposes IGN for scalable Gaussian process networks.

Study OOD generalization in meta-reinforcement learning using information theory.

Improved KSD test for better detection of differences in distributions.

PDHAMS improves sampling for discrete distributions with quadratic potential functions.

We establish upper bounds for the minimal number of hidden units for which a binary stochastic feedforward network with sigmoid activation probabilities and a single hidden layer is a universal approximator of Markov kernels. We show that each possible probabilistic assignment of the states of $n$ output units, given t…

Study Markov chain gradient descent in Hilbert spaces for quadratic loss.

Kernel transfer operators, which can be regarded as approximations of transfer operators such as the Perron-Frobenius or Koopman operator in reproducing kernel Hilbert spaces, are defined in terms of covariance and cross-covariance operators and have been shown to be closely related to the conditional mean embedding fr…

New method improves RL in continuous spaces with kernel smoothing.

Products of Hidden Markov Models(PoHMMs) are an interesting class of generative models which have received little attention since their introduction. This maybe in part due to their more computationally expensive gradient-based learning algorithm,and the intractability of computing the log likelihood of sequences under…