arXiv research

New algorithm detects changes in Markov kernels with unknown post-change kernel.

Stochastic kernel based dimensionality reduction approaches have become popular in the last decade. The central component of many of these methods is a symmetric kernel that quantifies the vicinity between pairs of data points and a kernel-induced Markov chain on the data. Typically, the Markov chain is fully specified…

Identifies conditions for multiple invariant probabilities in Markov kernels.

New method trains Markov kernels for efficient sampling.

A new HMM model captures kernel dependencies using context-specific Bayesian networks.

We consider a heat kernel approach for the development of stochastic pricing kernels. The kernels are constructed by positive propagators, which are driven by time-inhomogeneous Markov processes. We multiply such a propagator with a positive, time-dependent and decreasing weight function, and integrate the product over…

New method detects changes in high-dimensional Markov processes without explicit likelihood evaluation.

We develop algorithms with low regret for learning episodic Markov decision processes based on kernel approximation techniques. The algorithms are based on both the Upper Confidence Bound (UCB) as well as Posterior or Thompson Sampling (PSRL) philosophies, and work in the general setting of continuous state and action …

New concentration inequality for U-statistics of Markov chains.

New method tunes SMC samplers efficiently without high costs.

Paper improves generalization bounds for multi-kernel learning with mixed datasets.

This paper develops tools for nonreversible MCMC with convergence guarantees.

Kernel-based methods improve policy evaluation in MRP models.

The paper analyzes covariate shift in nonparametric regression with Markovian data.

A new method calculates intrinsic effective sample size for manifold-valued data.

We consider Markov models of stochastic processes where the next-step conditional distribution is defined by a kernel density estimator (KDE), similar to Markov forecast densities and certain time-series bootstrap schemes. The KDE Markov models (KDE-MMs) we discuss are nonlinear, nonparametric, fully probabilistic repr…

We consider online learning for minimizing regret in unknown, episodic Markov decision processes (MDPs) with continuous states and actions. We develop variants of the UCRL and posterior sampling algorithms that employ nonparametric Gaussian process priors to generalize across the state and action spaces. When the trans…

We introduce a new geometric approach that constructs a transition kernel of Markov chain. Our method always minimizes the average rejection rate and even reduce it to zero in many relevant cases, which cannot be achieved by conventional methods, such as the Metropolis-Hastings algorithm or the heat bath algorithm (Gib…

Improved KSD test for better detection of differences in distributions.

We establish upper bounds for the minimal number of hidden units for which a binary stochastic feedforward network with sigmoid activation probabilities and a single hidden layer is a universal approximator of Markov kernels. We show that each possible probabilistic assignment of the states of $n$ output units, given t…

Study Markov chain gradient descent in Hilbert spaces for quadratic loss.

New method improves RL in continuous spaces with kernel smoothing.

DenseHMM improves HMMs by learning dense representations that enable gradient-based optimization.

We solve a Schrödinger bridge with a quadratic state cost, finding a closed-form solution.

Algorithm finds Nash equilibria in complex games with function approximation.

Method infers causal structure from system behaviors using RKHS and kernel $ε$-machines.

In machine learning, a nonparametric forecasting algorithm for time series data has been proposed, called the kernel spectral hidden Markov model (KSHMM). In this paper, we propose a technique for short-term wind-speed prediction based on KSHMM. We numerically compared the performance of our KSHMM-based forecasting tec…

New volatility model for option pricing with time-varying risk premium.

Developing feature selection algorithms that move beyond a pure correlational to a more causal analysis of observational data is an important problem in the sciences. Several algorithms attempt to do so by discovering the Markov blanket of a target, but they all contain a forward selection step which variables must pas…

New bounds for SMC show its advantage over MCMC in multimodal distributions.

Proposes ABC method for discrete data, improving likelihood-free inference.

We propose a novel distributed inference algorithm for continuous graphical models, by extending Stein variational gradient descent (SVGD) to leverage the Markov dependency structure of the distribution of interest. Our approach combines SVGD with a set of structured local kernel functions defined on the Markov blanket…

BARK optimizes black-box functions using Bayesian Additive Regression Trees.

The existence of stationary Markov perfect equilibria in stochastic games is shown under a general condition called "(decomposable) coarser transition kernels". This result covers various earlier existence results on correlated equilibria, noisy stochastic games, stochastic games with finite actions and state-independe…

We consider the problem of learning low-dimensional representations for large-scale Markov chains. We formulate the task of representation learning as that of mapping the state space of the model to a low-dimensional state space, called the kernel space. The kernel space contains a set of meta states which are desired …

Stein importance sampling is a widely applicable technique based on kernelized Stein discrepancy, which corrects the output of approximate sampling algorithms by reweighting the empirical distribution of the samples. A general analysis of this technique is conducted for the previously unconsidered setting where samples…

Existing Markov Chain Monte Carlo (MCMC) methods are either based on general-purpose and domain-agnostic schemes which can lead to slow convergence, or hand-crafting of problem-specific proposals by an expert. We propose A-NICE-MC, a novel method to train flexible parametric Markov chain kernels to produce samples with…

Researchers approximate conditional expectation operators using kernel methods.

Extended elliptical slice sampling for infinite-dimensional spaces, proving reversibility.

Transformers with multiple layers learn to estimate bigram distributions, while single-layer models often get stuck in unigram local minima.

New RL method explores environments without rewards, achieving efficient policy generation.

New density estimator from Markov Chains outperforms KDE.

We investigate the statistical complexity of estimating the parameters of a discrete-state Markov chain kernel from a single long sequence of state observations. In the finite case, we characterize (modulo logarithmic factors) the minimax sample complexity of estimation with respect to the operator infinity norm, while…

Generative models using PDMPs with explicit jump rates and kernels.

We propose kernel sequential Monte Carlo (KSMC), a framework for sampling from static target densities. KSMC is a family of sequential Monte Carlo algorithms that are based on building emulator models of the current particle system in a reproducing kernel Hilbert space. We here focus on modelling nonlinear covariance s…

The paper develops new inequalities for Markov chain sums, linking them to mixing time.

Schrödinger bridge solved with Weyl calculus for quadratic state cost.

Kernel methods summarize and integrate posterior similarity matrices from Bayesian clustering.