New algorithm tests Markov chains without hitting.
problem Testing Markov chains with unknown transition matrix.
method Combining approximation algorithms and spectral analysis.
result Efficient testing of Markov chains without hitting time dependence.
Study Markov chain gradient descent in Hilbert spaces for quadratic loss.
problem Approximating optimal solutions for quadratic loss functions.
method Developed a Markov chain-based stochastic gradient algorithm in Hilbert spaces.
result Established probabilistic upper bounds on convergence.
We approximate sticky diffusions using Markov chains for efficient simulation.
problem Approximating sticky diffusions for accurate simulation.
method CTMC approximation of sticky diffusions, efficient matrix exponentials, and Euler scheme comparison.
result Second order convergence of CTMC approximation for sticky diffusions.
Deviation inequalities for stochastic approximation methods.
problem Establishing bounds on the deviation of stochastic approximation methods.
method Martingale approximation method for separately Lipschitz functions.
result Established various deviation inequalities for stochastic approximation by averaging and minimization.
Study on Mirror Langevin diffusions for convergence rates and Markov chain approximations.
problem Convergence rates and approximations for Mirror Langevin diffusions.
method Lyapunov function methods, Poincaré and log-Sobolev inequalities, Gibbs sampler, Sinkhorn Markov chain.
result Sufficient conditions for exponential convergence of Mirror Langevin diffusions and a guaranteed convergence rate for the Markov chain approximation.
Study shows TD(0) with linear approx. converges for reversible Markov chains.
problem TD(0) divergence with off-policy and function approximation.
method Analyzes standard TD(0) with reversible Markov chains, adapting stochastic approximation framework.
result Establishes convergence with probability one for projected Bellman error = 0.
Estimates Markov chain variance efficiently without storing samples.
problem Estimating the asymptotic variance of Markov chain functions.
method Linear stochastic approximation of Poisson equation solution.
result Optimal MSE convergence rate with finite sample guarantees.
We consider a Hidden Markov Model (HMM) where the integrated continuous-time Markov chain can be observed at discrete time points perturbed by a Brownian motion. The aim is to derive a filter for the underlying continuous-time Markov chain. The recursion formula for the discrete-time filter is easy to derive, however i…
Uniform TD(0) bound derived for function approximation with Markov noise.
problem Uniform concentration bound for TD(0) with function approximation.
method Contractive stochastic approximation, martingale and Markov noises, Poisson equation, relaxed concentration inequalities.
result Uniform all-time concentration bound for TD(0) with linear function approximation.
Probabilistic models are conceptually powerful tools for finding structure in data, but their practical effectiveness is often limited by our ability to perform inference in them. Exact inference is frequently intractable, so approximate inference is often performed using Markov chain Monte Carlo (MCMC). To achieve the…
New method for fluid approximation of CTMCs without population structure.
problem Approximating the macro-scale behavior of large CTMCs.
method Spectral analysis of CTMC transition matrix, diffusion maps, Gaussian process regression.
result Construct an ODE approximating CTMC mean in continuous space.
The paper estimates key metrics for linear models with Markov or hidden Markov sources.
problem Estimating free energy, mutual information, and MMSE for linear models with specific signal priors.
method Replica analysis in statistical physics, focusing on Markov and hidden Markov sources.
result The linear model with Markov or hidden Markov sources can be simplified into decoupled AWGN channels.
Markov chains and diffusion processes are indispensable tools in machine learning and statistics that are used for inference, sampling, and modeling. With the growth of large-scale datasets, the computational cost associated with simulating these stochastic processes can be considerable, and many algorithms have been p…
Paper develops a method to approximate Markov chains with fewer states.
problem Identifying the state aggregation structure of Markov chains with fewer states.
method Proposes a convex optimization problem with a nonnegative factorization approach.
result The method likely converges to the global solution and outperforms existing methods.
Investor selects portfolios based on news attention in a hidden Markov model.
problem Mean-variance portfolio selection in a dynamic attention context.
method Closed-loop equilibrium strategies via extended HJB equation and Markov chain approximation.
result Equilibrium strategies found through iterative algorithm and numerical examples.
Graphical models are popular statistical tools which are used to represent dependent or causal complex systems. Statistically equivalent causal or directed graphical models are said to belong to a Markov equivalent class. It is of great interest to describe and understand the space of such classes. However, with curren…
The paper studies how quickly samples from Langevin dynamics become independent.
problem Understanding the dependence between samples along Langevin dynamics and related algorithms.
method Measures dependence via Φ-mutual information and proves strong data processing inequalities. result The Φ-mutual information between samples decreases exponentially to zero. New bounds for SMC show its advantage over MCMC in multimodal distributions.
problem Estimating expectations under multimodal distributions with slow global mixing.
method Proves finite sample complexities for SMC with local mixing times, addressing bias through sequential resampling.
result SMC provides fully polynomial time approximation for multimodal problems.
Study approximates financial market with discrete-time models.
problem Approximating continuous-time financial market models with discrete-time.
method Constructs discrete-time market models with Markov switching and proves convergence.
result Discrete-time models converge to continuous-time Black-Scholes model with Markov switching.
New algorithms decode Markov chains with near-optimal performance, even with small latency.
problem Online decoding of nth order ergodic Markov chains with latency constraints. method Deterministic and randomized algorithms using dynamic programs, with lower bounds established.
result Near-optimal performance of algorithms with minimal latency, outperforming existing methods.
Proposes MIVI for efficient posterior estimation and design of MCMC transitions.
problem Efficiently estimating posterior distributions in constrained time.
method Combines variational inference and MCMC with a variational distribution and optimized Markov chain.
result Optimized Markov chain improves variational distribution and vice versa, leading to more accurate posteriors.
Paper introduces a diagnostic for approximate inference methods.
problem Estimating errors in probabilistic inference algorithms, especially for approximate methods.
method Repeatedly simulate datasets from the prior and perform inference on each, estimating a symmetric KL-divergence.
result A diagnostic for approximate inference methods can be estimated using symmetric KL-divergence.
New method simulates sticky boundaries in multidimensional diffusions.
problem Simulating sticky boundaries in multidimensional diffusions.
method Approximate sticky diffusion by a Markov chain, using either finite difference or matching local moments.
result Validates both construction methods for first-order simulation schemes.
First DP MCMC algorithm for arbitrary models.
problem Privacy-preserving Bayesian inference in arbitrary models.
method Decomposition of Barker acceptance test for Rényi DP privacy cost.
result First general DP MCMC algorithm with improved privacy guarantees.
We analyze a new Markov chain model for better sampling and optimization.
problem Developing a new Markov chain model for improved sampling and optimization.
method We introduce a new class of Ito chains with arbitrary noise and inexact drift/diffusion coefficients, proving a bound in W2-distance. result Our analysis provides improved or first results for various applications like SGLD, sampling, and boosting.
Novel bounds improve TD learning consistency in RL.
problem Analyzing Temporal Difference learning's performance.
method High-dimensional concentration inequalities and Berry-Esseen bounds for Markov chain induced martingales.
result Sharp high-probability consistency guarantee for TD learning, matching asymptotic variance up to logarithmic factors.
The paper analyzes stability of random matrix products with Markovian noise.
problem Analyzing stability of random matrix products with Markovian noise.
method Using a super-Lyapunov drift condition and controlled growth of matrix-valued functions, the paper provides an exponential stability result for the p-th moment of random matrix product.
result Finite-time p-th moment bounds for linear stochastic approximation and TD learning algorithms.
The paper develops a Hoeffding inequality for Markov chains and applies it to bandit problems.
problem Developing a Hoeffding inequality for Markov chains and applying it to bandit problems.
method Developed a Hoeffding inequality for the partial sums of an irreducible Markov chain on a finite state space.
result Demonstrated the inequality's effectiveness in identifying approximately best Markovian arms and minimizing regret in Markovian bandits.
The paper tackles learning from non-irreducible Markov chains, proving learnability and generalization bounds.
problem Learning from temporal dependent data with non-irreducible Markov chains.
method Uniform convergence and generalization bounds for sample error under uniform ergodicity.
result Learnability and generalization bounds for approximate sample error minimization algorithm.
Regime-switching models, in particular Hidden Markov Models (HMMs) where the switching is driven by an unobservable Markov chain, are widely-used in financial applications, due to their tractability and good econometric properties. In this work we consider HMMs in continuous time with both constant and switching volati…
Paper approximates rough stochastic local volatility models for efficient computation.
problem No unified method for rough stochastic local volatility models.
method Semimartingale and continuous-time Markov chain approximation.
result Fast CTMC algorithm with weak convergence proved.
Method calculates Parisian stopping times and option prices using Markov chains.
problem Computing distribution and pricing of Parisian stopping times under Markov processes.
method Continuous-time Markov chain approximation to solve for distribution and convergence analysis.
result Sharp convergence rate and efficient method for diffusion and jump models.
New Stein Points use Markov chains to select points, reducing computational cost.
problem Approximating a probability measure with an empirical measure.
method Select points based on Markov chain sample paths, solving a non-convex optimisation problem.
result Significantly reduces computational cost and establishes consistency guarantees.
This paper develops numerical methods for finding optimal dividend pay-out and reinsurance policies. A generalized singular control formulation of surplus and discounted payoff function are introduced, where the surplus is modeled by a regime-switching process subject to both regular and singular controls. To approxima…
Stochastic approximation algorithms show exponential progress bounds.
problem Analyzing the convergence of stochastic approximation algorithms.
method Developed geometric ergodicity proofs to establish exponential concentration bounds.
result Proved faster convergence rates for specific algorithms.
Optimizes MCMC chains with neural control variates.
problem Reducing variance in Markov Chain Monte Carlo (MCMC) simulations.
method Uses neural networks as control variates to minimize asymptotic variance.
result Derives optimal convergence rate under various ergodicity assumptions.
The paper proposes methods to estimate MCMC quality with couplings, bounding Wasserstein distance.
problem Improving MCMC efficiency without sacrificing asymptotic consistency.
method Estimators based on couplings of Markov chains to assess quality of asymptotically biased sampling methods.
result Empirical upper bounds of Wasserstein distance for assessing MCMC quality.
The paper develops fast Bayesian methods for estimating huge PVARs with competitive forecasts.
problem Computational and statistical issues in estimating PVARs with many parameters.
method Integrated rotated Gaussian approximations, exploiting domestic over international information, and fast approximations for international coefficients.
result Produces competitive forecasts quickly using a huge world economy model.
A new eigenvalue-based method speeds up Monte Carlo simulations.
problem Reducing the number of paths needed for accurate Monte Carlo simulations.
method Eigenvalue-based approximation of Markov Chain Monte Carlo.
result Significant variance reduction and comparable results to traditional Monte Carlo.
New method resolves time order in genetic mutation models.
problem Underspecification in modeling genetic mutation time evolution.
method Continuous-time Markov chains with additional independent items.
result Additional items help determine time order and resolve underspecification.
New method reduces computational cost for Bayesian inference.
problem High computational cost in Bayesian inference.
method Coreset MCMC, a Markov chain targeting coreset posterior.
result Coreset MCMC provides higher quality posterior approximations and reduced computational cost.
Efficiently infers coupled hidden Markov models with noisy discrete observations.
problem Intractable inference for coupled continuous-time Markov chains with discrete observations.
method Latent Interacting Particle Systems, look-ahead functions, twisted Sequential Monte Carlo sampling.
result Demonstrated effectiveness on latent SIRS model and wildfire spread dynamics.
Improved VAEs by using undirected graphical models as approximate posteriors.
problem Mismatch between approximate and true posterior in VAEs.
method Trained undirected graphical models using backpropagation through Markov chain Monte Carlo updates.
result Undirected models outperform directed models in VAEs.
A key task in Bayesian statistics is sampling from distributions that are only specified up to a partition function (i.e., constant of proportionality). However, without any assumptions, sampling (even approximately) can be #P-hard, and few works have provided "beyond worst-case" guarantees for such settings. For log-c…
The paper provides concentration inequalities for Markov chain variance estimators.
problem Estimating the variance of Markov chains with concentration properties.
method Martingale decomposition method for uniformly geometrically ergodic Markov chains.
result Explicit control of the p-th moment of the OBM estimator difference and dependence on p and mixing time.
Gen-Oja efficiently computes principal vectors and canonical correlations in streaming data.
problem Principal Generalized Eigenvector computation and Canonical Correlation Analysis in stochastic settings.
method Gen-Oja is a simple and efficient algorithm that leverages two-time-scale stochastic approximation and fast-mixing Markov chains.
result Gen-Oja achieves optimal convergence rates for these problems.
A new method simulates a lazy version of a Markov chain for empirical inference.
problem Estimating and testing unknown Markov chains with limited data.
method Simulates an α-lazy version of an unknown Markov chain, making it ergodic.
result The pseudo spectral gap can be applied to non-ergodic Markov chains.
Develops a new risk measure for Markov chains' asymptotic behavior.
problem Lack of risk measures for asymptotic regimes of Markov chains.
method Simulation-based approach using large deviations theory, density estimation, and stochastic approximation.
result Developed Asymptotic CVaR (ACVaR) for Markov chains.