JaxSGMC simplifies SG-MCMC for Bayesian deep learning.
problem Uncertainty quantification in deep learning models.
method Modular stochastic gradient MCMC in JAX.
result Facilitates trustworthy neural network predictions.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “MCMC optimization”
Proposes MIVI for efficient posterior estimation and design of MCMC transitions.
problem Efficiently estimating posterior distributions in constrained time.
method Combines variational inference and MCMC with a variational distribution and optimized Markov chain.
result Optimized Markov chain improves variational distribution and vice versa, leading to more accurate posteriors.
Monte Carlo (MC) methods are widely used for Bayesian inference and optimization in statistics, signal processing and machine learning. A well-known class of MC methods are Markov Chain Monte Carlo (MCMC) algorithms. In order to foster better exploration of the state space, specially in high-dimensional applications, s…
Stochastic gradient Markov chain Monte Carlo (SG-MCMC) has been increasingly popular in Bayesian learning due to its ability to deal with large data. A standard SG-MCMC algorithm simulates samples from a discretized-time Markov chain to approximate a target distribution. However, the samples are typically highly correl…
There has been recent interest in developing scalable Bayesian sampling methods such as stochastic gradient MCMC (SG-MCMC) and Stein variational gradient descent (SVGD) for big-data analysis. A standard SG-MCMC algorithm simulates samples from a discrete-time Markov chain to approximate a target distribution, thus samp…
Unified framework for MCMC algorithms simplifies their design and application.
problem Diverse MCMC algorithms with varying principles and applications.
method Involutive MCMC (iMCMC) framework unifying MCMC approaches.
result Unified view of MCMC algorithms facilitates the development of new and more efficient algorithms.
This work optimizes MCMC algorithms for modern accelerators without synchronization overheads.
problem Synchronization overheads in vectorized MCMC algorithms.
method Designing single-chain MCMC algorithms as finite state machines (FSMs) to avoid synchronization.
result Achieves up to an order of magnitude speed-up in experiments.
Speeding up Markov Chain Monte Carlo (MCMC) for datasets with many observations by data subsampling has recently received considerable attention. A pseudo-marginal MCMC method is proposed that estimates the likelihood by data subsampling using a block-Poisson estimator. The estimator is a product of Poisson estimators,…
Paper proposes first unlearning algorithm for MCMC models.
problem Enforcing right to be forgotten in AI causes high costs for data deletion.
method Converts MCMC unlearning to explicit optimization problem, designs MCMC influence function.
result MCMC unlearning does not compromise generalizability of models.
A new method combines MCMC and VI using contrastive divergence.
problem Improving variational inference by incorporating MCMC steps.
method Introducing variational contrastive divergence (VCD) to optimize variational parameters.
result Optimizing VCD leads to better predictive performance in latent variable models.
New MCMC method tackles label-switching problem for clustering.
problem Label-switching problem impedes MCMC efficiency in discrete clustering.
method Formulated optimal transport couplings for partition space.
result Our method efficiently overcomes label-switching problem.
New methods tackle complex inverse problems with scalable optimization-based MCMC.
problem Estimating high-dimensional model parameters and hyperparameters in nonlinear hierarchical statistical inverse problems.
method Optimization-based Markov chain Monte Carlo (MCMC) methods using RTO and pseudo-marginal MCMC.
result Efficient sampling tools for hierarchical Bayesian inversion with robust performance to model parameter dimensions.
Stochastic gradient Markov chain Monte Carlo (SG-MCMC) methods are Bayesian analogs to popular stochastic optimization methods; however, this connection is not well studied. We explore this relationship by applying simulated annealing to an SGMCMC algorithm. Furthermore, we extend recent SG-MCMC methods with two key co…
Optimizes MCMC chains with neural control variates.
problem Reducing variance in Markov Chain Monte Carlo (MCMC) simulations.
method Uses neural networks as control variates to minimize asymptotic variance.
result Derives optimal convergence rate under various ergodicity assumptions.
Accelerated Langevin algorithm improves MCMC sampling efficiency.
problem Improving the efficiency of MCMC sampling methods.
method Formulated gradient-based MCMC as optimization on probability measures, showing underdamped Langevin performs accelerated gradient descent.
result Accelerated rates can be achieved for nonconvex functions using the Langevin algorithm.
The study examines MCMC methods for arbitrary objectives and finds likelihood sharpness impacts performance and regularization.
problem Limitations of MCMC methods for arbitrary objective functions.
method Two-block MCMC framework with Metropolis-Hastings and Gibbs sampling, exploring likelihood curvature and sharpness.
result Likelihood sharpness governs in-sample performance and regularization inferred by training data.
This paper analyzes MCMC algorithms on large graphs using Dirichlet forms.
problem Analyzing the behavior of MCMC algorithms in high-dimensional problems.
method Utilizes Mosco convergence of Dirichlet forms to study RWM algorithm on large graphs.
result Demonstrates the advantages of Dirichlet form approach over standard diffusion methods.
New method improves high-dimensional Bayesian optimization efficiency using MCMC.
problem High-dimensional optimization challenges and computational complexity.
method Markov Chain Monte Carlo (MCMC) to efficiently sample from approximated posterior.
result Metropolis-Hastings and Langevin Dynamics versions outperform state-of-the-art methods.
New analysis of SGD with MCMC gradient estimator shows convergence rate and saddle point escape.
problem Analyzing SGD with MCMC gradient estimator under complex conditions.
method Introduced MCMC-SGD, analyzed convergence rate and saddle point escape using Bernstein inequality.
result Proven first order convergence rate and saddle point escape at least steps.
Optimal preconditioning improves Langevin sampling efficiency.
problem Improving sampling efficiency in high-dimensional target distributions.
method Optimal preconditioning using Fisher information, applied to MALA.
result Adaptive MCMC scheme significantly outperforms other methods.
New model improves MCMC efficiency and multi-modal distribution exploration.
problem Inefficient and slow MCMC methods for complex distributions.
method Deep involutive generative models for Metropolis-Hastings updates.
result Deep involutive models can learn complex MCMC updates efficiently.
Stochastic gradient Markov Chain Monte Carlo (SG-MCMC) has been developed as a flexible family of scalable Bayesian sampling algorithms. However, there has been little theoretical analysis of the impact of minibatch size to the algorithm's convergence rate. In this paper, we prove that under a limited computational bud…
We introduce Tempered Geodesic Markov Chain Monte Carlo (TG-MCMC) algorithm for initializing pose graph optimization problems, arising in various scenarios such as SFM (structure from motion) or SLAM (simultaneous localization and mapping). TG-MCMC is first of its kind as it unites asymptotically global non-convex opti…
Recent advances in Bayesian learning with large-scale data have witnessed emergence of stochastic gradient MCMC algorithms (SG-MCMC), such as stochastic gradient Langevin dynamics (SGLD), stochastic gradient Hamiltonian MCMC (SGHMC), and the stochastic gradient thermostat. While finite-time convergence properties of th…
Neural network MCMC sampler maximizes proposal entropy for efficient sampling.
problem Inefficient sampling from complex probability distributions.
method Proposes a neural network MCMC sampler that maximizes proposal entropy.
result Significantly higher efficiency in various sampling tasks.
AI-driven framework optimizes MCMC-based preconditioners for faster linear system solving.
problem Slow convergence of Krylov subspace solvers for ill-conditioned matrices.
method Graph neural surrogate and Bayesian optimization for AI-tuned MCMC parameters.
result 50% reduction in iterations to convergence on unseen system.
Paper proposes a fast method for learning deep latent variable models.
problem Learning deep generative models with hierarchical latent variables.
method Noise initialized short run MCMC with variational optimization of step size.
result The method outperforms VAE in reconstruction and synthesis quality.
This paper proposes a new randomized strategy for adaptive MCMC using Bayesian optimization. This approach applies to non-differentiable objective functions and trades off exploration and exploitation to reduce the number of potentially costly objective function evaluations. We demonstrate the strategy in the complex s…
A new method de-randomizes MCMC dynamics using the Stein operator.
problem Estimating complex target distributions in Bayesian inference.
method De-randomized kernel-based particle samplers that discretize the fiber-gradient Hamiltonian flow.
result GSVGD de-randomizes complex MCMC dynamics, maintaining high sample quality.
A new MCMC method combines low and high-fidelity models to reduce computation.
problem Inefficient computation of expensive target densities in scientific applications.
method Pseudo-marginal MCMC approach using a telescoping series of low-fidelity models.
result Asymptotically exact multi-fidelity MCMC algorithms for reduced computational cost.
Convergence Acceleration of Markov Chain Monte Carlo-based Gradient Descent by Deep Unfoldingcond-mat.dis-nn
Deep unfolding accelerates MCMC-based COP solvers.
problem Optimizing combinatorial problems with MCMC and gradient descent.
method Combines MCMC and gradient descent, trains step sizes, uses variance estimation for non-differentiable MCMC.
result Significantly accelerates convergence speed for COPs.
This work uses neural networks to optimize decision-making processes constrained by limited information.
problem Decision-making under resource constraints.
method Adaptive neural network priors integrated with anytime MCMC optimization.
result Neural network priors improve decision-making efficiency over fixed priors.
Adaptive stopping in MCMC using classifier-based dynamics
problem Sampling from complex, unnormalized probability densities
method Training state-dependent neural classifiers
result Significant reduction in average trajectory lengths
Researchers analyze inverse optimal transport, deriving theoretical and empirical insights.
problem Understanding the inverse problem of inferring cost matrices from optimal couplings.
method Formalized and analyzed using entropy-regularized optimal transport, with theoretical and empirical contributions.
result Characterization of the manifold of cross-ratio equivalent costs and derivation of an MCMC sampler.
Improved sampling accuracy in SG-MCMC methods via non-uniform gradient subsampling.
problem Computational inefficiency and sampling error in stochastic gradient MCMC methods.
method Proposes a non-uniform subsampling scheme to reduce sampling error in EWSG, a variant of SG-MCMC.
result EWSG reduces sampling error compared to uniform subsampling, improving accuracy without sacrificing convergence speed.
Markov-Chain Monte Carlo Approximation of the Ideal Observer using Generative Adversarial Networkseess.SP
New method approximates Ideal Observer using GANs and MCMC.
problem Intractable computation of Ideal Observer test statistic.
method Applying MCMC to SOMs learned by GANs.
result Extends applicability of MCMC for IO performance.
Optimizes MCMC output compression by selecting a subset of states.
problem Improving the efficiency of MCMC output compression.
method Greedy minimization of kernel Stein discrepancy for selecting a subset of states.
result The method provides an optimal subset of states for empirical approximation.
Subsampling MCMC speeds up Bayesian inference for large datasets.
problem Computational demands of MCMC for large datasets.
method Pseudo-marginal MCMC approach using data subsampling.
result Subsampling MCMC can significantly speed up MCMC methods.
New MCMC method speeds up inference for complex models.
problem Intractable Bayesian inference for large datasets and high-dimensional models.
method ADMM-type splitting strategy for MCMC sampling.
result Empirical and theoretical convergence rates established.
Bayesian inference improves deep learning performance and uncertainty.
problem Overfitting and lack of robustness in deep learning models.
method ATMC (Adaptive Noise MCMC) algorithm for sampling from posterior distributions of neural networks.
result ATMC outperforms optimization baselines in classification accuracy and test log-likelihood on Cifar10 and ImageNet benchmarks.
Machine learning speeds up LSM parameter optimization and uncertainty quantification.
problem Optimizing and assessing uncertainty of LSM parameters.
method Combining MCMC with Gaussian process regression.
result 50,000 times faster than direct MCMC application.
MCMC complexity matches optimization for large and .
problem Lack of theoretical understanding of MCMC complexity for large and .
method Comparison of MCMC, LA, and VI complexities for linear, logistic, and Poisson regression.
result MCMC complexity matches optimization complexity for .
A new MH-MCMC method using mini-batches and stochastic gradient for scalable inference.
problem Computational inefficiency of traditional MCMC algorithms for large datasets.
method Mini-batch MH-MCMC with reversible stochastic gradient proposal.
result The method provides approximate tempered stationary distribution and reasonable acceptance probabilities.
HDT improves MCMC on graphs with history-dependent sampling.
problem Efficient sampling from target distributions on general graphs with low computational overhead.
method History-driven target (HDT) framework that replaces the original target distribution with a history-dependent one.
result Near-zero variance performance and scalability to large graphs with memory-efficient implementation.
MCMC-driven learningstat.ML
Unified framework for MCMC and machine learning problems.
problem Intersection of MCMC and machine learning problems.
method Unified framework integrating various MCMC and machine learning techniques.
result Translation and generalization of theory and methods.
Develops a new framework to understand MCMC dynamics as flows on Wasserstein space.
problem Lack of understanding general MCMC dynamics in terms of flows on Wasserstein space.
method Introduces novel concepts to recognize MCMC dynamics as fiber-gradient Hamiltonian flows on Wasserstein space.
result Enables ParVI simulation of MCMC dynamics, enriching ParVI family with more efficient dynamics.
The paper explores a non-convergent MCMC method for EBM learning.
problem Learning energy-based models using traditional methods is challenging.
method The paper uses a non-convergent, non-mixing, and non-persistent short-run MCMC to learn EBM parameters.
result The learned short-run MCMC can generate realistic images and reconstruct/interpolate between images.
BiDAG R package learns and samples Bayesian network structures efficiently.
problem Efficiently learning and sampling Bayesian network structures.
method Hybrid approach combining PC algorithm, iterative order MCMC, and partition MCMC.
result BiDAG can handle both discrete and continuous data.