D-Wave computers struggle with sampling Boltzmann distributions efficiently.
problem Sampling Boltzmann distributions efficiently on D-Wave computers.
method Exploring various obstacles and remaining difficulties.
result Challenges remain in using D-Wave computers for efficient sampling.
A new method speeds up sampling of Boltzmann distribution in high-dimensional systems.
problem High computational cost of obtaining Jacobian of flow-based models in high dimensions.
method Flow perturbation method that incorporates stochastic perturbations and reweighting.
result Achieves unbiased sampling of Boltzmann distribution with orders of magnitude speedup.
Transferable Boltzmann Generators learn to sample unseen molecules efficiently.
problem Generating equilibrium samples of molecular systems.
method Boltzmann Generators using normalizing flows to learn transformations.
result Transferable Boltzmann Generators can predict zero-shot Boltzmann distributions for unseen molecules.
Quantum Boltzmann Machines trained on quantum annealers produce noisy synthetic data.
problem Training quantum Boltzmann machines on quantum annealers for financial data generation.
method Used D-Wave Advantage 4.1 quantum annealer to train QBMs and compare with classical RBMs.
result Quantum Boltzmann Machines trained on quantum annealers are noisier and less effective than classical RBMs.
EDG generates Boltzmann samples from latent variables efficiently.
problem Sampling from complex energy functions in high dimensions.
method Combines variational autoencoders and diffusion models; uses a decoder and diffusion-based encoder.
result EDG outperforms existing methods in various sampling tasks.
We present transductive Boltzmann machines (TBMs), which firstly achieve transductive learning of the Gibbs distribution. While exact learning of the Gibbs distribution is impossible by the family of existing Boltzmann machines due to combinatorial explosion of the sample space, TBMs overcome the problem by adaptively …
We propose a relaxation-based approximate inference algorithm that samples near-MAP configurations of a binary pairwise Markov random field. We experiment on MAP inference tasks in several restricted Boltzmann machines. We also use our underlying sampler to estimate the log-partition function of restricted Boltzmann ma…
CG-BGs combine flow-based models with PMFs to sample large systems efficiently.
problem Sampling equilibrium molecular configurations from the Boltzmann distribution is challenging.
method Coarse-grained Boltzmann Generators (CG-BGs) use flow-based models and learned PMFs for efficient sampling.
result CG-BGs provide a practical route for sampling larger molecular systems efficiently.
This paper provides a tutorial on Boltzmann Machines and Deep Belief Networks.
problem Understanding and applying Boltzmann Machines and Deep Belief Networks.
method Explains the structures, conditional distributions, Gibbs sampling, training methods, and deep belief networks of RBMs.
result Comprehensive overview of RBMs and DBNs, useful in various fields.
CAP-BM learns complex-valued data's amplitude and phase distributions.
problem Learning from complex-valued data with amplitude variation.
method Complex Amplitude-Phase Boltzmann machine (CAP-BM) with Gibbs sampling.
result Necessity of amplitude-amplitude coupling term in CAP-BM.
EWFM trains continuous flows with only energy evaluations, improving sample quality with fewer computations.
problem Efficiently sampling from complex, high-dimensional Boltzmann distributions using only energy evaluations.
method Energy-Weighted Flow Matching (EWFM) using importance sampling and iterative/annealed training.
result Improved sample quality with up to 3 orders of magnitude fewer energy evaluations compared to existing methods.
BNEM improves Boltzmann sampler efficiency.
problem Generating IID samples from Boltzmann distributions efficiently.
method Bootstrapped Noised Energy Matching (NEM) combined with diffusion-based learning and bootstrapping.
result BNEM achieves state-of-the-art performance with improved robustness.
Boltzmann machines are physics informed generative models with wide applications in machine learning. They can learn the probability distribution from an input dataset and generate new samples accordingly. Applying them back to physics, the Boltzmann machines are ideal recommender systems to accelerate Monte Carlo simu…
Learning in restricted Boltzmann machine is typically hard due to the computation of gradients of log-likelihood function. To describe the network state statistics of the restricted Boltzmann machine, we develop an advanced mean field theory based on the Bethe approximation. Our theory provides an efficient message pas…
Paper introduces a new sampling method combining Consistency Models with importance sampling.
problem Inherent errors in samples and high NFEs for high-quality samples in Boltzmann distributions.
method Combines Consistency Models with importance sampling to produce unbiased samples with minimal NFEs.
result Produces unbiased samples using only 6-25 NFEs, comparable to 100 NFEs for DDPMs.
Jeffreys Flow improves robustness of Boltzmann generators for rare event sampling.
problem Rare events and metastable trapping in sampling physical systems with rough energy landscapes.
method Introduces Jeffreys Flow, a robust generative framework using Parallel Tempering distillation and symmetric Jeffreys divergence to mitigate mode collapse and improve mode coverage.
result Minimizing Jeffreys divergence suppresses mode collapse and corrects inaccuracies in multi-modal distributions.
VT-DIS improves sampling from Boltzmann distributions with minimal overhead.
problem Bias in Monte Carlo estimates from score-based diffusion models.
method Variance-Tuned Diffusion Importance Sampling (VT-DIS) adapts noise covariance to correct bias.
result VT-DIS achieves effective sample sizes of 80%, 35%, and 3.5% on benchmarks, using less computational budget.
Computing equilibrium states in condensed-matter many-body systems, such as solvated proteins, is a long-standing challenge. Lacking methods for generating statistically independent equilibrium samples in "one shot", vast computational effort is invested for simulating these system in small steps, e.g., using Molecular…
A new sampler and temperature estimation method enable efficient learning of Boltzmann Machines.
problem Efficient learning of Boltzmann Machines (BMs) is challenging due to high training costs and difficulty in parallelization.
method Proposed a new Boltzmann sampler (Langevin SB, LSB) and an efficient method (Conditional Expectation Matching, CEM) for estimating inverse temperature.
result Established an efficient learning framework (Sampler-Adaptive Learning, SAL) for BMs with greater expressive power than Restricted Boltzmann Machines (RBMs).
ASBS improves sampling from Boltzmann distributions without importance weighting.
problem Sampling from Boltzmann distributions with known energies but unknown samples.
method Adjoint Schrödinger Bridge Sampler using kinetic-optimal transportation.
result ASBS achieves scalable and efficient sampling without importance weighting.
A new training method for normalizing flows without samples.
problem Training normalizing flows without samples but with energy functions.
method Interpolates energy functions to find a transport vector field.
result Optimizes transport vector field and energy function to satisfy continuity equation.
New method generates molecular conformations efficiently.
problem Generating accurate molecular conformations efficiently.
method Variational approximation of rotatable bond torsion angles as a mixture of von Mises distributions.
result VonMisesNet generates conformations orders of magnitude faster than existing methods.
Quantum annealing (QA) is a hardware-based heuristic optimization and sampling method applicable to discrete undirected graphical models. While similar to simulated annealing, QA relies on quantum, rather than thermal, effects to explore complex search spaces. For many classes of problems, QA is known to offer computat…
We show that the visible sector probability density function of the Riemann-Theta Boltzmann machine corresponds to a gaussian mixture model consisting of an infinite number of component multi-variate gaussians. The weights of the mixture are given by a discrete multi-variate gaussian over the hidden state space. This a…
A new training method for efficient Boltzmann generators.
problem Training equivariant continuous normalizing flows (CNFs) is computationally expensive.
method Equivariant flow matching, based on optimal transport flow matching.
result Equivariant flow matching yields more efficient flows with shorter integration paths.
The Boltzmann machine provides a useful framework to learn highly complex, multimodal and multiscale data distributions that occur in the real world. The default method to learn its parameters consists of minimizing the Kullback-Leibler (KL) divergence from training samples to the Boltzmann model. We propose in this wo…
iDEM generates samples from Boltzmann densities without data.
problem Generating statistically independent samples from unnormalized distributions.
method Iterative algorithm using energy and gradient for diffusion-based sampler training.
result iDEM achieves state-of-the-art performance and trains faster than existing methods.
Flow AIS Bootstrap improves flow training by generating samples in hard-to-reach regions.
problem Training flows with high variance and mode-seeking behavior.
method Augment flows with AIS and minimize α-divergence with α=2. result FAB learns Boltzmann distribution of alanine dipeptide without MD samples.
Restricted Boltzmann Machines (RBMs) are one of the fundamental building blocks of deep learning. Approximate maximum likelihood training of RBMs typically necessitates sampling from these models. In many training scenarios, computationally efficient Gibbs sampling procedures are crippled by poor mixing. In this work w…
Paper proposes efficient training for normalizing flows in Boltzmann generators.
problem Training normalizing flows for Boltzmann generators is computationally challenging and unstable.
method Regression Training of Normalizing Flows (RegFlow) using ℓ2-regression. result RegFlow enables efficient and stable training of normalizing flows for Boltzmann generators.
SCALLOP improves likelihood flow maps for efficient Boltzmann generation.
problem Efficient estimation of model likelihood in flow-based generative models.
method SCALLOP introduces a Hutchinson-free likelihood distillation objective for scalable flow-based models.
result SCALLOP achieves up to 10x inference speedup while improving performance.
D-Wave quantum annealing fails to improve sampling quality from RBMs compared to Gibbs sampling.
problem Improving sampling quality from RBMs using D-Wave quantum annealing.
method Comparison of D-Wave quantum annealing and Gibbs sampling for RBM sampling.
result D-Wave sampling does not significantly improve the number of local valleys compared to Gibbs sampling.
AIS method improves estimation of RBM partition function with reduced computational cost.
problem Efficiently estimating partition function of RBMs for large systems.
method Annealed Importance Sampling (AIS) with optimized initialization.
result Good estimation of partition function Z with reduced computational cost.
Improved sampling efficiency for molecular systems using path gradients after Flow Matching.
problem Improving sampling efficiency for complex molecular systems.
method Hybrid approach combining Flow Matching and path gradients.
result Up to a threefold increase in sampling efficiency for molecular systems.
A new method for sampling from complex distributions using Langevin samplers.
problem Sampling from unnormalized Boltzmann densities.
method Probability flow ODE derived from linear stochastic interpolants, employing Langevin samplers.
result Efficient simulation of the flow with non-asymptotic convergence rate.
Quantum machine learns faster by reverse annealing on AQCs.
problem Training RBMs on AQCs is hard due to low qubit connectivity.
method Embedding RBM nodes to virtual qubits, semantic quantum search, reverse annealing schedule.
result Reverse annealing accelerates RBM training and improves reconstruction scores.
New method uses RBM flows to find critical temperatures in Ising models.
problem Detecting critical temperatures in RBM flows without model topology information.
method Iterative sampling from RBM mapped on Ising model temperature space using a neural network thermometer.
result Flow of RBM trained on Ising spin configurations approaches critical temperature around kBTc/J≈2.269. One conjecture in both deep learning and classical connectionist viewpoint is that the biological brain implements certain kinds of deep networks as its back-end. However, to our knowledge, a detailed correspondence has not yet been set up, which is important if we want to bridge between neuroscience and machine learni…
New method uses quantum annealing and VAN for better statistical mechanics calculations.
problem Difficulty in computing partition function in statistical mechanics.
method Combines quantum annealing samples with variational autoregressive networks.
result Enhanced accuracy in finite-size Sherrington-Kirkpatrick model.
Proposes GM Score to evaluate GANs considering diversity, disentanglement, and discriminability.
problem Evaluation of GANs for sample quality and diversity.
method Integrates various factors including intra-class and inter-class diversity, disentanglement, and discriminability metrics.
result Demonstrates improved evaluation of GANs on MNIST dataset.
The inverse Potts problem to infer a Boltzmann distribution for homologous protein sequences from their single-site and pairwise amino acid frequencies recently attracts a great deal of attention in the studies of protein structure and evolution. We study regularization and learning methods and how to tune regularizati…
Bayesian optimization has become a popular method for high-throughput computing, like the design of computer experiments or hyperparameter tuning of expensive models, where sample efficiency is mandatory. In these applications, distributed and scalable architectures are a necessity. However, Bayesian optimization is mo…
Restricted Boltzmann machines (RBMs) are a powerful class of generative models, but their training requires computing a gradient that, unlike supervised backpropagation on typical loss functions, is notoriously difficult even to approximate. Here, we show that properly combining standard gradient updates with an off-gr…
The probability density function for the visible sector of a Riemann-Theta Boltzmann machine can be taken conditional on a subset of the visible units. We derive that the corresponding conditional density function is given by a reparameterization of the Riemann-Theta Boltzmann machine modelling the original probability…
Study explores D-Wave QA for RBM training, finds modest benefits but no significant improvements.
problem Improving RBM training using D-Wave quantum annealing.
method Hybrid sampling approach combining classical and D-Wave QA methods.
result No significant improvements in RBM training using D-Wave QA, but potential for CF mitigation.
We introduce a new method for training deep Boltzmann machines jointly. Prior methods require an initial learning pass that trains the deep Boltzmann machine greedily, one layer at a time, or do not perform well on classifi- cation tasks.
SBS uses SVGD to optimize continuous functions globally.
problem Global optimization of continuous Sobolev functions.
method Stein Boltzmann Sampling (SBS) with SVGD.
result SBS and its variants are highly competitive in global optimization.
RBM and DBM are represented as 2D tensor networks, revealing their expressive power and efficiency.
problem Understanding and optimizing RBM and DBM models.
method Representing RBM and DBM as 2D tensor networks and developing an efficient tensor network contraction algorithm.
result The proposed algorithm for computing partition functions is more accurate than state-of-the-art methods.