This work maps Boltzmann distributions to ARNNs for better physics-based model approximations.
problem Approximating Boltzmann distributions of binary systems.
method Exact mapping of Boltzmann distribution to autoregressive neural network architecture.
result New ARNN architectures derived from physical models show superior performance.
A new algorithm, Weighted Contrastive Divergence (WCD), improves on Contrastive Divergence (CD) for learning Boltzmann architectures.
problem Computational infeasibility of exact gradient computation in Boltzmann architectures.
method Proposes Weighted Contrastive Divergence (WCD) as a modification of Contrastive Divergence (CD) with small modifications to the negative phase.
result Experimental results show significant improvement of WCD over standard CD and persistent CD with minimal additional computational cost.
Transferable Boltzmann Generators learn to sample unseen molecules efficiently.
problem Generating equilibrium samples of molecular systems.
method Boltzmann Generators using normalizing flows to learn transformations.
result Transferable Boltzmann Generators can predict zero-shot Boltzmann distributions for unseen molecules.
A new machine learning model uses score matching to estimate probability densities efficiently.
problem Estimating probability density functions is challenging.
method Introduced a product Jacobi-Theta Boltzmann machine (pJTBM) and used score matching for efficient fitting.
result The pJTBM can fit probability densities more efficiently than the RTBM using score matching.
Restricted Boltzmann Machines (RBM) are a neural network model used in deep learning.
problem Representing probability distributions with hidden variables.
method Mathematical analysis and geometric study of RBM structures.
result Geometry of probability distributions in RBM models.
Despite their exceptional flexibility and popularity, the Monte Carlo methods often suffer from slow mixing times for challenging statistical physics problems. We present a general strategy to overcome this difficulty by adopting ideas and techniques from the machine learning community. We fit the unnormalized probabil…
The study compares classical and quantum information-based unsupervised generative models.
problem Comparing classical and quantum information-based unsupervised generative models.
method Analyzing information theoretical bounds and comparing RBM architectures on MNIST datasets.
result Quantum information-based models outperform classical models in certain scenarios.
Study evaluates quantum and classical conditional Boltzmann machines for time-series forecasting.
problem Time-series forecasting using quantum and classical conditional Boltzmann machines.
method Developed and compared four conditional energy-based forecasting architectures: Gaussian-Bernoulli CRBM, QCRBM, QQRBM, and QFeatureQRBM. Evaluated using symmetric hyperparameter optimisation.
result No systematic evidence of a quantum advantage in time-series forecasting at the available sample size.
EWFM trains continuous flows with only energy evaluations, improving sample quality with fewer computations.
problem Efficiently sampling from complex, high-dimensional Boltzmann distributions using only energy evaluations.
method Energy-Weighted Flow Matching (EWFM) using importance sampling and iterative/annealed training.
result Improved sample quality with up to 3 orders of magnitude fewer energy evaluations compared to existing methods.
Ordinal data is omnipresent in almost all multiuser-generated feedback - questionnaires, preferences etc. This paper investigates modelling of ordinal data with Gaussian restricted Boltzmann machines (RBMs). In particular, we present the model architecture, learning and inference procedures for both vector-variate and …
This paper shows RBMs can maintain strong performance even after extreme pruning, but only if done early in training.
problem The computational and environmental costs of large neural networks.
method Investigating the performance of RBMs under extreme pruning conditions, inspired by the Lottery Ticket Hypothesis.
result RBMs can achieve high-quality generative performance even after 80% pruning, but performance degrades sharply above a critical point.
Restricted Boltzmann machines (RBMs) are energy-based neural-networks which are commonly used as the building blocks for deep architectures neural architectures. In this work, we derive a deterministic framework for the training, evaluation, and use of RBMs based upon the Thouless-Anderson-Palmer (TAP) mean-field appro…
Deep networks outperform shallow ones in Ising model near criticality.
problem Comparing deep vs shallow neural networks for Ising model.
method Trained deep and shallow Boltzmann machines on Ising system data.
result Accuracy depends only on first hidden layer size, not network depth.
Online learning algorithm for RBM adapts network architecture to streaming data.
problem Adapting RBM architecture to streaming data for better feature representation and classification.
method Two-phase training: online generative phase for unsupervised feature learning, discriminative phase for classification.
result Online RBM converges to a stable, concise network architecture with inherently discriminative neurons.
Deep learning approximates system moments from data.
problem Approximating moments of spatial probabilistic systems.
method Dynamic Boltzmann Distributions (DBDs) with deep Boltzmann machines (DBMs).
result Learned moment closures improve generalization over traditional methods.
SCALLOP improves likelihood flow maps for efficient Boltzmann generation.
problem Efficient estimation of model likelihood in flow-based generative models.
method SCALLOP introduces a Hutchinson-free likelihood distillation objective for scalable flow-based models.
result SCALLOP achieves up to 10x inference speedup while improving performance.
We present a layered Boltzmann machine (BM) that can better exploit the advantages of a distributed representation. It is widely believed that deep BMs (DBMs) have far greater representational power than its shallow counterpart, restricted Boltzmann machines (RBMs). However, this expectation on the supremacy of DBMs ov…
New results connect RBMs to neural networks, improving learning efficiency.
problem Learning graphical models with latent variables is difficult.
method New connections to learning two-layer neural networks under ℓ∞ bounded input. result Improved algorithm for learning supervised RBMs.
New model enhances SPIM for solving low-rank combinatorial optimization and statistical learning problems.
problem Solving large-scale combinatorial optimization problems efficiently.
method Proposed a new computing model for SPIM that can handle low-rank interaction matrices.
result Demonstrated efficient learning, classification, and sampling of MNIST images using the model.
The restricted Boltzmann machine (RBM) is one of the fundamental building blocks of deep learning. RBM finds wide applications in dimensional reduction, feature extraction, and recommender systems via modeling the probability distributions of a variety of input data including natural images, speech signals, and custome…
We introduce Thurstonian Boltzmann Machines (TBM), a unified architecture that can naturally incorporate a wide range of data inputs at the same time. Our motivation rests in the Thurstonian view that many discrete data types can be considered as being generated from a subset of underlying latent continuous variables, …
Paper proposes efficient training for normalizing flows in Boltzmann generators.
problem Training normalizing flows for Boltzmann generators is computationally challenging and unstable.
method Regression Training of Normalizing Flows (RegFlow) using ℓ2-regression. result RegFlow enables efficient and stable training of normalizing flows for Boltzmann generators.
Hardware-accelerated RBM solves large combinatorial problems and integer factorization.
problem Solving large combinatorial optimization and integer factorization problems.
method Logically synthesized RBM architecture, hardware acceleration, and efficient training methods.
result Hardware-accelerated RBM factorizes 16-bit numbers with 10000x speed and 32x power improvements.
Conditional probabilities modeled using Riemann-Theta Boltzmann Machines.
problem Modeling conditional probabilities in Boltzmann machines.
method Deriving conditional density functions from Riemann-Theta Boltzmann machines.
result Conditional densities can be directly inferred from Riemann-Theta Boltzmann machines.
CRBMs improve financial regime detection with PCD and free energy analysis.
problem Detecting systemic risk regimes in financial time series.
method Extended RBM to CRBM with autoregressive conditioning and PCD. Decomposed free energy into magnitude and correlation components.
result CRBM's free energy metric distinguishes between magnitude shocks and market regimes.
TBMs learn Gibbs distribution adaptively from data.
problem Combinatorial explosion in Boltzmann machines.
method Adaptive construction of minimum required sample space.
result TBMs outperform other Boltzmann machines in efficiency and effectiveness.
D-Wave computers struggle with sampling Boltzmann distributions efficiently.
problem Sampling Boltzmann distributions efficiently on D-Wave computers.
method Exploring various obstacles and remaining difficulties.
result Challenges remain in using D-Wave computers for efficient sampling.
Quantum Boltzmann Machines trained on quantum annealers produce noisy synthetic data.
problem Training quantum Boltzmann machines on quantum annealers for financial data generation.
method Used D-Wave Advantage 4.1 quantum annealer to train QBMs and compare with classical RBMs.
result Quantum Boltzmann Machines trained on quantum annealers are noisier and less effective than classical RBMs.
Improved discrete VAEs using relaxed Boltzmann priors for better performance.
problem Training discrete VAEs with tighter importance-weighted bounds.
method Two approaches for relaxing Boltzmann machines to continuous distributions, based on generalized overlapping transformations and the Gaussian integral trick.
result These relaxations outperform previous discrete VAEs with Boltzmann priors on MNIST and OMNIGLOT datasets.
We introduce a new method for training deep Boltzmann machines jointly. Prior methods require an initial learning pass that trains the deep Boltzmann machine greedily, one layer at a time, or do not perform well on classifi- cation tasks.
RBM and DBM are represented as 2D tensor networks, revealing their expressive power and efficiency.
problem Understanding and optimizing RBM and DBM models.
method Representing RBM and DBM as 2D tensor networks and developing an efficient tensor network contraction algorithm.
result The proposed algorithm for computing partition functions is more accurate than state-of-the-art methods.
Stochastic gradient descent based algorithms are typically used as the general optimization tools for most deep learning models. A Restricted Boltzmann Machine (RBM) is a probabilistic generative model that can be stacked to construct deep architectures. For RBM with Bernoulli inputs, non-Euclidean algorithm such as st…
A new method speeds up sampling of Boltzmann distribution in high-dimensional systems.
problem High computational cost of obtaining Jacobian of flow-based models in high dimensions.
method Flow perturbation method that incorporates stochastic perturbations and reweighting.
result Achieves unbiased sampling of Boltzmann distribution with orders of magnitude speedup.
We show that deep narrow Boltzmann machines are universal approximators of probability distributions on the activities of their visible units, provided they have sufficiently many hidden layers, each containing the same number of units as the visible layer. We show that, within certain parameter domains, deep Boltzmann…
Neural Boltzmann Machines improve on CRBMs for modeling data.
problem Limited expressivity of CRBMs with noisy data.
method Convert CRBM parameters to neural networks.
result NBMs can approximate data likelihood better.
This paper provides a tutorial on Boltzmann Machines and Deep Belief Networks.
problem Understanding and applying Boltzmann Machines and Deep Belief Networks.
method Explains the structures, conditional distributions, Gibbs sampling, training methods, and deep belief networks of RBMs.
result Comprehensive overview of RBMs and DBNs, useful in various fields.
New method trains Boltzmann machines without supervision.
problem Training unsupervised learning models.
method Mixed binary quadratic feasibility problem formulation.
result Theory validated on XOR patterns.
CAP-BM learns complex-valued data's amplitude and phase distributions.
problem Learning from complex-valued data with amplitude variation.
method Complex Amplitude-Phase Boltzmann machine (CAP-BM) with Gibbs sampling.
result Necessity of amplitude-amplitude coupling term in CAP-BM.
In 1870s, L. Boltzmann proved the famous H-theorem for the Boltzmann equation in the kinetic theory of gas and gave the statistical interpretation of the thermodynamic entropy. In 2002, G. Perelman introduced the notion of W-entropy and proved the W-entropy formula for the Ricci flow. This plays a crucial role in…
We propose a relaxation-based approximate inference algorithm that samples near-MAP configurations of a binary pairwise Markov random field. We experiment on MAP inference tasks in several restricted Boltzmann machines. We also use our underlying sampler to estimate the log-partition function of restricted Boltzmann ma…
Neural-Network Quantum States connect to Tensor-Network states, enhancing quantum state representation.
problem Describing complex quantum wave functions efficiently.
method Introducing Neural-Network Quantum States and showing their connections to Tensor-Network states.
result Neural-Network Quantum States and String-Bond States can approximate chiral topological states with better accuracy.
A restricted Boltzmann machine (RBM) is an undirected graphical model constructed for discrete or continuous random variables, with two layers, one hidden and one visible, and no conditional dependency within a layer. In recent years, RBMs have risen to prominence due to their connection to deep learning. By treating a…
Reduced hidden units in RBM without performance loss.
problem Maintaining performance with fewer hidden units in RBM.
method Proposed algorithm to decrease hidden units while keeping performance constant.
result Demonstrated through numerical simulations that fewer hidden units can be used without performance degradation.
Boltzmann machines are physics informed generative models with wide applications in machine learning. They can learn the probability distribution from an input dataset and generate new samples accordingly. Applying them back to physics, the Boltzmann machines are ideal recommender systems to accelerate Monte Carlo simu…
Introduces a Boltzmann machine with Riemann-Theta functions for continuous and discrete states.
problem Modeling continuous and discrete states in neural networks.
method Develops a Boltzmann machine with continuous visible and discrete hidden states, solving probability density and conditional expectation analytically.
result Derives a novel parametric density function involving Riemann-Theta functions and uses it as an activation function in a feedforward neural network.
Paper improves molecular property prediction using denoising autoencoders.
problem Limited data for molecular property prediction from 3D structures.
method Pre-training via denoising for learning molecular force fields.
result Achieves new state-of-the-art performance on QM9 dataset.
A new sampler and temperature estimation method enable efficient learning of Boltzmann Machines.
problem Efficient learning of Boltzmann Machines (BMs) is challenging due to high training costs and difficulty in parallelization.
method Proposed a new Boltzmann sampler (Langevin SB, LSB) and an efficient method (Conditional Expectation Matching, CEM) for estimating inverse temperature.
result Established an efficient learning framework (Sampler-Adaptive Learning, SAL) for BMs with greater expressive power than Restricted Boltzmann Machines (RBMs).
Lean 2-layer RBMs achieve similar representational power as single-layer RBMs with fewer parameters.
problem Understanding and quantifying the representational power of multi-layer RBMs.
method Inherent Structure Capacity (ISC) and Lean RBMs.
result 2-layer RBMs can achieve the same representational power as single-layer RBMs with fewer parameters.