Reduced hidden units in RBM without performance loss.
problem Maintaining performance with fewer hidden units in RBM.
method Proposed algorithm to decrease hidden units while keeping performance constant.
result Demonstrated through numerical simulations that fewer hidden units can be used without performance degradation.
Conditional probabilities modeled using Riemann-Theta Boltzmann Machines.
problem Modeling conditional probabilities in Boltzmann machines.
method Deriving conditional density functions from Riemann-Theta Boltzmann machines.
result Conditional densities can be directly inferred from Riemann-Theta Boltzmann machines.
A new method estimates protein evolutionary fields and couplings from alignments.
problem Estimating evolutionary fields and couplings from protein sequence alignments.
method Boltzmann machine with parallel, persistent Markov chain Monte Carlo method.
result Improved precision in predicting contact residue pairs.
Structural RBM reduces parameters for image denoising and classification.
problem High parameter count in RBMs limits their applicability to large datasets.
method Introduces SRBM with constrained connections to reduce parameters.
result SRBM achieves better performance and faster training than vanilla RBM.
TBMs learn Gibbs distribution adaptively from data.
problem Combinatorial explosion in Boltzmann machines.
method Adaptive construction of minimum required sample space.
result TBMs outperform other Boltzmann machines in efficiency and effectiveness.
Boltzmann machines can discover new cluster Monte Carlo algorithms.
problem Discovering new cluster Monte Carlo algorithms for complex systems.
method Using Boltzmann machines to learn and generate samples from physical systems, identifying clusters.
result Boltzmann machines can discover unknown cluster Monte Carlo algorithms.
RBM and DBM are represented as 2D tensor networks, revealing their expressive power and efficiency.
problem Understanding and optimizing RBM and DBM models.
method Representing RBM and DBM as 2D tensor networks and developing an efficient tensor network contraction algorithm.
result The proposed algorithm for computing partition functions is more accurate than state-of-the-art methods.
A new machine learning model uses score matching to estimate probability densities efficiently.
problem Estimating probability density functions is challenging.
method Introduced a product Jacobi-Theta Boltzmann machine (pJTBM) and used score matching for efficient fitting.
result The pJTBM can fit probability densities more efficiently than the RTBM using score matching.
We introduce a new method for training deep Boltzmann machines jointly. Prior methods require an initial learning pass that trains the deep Boltzmann machine greedily, one layer at a time, or do not perform well on classifi- cation tasks.
Restricted Boltzmann Machines (RBM) are a neural network model used in deep learning.
problem Representing probability distributions with hidden variables.
method Mathematical analysis and geometric study of RBM structures.
result Geometry of probability distributions in RBM models.
Prior distributions of binarized natural images are learned by using a Boltzmann machine. According the results of this study, there emerges a structure with two sublattices in the interactions, and the nearest-neighbor and next-nearest-neighbor interactions correspondingly take two discriminative values, which reflect…
Proposes EM-like method to train Boltzmann machines with unconstrained connectivity.
problem Training Boltzmann machines with complex connectivity.
method Uses Monte Carlo approximation in E-step and approximates M-step gradient in M-step.
result EM-like method can be equivalent to contrastive divergence in restricted Boltzmann machines.
We show that deep narrow Boltzmann machines are universal approximators of probability distributions on the activities of their visible units, provided they have sufficiently many hidden layers, each containing the same number of units as the visible layer. We show that, within certain parameter domains, deep Boltzmann…
D-Wave computers struggle with sampling Boltzmann distributions efficiently.
problem Sampling Boltzmann distributions efficiently on D-Wave computers.
method Exploring various obstacles and remaining difficulties.
result Challenges remain in using D-Wave computers for efficient sampling.
Neural Boltzmann Machines improve on CRBMs for modeling data.
problem Limited expressivity of CRBMs with noisy data.
method Convert CRBM parameters to neural networks.
result NBMs can approximate data likelihood better.
Quantum Boltzmann Machines trained on quantum annealers produce noisy synthetic data.
problem Training quantum Boltzmann machines on quantum annealers for financial data generation.
method Used D-Wave Advantage 4.1 quantum annealer to train QBMs and compare with classical RBMs.
result Quantum Boltzmann Machines trained on quantum annealers are noisier and less effective than classical RBMs.
This paper provides a tutorial on Boltzmann Machines and Deep Belief Networks.
problem Understanding and applying Boltzmann Machines and Deep Belief Networks.
method Explains the structures, conditional distributions, Gibbs sampling, training methods, and deep belief networks of RBMs.
result Comprehensive overview of RBMs and DBNs, useful in various fields.
New method trains Boltzmann machines without supervision.
problem Training unsupervised learning models.
method Mixed binary quadratic feasibility problem formulation.
result Theory validated on XOR patterns.
CAP-BM learns complex-valued data's amplitude and phase distributions.
problem Learning from complex-valued data with amplitude variation.
method Complex Amplitude-Phase Boltzmann machine (CAP-BM) with Gibbs sampling.
result Necessity of amplitude-amplitude coupling term in CAP-BM.
We propose a relaxation-based approximate inference algorithm that samples near-MAP configurations of a binary pairwise Markov random field. We experiment on MAP inference tasks in several restricted Boltzmann machines. We also use our underlying sampler to estimate the log-partition function of restricted Boltzmann ma…
Improved discrete VAEs using relaxed Boltzmann priors for better performance.
problem Training discrete VAEs with tighter importance-weighted bounds.
method Two approaches for relaxing Boltzmann machines to continuous distributions, based on generalized overlapping transformations and the Gaussian integral trick.
result These relaxations outperform previous discrete VAEs with Boltzmann priors on MNIST and OMNIGLOT datasets.
Enhanced factored RBMs improve speech detection in noisy conditions.
problem Improving speech detection accuracy in noisy environments.
method Proposes EFTW-RBMs with conditional feature learning and low rank approximation.
result Outperforms existing 1D and 2D speech detection algorithms in various noisy conditions.
New method uses Boltzmann machines for compressed sensing of sparse signals without known correlation model.
problem Reconstructing sparse signals from limited measurements without prior correlation knowledge.
method Train a generative Boltzmann machine to infer signal structure, then use message-passing inference for reconstruction.
result Effective reconstruction even with fewer measurements than signal sparsity, as demonstrated on MNIST.
Introduces a Boltzmann machine with Riemann-Theta functions for continuous and discrete states.
problem Modeling continuous and discrete states in neural networks.
method Develops a Boltzmann machine with continuous visible and discrete hidden states, solving probability density and conditional expectation analytically.
result Derives a novel parametric density function involving Riemann-Theta functions and uses it as an activation function in a feedforward neural network.
A Gaussian restricted Boltzmann machine (GRBM) is a Boltzmann machine defined on a bipartite graph and is an extension of usual restricted Boltzmann machines. A GRBM consists of two different layers: a visible layer composed of continuous visible variables and a hidden layer composed of discrete hidden variables. In th…
RBMs model binary interactions with hidden node activation effects.
problem Understanding how RBM hidden node activation affects binary variable distributions.
method Investigated RBM marginal distributions with different hidden node activation functions.
result Found exact expressions for RBM marginals as interacting binary variables.
Graph clustering improved using Boltzmann machine heuristics.
problem Graph clustering to form densely connected clusters.
method Two mathematical programming formulations, two variations of Boltzmann machine heuristic.
result Boltzmann machine provides superior solutions and faster computation times.
AIS method improves estimation of RBM partition function with reduced computational cost.
problem Efficiently estimating partition function of RBMs for large systems.
method Annealed Importance Sampling (AIS) with optimized initialization.
result Good estimation of partition function Z with reduced computational cost.
CG-BGs combine flow-based models with PMFs to sample large systems efficiently.
problem Sampling equilibrium molecular configurations from the Boltzmann distribution is challenging.
method Coarse-grained Boltzmann Generators (CG-BGs) use flow-based models and learned PMFs for efficient sampling.
result CG-BGs provide a practical route for sampling larger molecular systems efficiently.
The Riemann-Theta Boltzmann machine's visible sector is sampled using a discrete multi-variate Gaussian.
problem Sampling the visible sector of the Riemann-Theta Boltzmann machine.
method Discrete multi-variate Gaussian over the hidden state space.
result The visible sector probability density function is an infinite mixture of multi-variate Gaussians.
Proposes ML-RBM for non-intrusive load monitoring without appliance-level data.
problem Non-intrusive load monitoring without appliance-level data.
method Multi-label Restricted Boltzmann Machine (ML-RBM)
result Experimental evaluation of proposed and state-of-the-art techniques.
We introduce Thurstonian Boltzmann Machines (TBM), a unified architecture that can naturally incorporate a wide range of data inputs at the same time. Our motivation rests in the Thurstonian view that many discrete data types can be considered as being generated from a subset of underlying latent continuous variables, …
Since learning is typically very slow in Boltzmann machines, there is a need to restrict connections within hidden layers. However, the resulting states of hidden units exhibit statistical dependencies. Based on this observation, we propose using l1/l2 regularization upon the activation possibilities of hidden unit…
Empirical Bayes method for Boltzmann machines avoids computational hardness.
problem Estimating hyperparameters of Boltzmann machines with intractable integrations.
method Replica method and Plefka expansion to avoid integrations.
result Simple and fast algorithm with a bias in estimates.
New algorithm reduces autocorrelation in HMC for lattice field theories.
problem Reduction of autocorrelation in HMC for lattice field theories.
method Hybrid Monte-Carlo algorithm with restricted Boltzmann machine.
result Reduction of autocorrelation in both symmetric and broken phases.
Convolutional-Restricted-Boltzmann-Machine learns relational order among time-related inputs.
problem Learning optimal relational order among multiple time-related inputs.
method Extended Convolutional-Restricted-Boltzmann-Machine with multiplicative units and reinforcement learning.
result The machine can learn the optimal relational order among inputs.
BEAMs use adversarial training to improve RBM performance.
problem Likelihood-based training fails to penalize high-probability regions.
method Adversarial training against RBM hidden layer activations.
result BEAMs outperform RBMs and GANs on benchmarks.
Improved Monte Carlo simulations using RBMs for phase transitions.
problem Slow mixing times in Monte Carlo simulations for complex systems.
method Fit unnormalized probability to a restricted Boltzmann machine and use its feature detection ability for efficient updates.
result Improved acceptance ratio and autocorrelation time near phase transition points.
Learning in restricted Boltzmann machine is typically hard due to the computation of gradients of log-likelihood function. To describe the network state statistics of the restricted Boltzmann machine, we develop an advanced mean field theory based on the Bethe approximation. Our theory provides an efficient message pas…
Study proposes an RBM with multivalued hidden variables to improve generalization.
problem Improving generalization in RBMs to prevent overfitting.
method Introduced an RBM with multivalued hidden variables as a simple extension of conventional RBMs.
result The proposed model outperforms conventional RBMs in contrastive divergence learning and MNIST classification.
A new sampler and temperature estimation method enable efficient learning of Boltzmann Machines.
problem Efficient learning of Boltzmann Machines (BMs) is challenging due to high training costs and difficulty in parallelization.
method Proposed a new Boltzmann sampler (Langevin SB, LSB) and an efficient method (Conditional Expectation Matching, CEM) for estimating inverse temperature.
result Established an efficient learning framework (Sampler-Adaptive Learning, SAL) for BMs with greater expressive power than Restricted Boltzmann Machines (RBMs).
Deep Boltzmann machines are in principle powerful models for extracting the hierarchical structure of data. Unfortunately, attempts to train layers jointly (without greedy layer-wise pretraining) have been largely unsuccessful. We propose a modification of the learning algorithm that initially recenters the output of t…
Paper proposes an effective mean-field inference method for NNBMs.
problem Inference in NNBMs is challenging due to their complex structure.
method Uses mean-field method and diagonal consistency method.
result Effective inference method for NNBMs is proposed.
We present a new statistical learning paradigm for Boltzmann machines based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes maximum entropy principle and from standard maximum likelihood estimation.We demonstrate the LME principle BY deriving …
The study compares classical and quantum information-based unsupervised generative models.
problem Comparing classical and quantum information-based unsupervised generative models.
method Analyzing information theoretical bounds and comparing RBM architectures on MNIST datasets.
result Quantum information-based models outperform classical models in certain scenarios.
We describe discrete restricted Boltzmann machines: probabilistic graphical models with bipartite interactions between visible and hidden discrete variables. Examples are binary restricted Boltzmann machines and discrete naive Bayes models. We detail the inference functions and distributed representations arising in th…
A novel quantum model improves RBM performance and is efficiently trainable.
problem Improving the performance of RBM models.
method Quantum model with parametrically coupled fermions to classical signals.
result The model outperforms classical RBM with the same number of hidden units.
Algorithm uses RBM to solve matching problems on weighted graphs.
problem Perfect matching problem on bipartite weighted graphs.
method Iterative RBM algorithm to maximize energy function and assignment.
result Algorithm successfully solves real-world matching problems.