The paper develops a new algorithm for RBMs using dynamical mean-field theory.
problem Learning in Restricted Boltzmann Machines (RBMs) with complex dependencies.
method Dynamical mean-field theory applied to RBMs with rectangular coupling matrices drawn from a bi-rotation invariant ensemble.
result The algorithm converges globally under a stability criterion, with rates matching numerical simulations.
Enhances DyBM for better financial time-series prediction.
problem Limitations of Gaussian DyBM in financial applications.
method Extends DyBM to handle second-order moments and generalized Gaussian distributions.
result Significant performance improvement in predicting financial time-series data.
Deep learning approximates system moments from data.
problem Approximating moments of spatial probabilistic systems.
method Dynamic Boltzmann Distributions (DBDs) with deep Boltzmann machines (DBMs).
result Learned moment closures improve generalization over traditional methods.
We propose a particularly structured Boltzmann machine, which we refer to as a dynamic Boltzmann machine (DyBM), as a stochastic model of a multi-dimensional time-series. The DyBM can have infinitely many layers of units but allows exact and efficient inference and learning when its parameters have a proposed structure…
Conditional probabilities modeled using Riemann-Theta Boltzmann Machines.
problem Modeling conditional probabilities in Boltzmann machines.
method Deriving conditional density functions from Riemann-Theta Boltzmann machines.
result Conditional densities can be directly inferred from Riemann-Theta Boltzmann machines.
We introduce Delay Pruning, a simple yet powerful technique to regularize dynamic Boltzmann machines (DyBM). The recently introduced DyBM provides a particularly structured Boltzmann machine, as a generative model of a multi-dimensional time-series. This Boltzmann machine can have infinitely many layers of units but al…
TBMs learn Gibbs distribution adaptively from data.
problem Combinatorial explosion in Boltzmann machines.
method Adaptive construction of minimum required sample space.
result TBMs outperform other Boltzmann machines in efficiency and effectiveness.
Proposes p-RBM for retaining memory in RBM models for stock market prediction.
problem Lack of memory in RBM models for dynamic data like time-series analysis.
method Generalizes RBM to p-RBM, capable of retaining p past states, and trains using contrastive divergence.
result p-RBM offers promising prediction potential for stock market direction.
RBM and DBM are represented as 2D tensor networks, revealing their expressive power and efficiency.
problem Understanding and optimizing RBM and DBM models.
method Representing RBM and DBM as 2D tensor networks and developing an efficient tensor network contraction algorithm.
result The proposed algorithm for computing partition functions is more accurate than state-of-the-art methods.
A new machine learning model uses score matching to estimate probability densities efficiently.
problem Estimating probability density functions is challenging.
method Introduced a product Jacobi-Theta Boltzmann machine (pJTBM) and used score matching for efficient fitting.
result The pJTBM can fit probability densities more efficiently than the RTBM using score matching.
We introduce a new method for training deep Boltzmann machines jointly. Prior methods require an initial learning pass that trains the deep Boltzmann machine greedily, one layer at a time, or do not perform well on classifi- cation tasks.
Restricted Boltzmann Machines (RBM) are a neural network model used in deep learning.
problem Representing probability distributions with hidden variables.
method Mathematical analysis and geometric study of RBM structures.
result Geometry of probability distributions in RBM models.
We show that deep narrow Boltzmann machines are universal approximators of probability distributions on the activities of their visible units, provided they have sufficiently many hidden layers, each containing the same number of units as the visible layer. We show that, within certain parameter domains, deep Boltzmann…
D-Wave computers struggle with sampling Boltzmann distributions efficiently.
problem Sampling Boltzmann distributions efficiently on D-Wave computers.
method Exploring various obstacles and remaining difficulties.
result Challenges remain in using D-Wave computers for efficient sampling.
Boltzmann machines are physics informed generative models with wide applications in machine learning. They can learn the probability distribution from an input dataset and generate new samples accordingly. Applying them back to physics, the Boltzmann machines are ideal recommender systems to accelerate Monte Carlo simu…
Neural Boltzmann Machines improve on CRBMs for modeling data.
problem Limited expressivity of CRBMs with noisy data.
method Convert CRBM parameters to neural networks.
result NBMs can approximate data likelihood better.
Quantum Boltzmann Machines trained on quantum annealers produce noisy synthetic data.
problem Training quantum Boltzmann machines on quantum annealers for financial data generation.
method Used D-Wave Advantage 4.1 quantum annealer to train QBMs and compare with classical RBMs.
result Quantum Boltzmann Machines trained on quantum annealers are noisier and less effective than classical RBMs.
This paper provides a tutorial on Boltzmann Machines and Deep Belief Networks.
problem Understanding and applying Boltzmann Machines and Deep Belief Networks.
method Explains the structures, conditional distributions, Gibbs sampling, training methods, and deep belief networks of RBMs.
result Comprehensive overview of RBMs and DBNs, useful in various fields.
New method trains Boltzmann machines without supervision.
problem Training unsupervised learning models.
method Mixed binary quadratic feasibility problem formulation.
result Theory validated on XOR patterns.
CAP-BM learns complex-valued data's amplitude and phase distributions.
problem Learning from complex-valued data with amplitude variation.
method Complex Amplitude-Phase Boltzmann machine (CAP-BM) with Gibbs sampling.
result Necessity of amplitude-amplitude coupling term in CAP-BM.
We propose a relaxation-based approximate inference algorithm that samples near-MAP configurations of a binary pairwise Markov random field. We experiment on MAP inference tasks in several restricted Boltzmann machines. We also use our underlying sampler to estimate the log-partition function of restricted Boltzmann ma…
Improved discrete VAEs using relaxed Boltzmann priors for better performance.
problem Training discrete VAEs with tighter importance-weighted bounds.
method Two approaches for relaxing Boltzmann machines to continuous distributions, based on generalized overlapping transformations and the Gaussian integral trick.
result These relaxations outperform previous discrete VAEs with Boltzmann priors on MNIST and OMNIGLOT datasets.
Reduced hidden units in RBM without performance loss.
problem Maintaining performance with fewer hidden units in RBM.
method Proposed algorithm to decrease hidden units while keeping performance constant.
result Demonstrated through numerical simulations that fewer hidden units can be used without performance degradation.
Introduces a Boltzmann machine with Riemann-Theta functions for continuous and discrete states.
problem Modeling continuous and discrete states in neural networks.
method Develops a Boltzmann machine with continuous visible and discrete hidden states, solving probability density and conditional expectation analytically.
result Derives a novel parametric density function involving Riemann-Theta functions and uses it as an activation function in a feedforward neural network.
A Gaussian restricted Boltzmann machine (GRBM) is a Boltzmann machine defined on a bipartite graph and is an extension of usual restricted Boltzmann machines. A GRBM consists of two different layers: a visible layer composed of continuous visible variables and a hidden layer composed of discrete hidden variables. In th…
Graph clustering improved using Boltzmann machine heuristics.
problem Graph clustering to form densely connected clusters.
method Two mathematical programming formulations, two variations of Boltzmann machine heuristic.
result Boltzmann machine provides superior solutions and faster computation times.
The Riemann-Theta Boltzmann machine's visible sector is sampled using a discrete multi-variate Gaussian.
problem Sampling the visible sector of the Riemann-Theta Boltzmann machine.
method Discrete multi-variate Gaussian over the hidden state space.
result The visible sector probability density function is an infinite mixture of multi-variate Gaussians.
Paper explores Monge-Ampère in deep learning and quantum geometry.
problem Understanding the Monge-Ampère equation in deep learning.
method Review of Boltzmann learning, connection to optimal transport, insights from quantum geometry, renormalization group flow.
result Space of covariance matrices in learning dynamics coincides with the CAH cone.
Since learning is typically very slow in Boltzmann machines, there is a need to restrict connections within hidden layers. However, the resulting states of hidden units exhibit statistical dependencies. Based on this observation, we propose using l1/l2 regularization upon the activation possibilities of hidden unit…
Empirical Bayes method for Boltzmann machines avoids computational hardness.
problem Estimating hyperparameters of Boltzmann machines with intractable integrations.
method Replica method and Plefka expansion to avoid integrations.
result Simple and fast algorithm with a bias in estimates.
We propose an expectation-maximization-like(EMlike) method to train Boltzmann machine with unconstrained connectivity. It adopts Monte Carlo approximation in the E-step, and replaces the intractable likelihood objective with efficiently computed objectives or directly approximates the gradient of likelihood objective i…
BEAMs use adversarial training to improve RBM performance.
problem Likelihood-based training fails to penalize high-probability regions.
method Adversarial training against RBM hidden layer activations.
result BEAMs outperform RBMs and GANs on benchmarks.
New method combines deep learning and quantum mechanics for efficient molecular statistics.
problem Computational expense in extracting statistics from molecular systems.
method Adaptive Markov chain Monte Carlo with Normalizing Flow and MLP for quantum accuracy.
result Rapid convergence to Boltzmann distribution and accurate thermodynamic observables.
Learning in restricted Boltzmann machine is typically hard due to the computation of gradients of log-likelihood function. To describe the network state statistics of the restricted Boltzmann machine, we develop an advanced mean field theory based on the Bethe approximation. Our theory provides an efficient message pas…
Study proposes an RBM with multivalued hidden variables to improve generalization.
problem Improving generalization in RBMs to prevent overfitting.
method Introduced an RBM with multivalued hidden variables as a simple extension of conventional RBMs.
result The proposed model outperforms conventional RBMs in contrastive divergence learning and MNIST classification.
A new sampler and temperature estimation method enable efficient learning of Boltzmann Machines.
problem Efficient learning of Boltzmann Machines (BMs) is challenging due to high training costs and difficulty in parallelization.
method Proposed a new Boltzmann sampler (Langevin SB, LSB) and an efficient method (Conditional Expectation Matching, CEM) for estimating inverse temperature.
result Established an efficient learning framework (Sampler-Adaptive Learning, SAL) for BMs with greater expressive power than Restricted Boltzmann Machines (RBMs).
We study historical dynamics of joint equilibrium distribution of stock returns in the U.S. stock market using the Boltzmann distribution model being parametrized by external fields and pairwise couplings. Within Boltzmann learning framework for statistical inference, we analyze historical behavior of the parameters in…
DBS improves convergence in reinforcement learning.
problem Softmax operator convergence issues in reinforcement learning.
method Dynamic Boltzmann Softmax (DBS) updates value function.
result DBS enables better value function estimation and convergence.
Deep Boltzmann machines are in principle powerful models for extracting the hierarchical structure of data. Unfortunately, attempts to train layers jointly (without greedy layer-wise pretraining) have been largely unsuccessful. We propose a modification of the learning algorithm that initially recenters the output of t…
This paper uses RBM to calculate conditional probabilities for nonlinear system identification.
problem Challenges in obtaining probability distributions for nonlinear system identification.
method Modified RBM to calculate joint, input, and conditional probabilities.
result The method outperforms other black-box models in noisy, complex systems.
We present a new statistical learning paradigm for Boltzmann machines based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes maximum entropy principle and from standard maximum likelihood estimation.We demonstrate the LME principle BY deriving …
BNEM improves Boltzmann sampler efficiency.
problem Generating IID samples from Boltzmann distributions efficiently.
method Bootstrapped Noised Energy Matching (NEM) combined with diffusion-based learning and bootstrapping.
result BNEM achieves state-of-the-art performance with improved robustness.
The Unlearning algorithm improves neural network performance in memory tasks.
problem Improving neural network performance in memory tasks.
method Simplified Unlearning algorithm, structured training data, and new regularization technique for Boltzmann Machines.
result The Unlearning rule outperforms traditional Boltzmann-Machine learning in learning hidden probability distributions.
The study compares classical and quantum information-based unsupervised generative models.
problem Comparing classical and quantum information-based unsupervised generative models.
method Analyzing information theoretical bounds and comparing RBM architectures on MNIST datasets.
result Quantum information-based models outperform classical models in certain scenarios.
We describe discrete restricted Boltzmann machines: probabilistic graphical models with bipartite interactions between visible and hidden discrete variables. Examples are binary restricted Boltzmann machines and discrete naive Bayes models. We detail the inference functions and distributed representations arising in th…
A novel quantum model improves RBM performance and is efficiently trainable.
problem Improving the performance of RBM models.
method Quantum model with parametrically coupled fermions to classical signals.
result The model outperforms classical RBM with the same number of hidden units.
Algorithm uses RBM to solve matching problems on weighted graphs.
problem Perfect matching problem on bipartite weighted graphs.
method Iterative RBM algorithm to maximize energy function and assignment.
result Algorithm successfully solves real-world matching problems.
We introduce a Deep Boltzmann Machine model suitable for modeling and extracting latent semantic representations from a large unstructured collection of documents. We overcome the apparent difficulty of training a DBM with judicious parameter tying. This parameter tying enables an efficient pretraining algorithm and a …