arXiv research

Value function estimation is an important task in reinforcement learning, i.e., prediction. The Boltzmann softmax operator is a natural value estimator and can provide several benefits. However, it does not satisfy the non-expansion property, and its direct use may fail to converge even in value iteration. In this pape…

New geometric perspective for optimal learning on hexagonal structures.

DO-EM framework for quantum models improves generative tasks.

A softmax operator applied to a set of values acts somewhat like the maximization function and somewhat like an average. In sequential decision making, softmax is often used in settings where it is necessary to maximize utility but also to hedge against problems that arise from putting all of one's weight behind a sing…

New method models aptamer libraries as Boltzmann-weighted graph ensembles for better affinity predictions.

The sensitivity to risk that most people (hence, financial operators) feel affects the dynamics of financial transactions. Here we present an approach to this problem based on a current generalization of Boltzmann-Gibbs statistical mechanics.

The paper proposes a method to sample quantum field configurations using neural operators and flows.

Transferable Boltzmann Generators learn to sample unseen molecules efficiently.

Extracting automatically the complex set of features composing real high-dimensional data is crucial for achieving high performance in machine--learning tasks. Restricted Boltzmann Machines (RBM) are empirically known to be efficient for this purpose, and to be able to generate distributed and graded representations of…

The restricted Boltzmann machine (RBM) is a flexible tool for modeling complex data, however there have been significant computational difficulties in using RBMs to model high-dimensional multinomial observations. In natural language processing applications, words are naturally modeled by K-ary discrete distributions, …

Proposes an alternative method to train RBMs with binary synapses using Bayesian learning rule.

D-Wave computers struggle with sampling Boltzmann distributions efficiently.

We present transductive Boltzmann machines (TBMs), which firstly achieve transductive learning of the Gibbs distribution. While exact learning of the Gibbs distribution is impossible by the family of existing Boltzmann machines due to combinatorial explosion of the sample space, TBMs overcome the problem by adaptively …

In this note we determine the first two derivatives of the classical Boltzmann-Shannon entropy of the conjugate heat equation on general evolving manifolds. Based on the second derivative of the Boltzmann-Shannon entropy, we construct Perelman's F and W entropy in abstract geometric flows. Monotonicity of the entropies…

Quantum Boltzmann Machines trained on quantum annealers produce noisy synthetic data.

The probability density function for the visible sector of a Riemann-Theta Boltzmann machine can be taken conditional on a subset of the visible units. We derive that the corresponding conditional density function is given by a reparameterization of the Riemann-Theta Boltzmann machine modelling the original probability…

We introduce a new method for training deep Boltzmann machines jointly. Prior methods require an initial learning pass that trains the deep Boltzmann machine greedily, one layer at a time, or do not perform well on classifi- cation tasks.

RBM and DBM are represented as 2D tensor networks, revealing their expressive power and efficiency.

A new method speeds up sampling of Boltzmann distribution in high-dimensional systems.

We show that deep narrow Boltzmann machines are universal approximators of probability distributions on the activities of their visible units, provided they have sufficiently many hidden layers, each containing the same number of units as the visible layer. We show that, within certain parameter domains, deep Boltzmann…

A new machine learning model uses score matching to estimate probability densities efficiently.

Neural Boltzmann Machines improve on CRBMs for modeling data.

This paper provides a tutorial on Boltzmann Machines and Deep Belief Networks.

We propose a relaxation-based approximate inference algorithm that samples near-MAP configurations of a binary pairwise Markov random field. We experiment on MAP inference tasks in several restricted Boltzmann machines. We also use our underlying sampler to estimate the log-partition function of restricted Boltzmann ma…

CAP-BM learns complex-valued data's amplitude and phase distributions.

In 1870s, L. Boltzmann proved the famous $H$-theorem for the Boltzmann equation in the kinetic theory of gas and gave the statistical interpretation of the thermodynamic entropy. In 2002, G. Perelman introduced the notion of $W$-entropy and proved the $W$-entropy formula for the Ricci flow. This plays a crucial role in…

Boltzmann machines are physics informed generative models with wide applications in machine learning. They can learn the probability distribution from an input dataset and generate new samples accordingly. Applying them back to physics, the Boltzmann machines are ideal recommender systems to accelerate Monte Carlo simu…

Boltzmann machines are powerful distributions that have been shown to be an effective prior over binary latent variables in variational autoencoders (VAEs). However, previous methods for training discrete VAEs have used the evidence lower bound and not the tighter importance-weighted bound. We propose two approaches fo…

A new sampler and temperature estimation method enable efficient learning of Boltzmann Machines.

A Gaussian restricted Boltzmann machine (GRBM) is a Boltzmann machine defined on a bipartite graph and is an extension of usual restricted Boltzmann machines. A GRBM consists of two different layers: a visible layer composed of continuous visible variables and a hidden layer composed of discrete hidden variables. In th…

EDG generates Boltzmann samples from latent variables efficiently.

BNEM improves Boltzmann sampler efficiency.

Inverse problem solved for relativistic Boltzmann equation on spacetime.

Graph clustering improved using Boltzmann machine heuristics.

Since learning is typically very slow in Boltzmann machines, there is a need to restrict connections within hidden layers. However, the resulting states of hidden units exhibit statistical dependencies. Based on this observation, we propose using $l_1/l_2$ regularization upon the activation possibilities of hidden unit…

We propose an expectation-maximization-like(EMlike) method to train Boltzmann machine with unconstrained connectivity. It adopts Monte Carlo approximation in the E-step, and replaces the intractable likelihood objective with efficiently computed objectives or directly approximates the gradient of likelihood objective i…

Statistical models of economic distributions lead to Boltzmann distributions rather than a Pareto power law. This result is supported by two facts: 1. the distributions of income, car sales, marriages or jobs are a matter of chances and luck and not of reason! 2. Data for property, automobile sales, marriages and job m…

In this paper, we build on the biquasiles and dual graph diagrams introduced in arXiv:1610.06969. We introduce \textit{biquasile Boltzmann weights} that enhance the previous knot coloring invariant defined in terms of finite biquasiles and provide examples differentiating links with the same counting invariant, demonst…

EKM addresses imbalanced data clustering by repelling centroids in large clusters.

CG-BGs combine flow-based models with PMFs to sample large systems efficiently.

Learning in restricted Boltzmann machine is typically hard due to the computation of gradients of log-likelihood function. To describe the network state statistics of the restricted Boltzmann machine, we develop an advanced mean field theory based on the Bethe approximation. Our theory provides an efficient message pas…

The restricted Boltzmann machine is a network of stochastic units with undirected interactions between pairs of visible and hidden units. This model was popularized as a building block of deep learning architectures and has continued to play an important role in applied and theoretical machine learning. Restricted Bolt…

Near-Exponential Convergence Rates for kNN Classification

The Boltzmann machine provides a useful framework to learn highly complex, multimodal and multiscale data distributions that occur in the real world. The default method to learn its parameters consists of minimizing the Kullback-Leibler (KL) divergence from training samples to the Boltzmann model. We propose in this wo…

We propose a method to decrease the number of hidden units of the restricted Boltzmann machine while avoiding decrease of the performance measured by the Kullback-Leibler divergence. Then, we demonstrate our algorithm by using numerical simulations.

In emissions trading, the initial allocation of permits is an intractable issue because it needs to be essentially fair to the participating countries. There are many ways to distribute a given total amount of emissions permits among countries, but the existing distribution methods, such as auctioning and grandfatherin…

Deep Boltzmann machines are in principle powerful models for extracting the hierarchical structure of data. Unfortunately, attempts to train layers jointly (without greedy layer-wise pretraining) have been largely unsuccessful. We propose a modification of the learning algorithm that initially recenters the output of t…

In recent work, Boltzmann and Fokker-Planck equations were derived for the "Yard-Sale Model" of asset exchange. For the version of the model without redistribution, it was conjectured, based on numerical evidence, that the time-asymptotic state of the model was oligarchy -- complete concentration of wealth by a single …