arXiv research

In 1870s, L. Boltzmann proved the famous $H$-theorem for the Boltzmann equation in the kinetic theory of gas and gave the statistical interpretation of the thermodynamic entropy. In 2002, G. Perelman introduced the notion of $W$-entropy and proved the $W$-entropy formula for the Ricci flow. This plays a crucial role in…

We present a new statistical learning paradigm for Boltzmann machines based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes maximum entropy principle and from standard maximum likelihood estimation.We demonstrate the LME principle BY deriving …

We study a Boltzmann's type entropy functional (which appeared in existing literature) defined on Kähler metrics of a fixed Kähler class. The critical points of this functional are gradient Kähler-Ricci solitons, and the functional was known to be monotonically increasing along the Kähler-Ricci flow in the canonical cl…

The notion of utility maximising entropy (u-entropy) of a probability density, which was introduced and studied by Slomczynski and Zastawniak (Ann. Prob 32 (2004) 2261-2285, arXiv:math.PR/0410115 v1), is extended in two directions. First, the relative u-entropy of two probability measures in arbitrary probability space…

In this note we determine the first two derivatives of the classical Boltzmann-Shannon entropy of the conjugate heat equation on general evolving manifolds. Based on the second derivative of the Boltzmann-Shannon entropy, we construct Perelman's F and W entropy in abstract geometric flows. Monotonicity of the entropies…

We give a notion of entropy for general gemetric structures, which generalizes well-known notions of topological entropy of vector fields and geometric entropy of foliations, and which can also be applied to singular objects, e.g. singular foliations, singular distributions, and Poisson structures. We show some basic p…

On a Riemannian manifold, lower Ricci curvature bounds are known to be characterized by geodesic convexity properties of various entropies with respect to the Kantorovich-Rubinstein-Wasserstein square distance from optimal transportation. These notions also make sense in a (nonsmooth) metric measure setting, where they…

The paper applies math and physics to language models, introducing entropy and geometric concepts.

Rate GENERIC extends thermodynamics principles to non-equilibrium systems.

The study analyzes the evolution of Gaussian measures under a specific gradient flow.

We prove the eventological $H$-theorem that complements the Boltzmann H-theorem from statistical mechanics and serves as a mathematical excuse (mathematically no less convincing than the Boltzmann H-theorem for the second law of thermodynamics) for what can be called "the second law of eventology", which justifies the …

Paper introduces a new measure combining entropy and Gini index.

Researchers use Gaussian processes to approximate Lagrange multipliers for Maximum-Entropy distributions.

We introduce the notions of `super-Ricci flows' and `Ricci flows' for time-dependent families of metric measure spaces $(X,d_t,m_t)_{t\in I}$. The former property is proven to be stable under suitable space-time versions of mGH-convergence. Uniformly bounded families of super-Ricci flows are compact. In the spirit of t…

Kelly criterion, that maximizes the expectation value of the logarithm of wealth for bookmaker bets, gives an advantage over different class of strategies. We use projective symmetries for a explanation of this fact. Kelly's approach allows for an interesting financial interpretation of the Boltzmann/Shannon entropy. A…

We review some approaches to the understanding of fluctuations in some models used to describe socio and economic systems. Our approach builds on the development of a simple Langevin equation that characterises stochastic processes. This provides a unifying approach that allows first a straightforward description of th…

We study the heat equation on time-dependent metric measure spaces (as well as the dual and the adjoint heat equation) and prove existence, uniqueness and regularity. Of particular interest are properties which characterize the underlying space as a super Ricci flow as previously introduced by the second author. Our ma…

Restricted Boltzmann machines (RBMs) are a powerful class of generative models, but their training requires computing a gradient that, unlike supervised backpropagation on typical loss functions, is notoriously difficult even to approximate. Here, we show that properly combining standard gradient updates with an off-gr…

EDLP samples flat modes in discrete spaces using entropy.

Entropy formula derived for deep linear networks using geometric analysis.

The paper extends entropy maximization to multiscale settings and applies it to neural networks.

A new AI framework reduces costs and improves performance.

In this paper, we propose a novel maximum causal Tsallis entropy (MCTE) framework for imitation learning which can efficiently learn a sparse multi-modal policy distribution from demonstrations. We provide the full mathematical analysis of the proposed framework. First, the optimal solution of an MCTE problem is shown …

The cornerstone of Boltzmann-Gibbs ($BG$) statistical mechanics is the Boltzmann-Gibbs-Jaynes-Shannon entropy $S_{BG} \equiv -k\int dx f(x)\ln f(x)$, where $k$ is a positive constant and $f(x)$ a probability density function. This theory has exibited, along more than one century, great success in the treatment of syste…

Maximum entropy deep reinforcement learning (RL) methods have been demonstrated on a range of challenging continuous tasks. However, existing methods either suffer from severe instability when training on large off-policy data or cannot scale to tasks with very high state and action dimensionality such as 3D humanoid l…

Statistical mechanics explains income and wealth distribution in developed economies.

In this survey paper, we give an overview of our recent works on the study of the $W$-entropy for the heat equation associated with the Witten Laplacian on super-Ricci flows and the Langevin deformation on Wasserstein space over Riemannian manifolds. Inspired by Perelman's seminal work on the entropy formula for the Ri…

Modified Gibbs-Helmholtz equation geometric models for thermodynamics.

In this paper, we prove the characterization of the $(K, \infty)$-super Perelman Ricci flows by various functional inequalities and gradient estimate for the heat semigroup generated by the Witten Laplacian on manifolds equipped with time dependent metrics and potentials. As a byproduct, we derive the Hamilton type dim…

This review explores entropy applications in data analysis and machine learning.

Deep networks have enabled reinforcement learning to scale to more complex and challenging domains, but these methods typically require large quantities of training data. An alternative is to use sample-efficient episodic control methods: neuro-inspired algorithms which use non-/semi-parametric models that predict valu…

We analyze a simple asset transfer model in which the transfer amount is a fixed fraction $f$ of the giver's wealth. The model is analyzed in a new way by Laplace transforming the master equation, solving it analytically and numerically for the steady-state distribution, and exploring the solutions for various values o…

DO-EM framework for quantum models improves generative tasks.

The Bregman divergence (Bregman distance, Bregman measure of distance) is a certain useful substitute for a distance, obtained from a well-chosen function (the "Bregman function"). Bregman functions and divergences have been extensively investigated during the last decades and have found applications in optimization, o…

The paper introduces a new price model based on entropy that better fits high-frequency market data.

Transferable Boltzmann Generators learn to sample unseen molecules efficiently.

Study optimal transport on null hypersurfaces and null energy condition.

The restricted Boltzmann machine (RBM) is one of the fundamental building blocks of deep learning. RBM finds wide applications in dimensional reduction, feature extraction, and recommender systems via modeling the probability distributions of a variety of input data including natural images, speech signals, and custome…

A plethora of natural, artificial and social systems exist which do not belong to the Boltzmann-Gibbs (BG) statistical-mechanical world, based on the standard additive entropy $S_{BG}$ and its associated exponential BG factor. Frequent behaviors in such complex systems have been shown to be closely related to $q$-stati…

D-Wave computers struggle with sampling Boltzmann distributions efficiently.

We present transductive Boltzmann machines (TBMs), which firstly achieve transductive learning of the Gibbs distribution. While exact learning of the Gibbs distribution is impossible by the family of existing Boltzmann machines due to combinatorial explosion of the sample space, TBMs overcome the problem by adaptively …

Quantum Boltzmann Machines trained on quantum annealers produce noisy synthetic data.

The probability density function for the visible sector of a Riemann-Theta Boltzmann machine can be taken conditional on a subset of the visible units. We derive that the corresponding conditional density function is given by a reparameterization of the Riemann-Theta Boltzmann machine modelling the original probability…

We introduce a new method for training deep Boltzmann machines jointly. Prior methods require an initial learning pass that trains the deep Boltzmann machine greedily, one layer at a time, or do not perform well on classifi- cation tasks.

RBM and DBM are represented as 2D tensor networks, revealing their expressive power and efficiency.

A new method speeds up sampling of Boltzmann distribution in high-dimensional systems.

We show that deep narrow Boltzmann machines are universal approximators of probability distributions on the activities of their visible units, provided they have sufficiently many hidden layers, each containing the same number of units as the visible layer. We show that, within certain parameter domains, deep Boltzmann…

A new machine learning model uses score matching to estimate probability densities efficiently.