New Holder bounds improve variational inference by flattening thermodynamic curves.
problem Improving variational inference by addressing performance gaps between theory and practice.
method Generalizing thermodynamic integration to weighted Holder mean, introducing Holder bounds.
result Holder bounds promise a one-step approximation of exact marginal log-likelihood.
We introduce the thermodynamic variational objective (TVO) for learning in both continuous and discrete deep generative models. The TVO arises from a key connection between variational inference and thermodynamic integration that results in a tighter lower bound to the log marginal likelihood than the standard variatio…
Unified framework for understanding TVO and improving model learning.
problem Improving the tightness and efficiency of variational inference bounds.
method Exponential family interpretation and equal spacing in moment parameters.
result Unified framework and improved gradient estimator for TVO.
Dirac structures are geometric objects that generalize both Poisson structures and presymplectic structures on manifolds. They naturally appear in the formulation of constrained mechanical systems. In this paper, we show that the evolution equa- tions for nonequilibrium thermodynamics admit an intrinsic formulation in …
Geometric structures help in understanding thermodynamics.
problem Understanding thermodynamic systems using geometric methods.
method Using almost cosymplectic structures and variational arguments.
result Evolution equations are derived and discussed.
PA reinterpreted as SB problem, unifying thermodynamics and optimal transport.
problem Optimizing discrete-time paths between probability distributions.
method Schrödinger Bridge theory, solving variational problems.
result PA's reweighting step derived from Schrödinger system.
Extends likelihood ratio exponential families to analyze various optimization methods.
problem Analyzing optimization methods like rate-distortion and information bottleneck.
method Linking geometric mixture paths to exponential families and using hypothesis testing.
result Provides a common mathematical framework for understanding these methods.
Paper variates Navier-Stokes-Fourier system for thermodynamic consistency.
problem Modeling compressible fluid dynamics with thermodynamic constraints.
method Variational discretization with discrete exterior calculus.
result Derives a nonholonomic variational integrator for NSF system.
In this work we offer a framework for reasoning about a wide class of existing objectives in machine learning. We develop a formal correspondence between this work and thermodynamics and discuss its implications.
Rate GENERIC extends thermodynamics principles to non-equilibrium systems.
problem Understanding non-equilibrium thermodynamics and its relation to equilibrium thermodynamics.
method Developed a geometrical framework for rate GENERIC, extending Onsager's variational principle.
result Rate GENERIC structure provides a new perspective on thermodynamics in non-equilibrium systems.
This study compares different thermodynamic structure-informed neural networks for solving differential equations.
problem Improving the accuracy and physical consistency of neural network solutions to differential equations.
method Comprehensive evaluation of various thermodynamic formulations in physics-informed neural networks.
result Newtonian-residual-based PINNs fail to reliably recover physical quantities, while structure-preserving formulations enhance accuracy and robustness.
Schrödinger and Onsager's ideas linked in nonequilibrium thermodynamics.
problem Linking Schrödinger's variational problem with Onsager's nonequilibrium statistical mechanics.
method Analyzing the historical context and comparing the two approaches.
result Schrödinger's ideas have not yet been fully integrated into the classical context of Onsager's work.
Geometric study of thermodynamics using cotangent bundles.
problem Formulating classical thermodynamics using geometric structures.
method Contact geometry on cotangent bundles, homogeneous coordinates for intensive variables.
result Geometric formulation of thermodynamics with homogeneity properties.
The paper solves a thermodynamics problem about crystal shape.
problem Understanding if minimizing free energy with convex potential and mass constraint generates a convex crystal.
method Utilized a stability theorem, convexity, and a new maximum principle approach to prove a three-dimensional convexity theorem.
result Completely settled the Almgren problem in R3 under generic conditions. Paper improves variance control in importance weighted variational bounds.
problem Improving the variance of gradient estimators for IWAE.
method Develops a novel control variate that grows SNR as √K for large K.
result Empirically, the method yields superior variance reduction for generative models.
In this work we show that the systems of balance equations (balance systems) of continuum thermodynamics occupy a natural place in the variational bicomplex formalism. We apply the vertical homotopy decomposition to get a local splitting (in a convenient domain) of a general balance system as the sum of a Lagrangian pa…
In this paper we study a class of physical systems that combine a finite number of mechanical and thermodynamic observables. We call them finite dimensional thermo-mechanical systems. We introduce these systems by means of simple examples. The evolution equations of the involved observables are obtained in each example…
The formal structure of geometrical thermodynamics is reviewed with particular emphasis on the geometry of equilibria submanifolds. On these submanifolds thermodynamic metrics are defined as the Hessian of thermodynamic potentials. Links between geometry and thermodynamics are explored for single and multiple component…
The thermodynamics of markets is analyzed, revealing parallels with thermodynamic laws.
problem Understanding economic systems through thermodynamic principles.
method Derivation and analysis of thermodynamic laws applied to economic systems.
result Deep analogy between thermodynamic and economic parameters established.
New framework reveals thermodynamic principles for LLM training.
problem Understanding the training dynamics of large language models.
method Introducing Neural Thermodynamic Laws (NTL) under river-valley loss landscape assumptions.
result Key thermodynamic quantities and principles naturally emerge in LLM training.
Paper uses referenced thermodynamic integration for Bayesian model selection in a complex COVID-19 transmission model.
problem Bayesian model selection with uncertainty and misleading metrics.
method Referenced thermodynamic integration for intractable high-dimensional distributions.
result Favourable convergence performance in model selection for COVID-19 transmission.
Geometric approach to quantum thermodynamics models state spaces and processes.
problem Quantum thermodynamics in the regime of non-equilibrium states.
method Contact geometry and principal fiber bundles to model quantum state spaces and processes.
result Geometric formulation reveals the fundamental thermodynamic relations and unattainability of the third law.
For common people, in contrast to brokers, bankers, and those who play on rising and falling prices of stocks, the stock market law is based on the simple fact that the depositors aim for financial profit at any given concrete stage. The common depositor cannot cause any significant variations in prices. This concept s…
Machine learning generates coarse-grained force fields for molecular dynamics.
problem Creating thermodynamically consistent coarse-grained models for larger systems.
method Hybrid architecture using graph neural networks to learn molecular features.
result Framework reproduces thermodynamics for small biomolecular systems.
We formulate the variational problem for AdS gravity with Dirichlet boundary conditions and demonstrate that the covariant counterterms are necessary to make the variational problem well-posed. The holographic charges associated with asymptotic symmetries are then rederived via Noether's theorem and `covariant phase sp…
By treating the financial market as a thermodynamic system, we establish a one-to-one correspondence between thermodynamic variables and economic quantities. Measured by the expected loss under the worst-case scenario, financial risk caused by model uncertainty is regarded as a result of the interaction between financi…
Following the thermodynamic formulation of multifractal measure that was shown to be capable of detecting large fluctuations at an early stage, here we propose a new index which permits us to distinguish events like financial crisis in real time . We calculate the partition function from where we obtain thermodynamic q…
Study on thermodynamic costs of simple linear regression.
problem Understanding thermodynamic costs in machine learning models.
method Approximated thermodynamic lower bounds for exact and stochastic linear regression.
result Derived scaling laws for optimal dataset size based on generalization error.
Geodesic coordinates derived for a specific metric in surface group representations.
problem Computing geodesic coordinates for a specific metric in surface group representations.
method Using thermodynamic formalism and gauge-theoretic formulas, computing first and second derivatives of the pressure metric.
result First derivatives of the pressure metric vanish at the Fuchsian locus.
Diffusion models' speed-accuracy relations derived from thermodynamics.
problem Understanding the trade-off between model speed and accuracy.
method Connecting diffusion models to thermodynamics and optimal transport.
result Speed-accuracy relations derived, providing insights into optimal learning protocols.
Financial market dynamics compared to thermodynamics.
problem Understanding the dynamics of financial markets through thermodynamic principles.
method Analogy with Szilárd information engine to derive market temperature and information extraction.
result Informed traders' gains are bounded by market temperature and information.
The author proposes a finance trading strategy named Entropy Oriented Trading and apply thermodynamics on the strategy. The state variables are chosen so that the strategy satisfies the second law of thermodynamics. Using the law, the author proves that the rate of investment (ROI) of the strategy is equal to or more t…
Derives fluctuation theorems and thermodynamic uncertainty relations for systems modeled as Bayes nets.
problem Entropy production in interacting systems modeled as Bayes nets.
method Derives fluctuation theorems and thermodynamic uncertainty relations for arbitrary sets and conditioned sets of systems in Bayes nets.
result Relates the entropy production of the overall system to the precisions of probability currents in individual systems.
The paper applies thermodynamics to financial markets to prove no-arbitrage constraints.
problem No arbitrage in financial markets under price impact.
method Stochastic thermodynamics applied to financial trading cycles.
result Proves any round-trip trading strategy yields non-positive expected profit.
We propose a new method of valuation of portfolios and their respective investing strategies. To this end we define a canonical ensemble of portfolios that allows to use the formalism thermodynamics.
Upper bound derived for informed traders' gains in a model, akin to thermodynamics.
problem Informed traders' gains in a financial model with finite horizon.
method Bayesian inference and entropic inequality.
result Upper bound for expected gain, analogous to thermodynamics.
In statistics and machine learning, approximation of an intractable integration is often achieved by using the unbiased Monte Carlo estimator, but the variances of the estimation are generally high in many applications. Control variates approaches are well-known to reduce the variance of the estimation. These control v…
The paper provides a geometric framework for understanding non-equilibrium thermodynamics.
problem Unclear geometric structure of GENERIC in non-equilibrium thermodynamics.
method Cotangent lifts of dynamics, splitting into holonomic and vertical representatives, and formulation within contact geometry.
result Physical meaning and explicit formulation of the second law of thermodynamics within evolution equations.
Work maximization guides machine learning models in adaptive systems.
problem How machine learning models can be optimized for thermodynamic efficiency.
method Introducing thermodynamic principle to compare with maximum-likelihood principle.
result Maximum-work models are equivalent to maximum-likelihood models in adaptive systems.
Develops thermodynamic formalism for quasimorphisms on negatively curved spaces.
problem Analyzing quasimorphisms on negatively curved spaces.
method Thermodynamic formalism framework, Banach isomorphism, weak Livšic cohomology.
result Establishes Central Limit Theorem and invariance principle for unbounded quasimorphisms.
With their origin in thermodynamics and symbolic dynamics, Gibbs measures are crucial tools to study the ergodic theory of the geodesic flow on negatively curved manifolds. We develop a framework (through Patterson-Sullivan densities) allowing us to get rid of compactness assumptions on the manifold, and prove many exi…
This work develops a machine learning approach to EOS models that accounts for thermodynamic constraints and model uncertainty.
problem Developing accurate equation of state models for high energy-density experiments with inherent uncertainties.
method Physics-informed Gaussian process regression (GPR) framework to capture model uncertainty and thermodynamic constraints.
result The proposed framework reduces prediction uncertainty by incorporating thermodynamic constraints, as demonstrated for diamond carbon EOS.
Develops neural networks for learning physics of complex systems by enforcing thermodynamics principles.
problem Learning physics of complex systems from incomplete experimental data.
method Integrates port-metriplectic formalism with neural networks to enforce thermodynamics principles.
result Neural networks can learn physics of complex systems by parts, reducing learning burden.
We study the indefinite metric G in the contact phase space (P,θ) of a homogeneous thermodynamical system introduced by R. Mrugala. We calculate the curvature tensor, Killing vector fields, second fundamental form of Legendre submanifolds of P - constitutive surfaces of different homogeneous thermodynamical syste…
We show that classical thermodynamics has a formulation in terms of Hamilton-Jacobi theory, analogous to mechanics. Even though the thermodynamic variables come in conjugate pairs such as pressure/volume or temperature/entropy, the phase space is odd-dimensional. For a system with n thermodynamic degrees of freedom it …
We formulate thermodynamics of economic systems in terms of an arbitrary probability distribution for a conserved economic quantity. As in statistical physics, thermodynamic macroeconomic variables emerge as the mean value of microeconomic variables and their determination is reduced to the computation of the partition…
TANNs integrate thermodynamics into ANN models for accurate, consistent predictions.
problem Lack of rigorous physics-based approach in ANN constitutive modeling.
method TANNs encode thermodynamics principles in neural network architecture using automatic differentiation.
result TANNs produce thermodynamically consistent predictions without requiring large datasets.
The paper analyzes how SGD visits different regions of a non-convex problem's state space.
problem Understanding the long-run distribution of stochastic gradient descent in non-convex problems.
method Large deviations theory and randomly perturbed dynamical systems.
result The long-run distribution of SGD resembles the Boltzmann-Gibbs distribution with temperature equal to the step-size.