Diffusion models' speed-accuracy relations derived from thermodynamics.
problem Understanding the trade-off between model speed and accuracy.
method Connecting diffusion models to thermodynamics and optimal transport.
result Speed-accuracy relations derived, providing insights into optimal learning protocols.
Study on thermodynamic costs of simple linear regression.
problem Understanding thermodynamic costs in machine learning models.
method Approximated thermodynamic lower bounds for exact and stochastic linear regression.
result Derived scaling laws for optimal dataset size based on generalization error.
New method learns stochastic thermodynamics from system currents.
problem Understanding entropy production in complex dynamical systems.
method Constructs learning framework using currents and machine learning loss functions.
result Derives loss functions for thermodynamic functions directly from dynamics.
The paper applies thermodynamics to financial markets to prove no-arbitrage constraints.
problem No arbitrage in financial markets under price impact.
method Stochastic thermodynamics applied to financial trading cycles.
result Proves any round-trip trading strategy yields non-positive expected profit.
Traditionally, neural networks are parameterized using optimization procedures such as stochastic gradient descent, RMSProp and ADAM. These procedures tend to drive the parameters of the network toward a local minimum. In this article, we employ alternative "sampling" algorithms (referred to here as "thermodynamic para…
Schrödinger and Onsager's ideas linked in nonequilibrium thermodynamics.
problem Linking Schrödinger's variational problem with Onsager's nonequilibrium statistical mechanics.
method Analyzing the historical context and comparing the two approaches.
result Schrödinger's ideas have not yet been fully integrated into the classical context of Onsager's work.
Work maximization guides machine learning models in adaptive systems.
problem How machine learning models can be optimized for thermodynamic efficiency.
method Introducing thermodynamic principle to compare with maximum-likelihood principle.
result Maximum-work models are equivalent to maximum-likelihood models in adaptive systems.
Recent research has considered the stochastic thermodynamics of multiple interacting systems, representing the overall system as a Bayes net. I derive fluctuation theorems governing the entropy production (EP)of arbitrary sets of the systems in such a Bayes net. I also derive ``conditional'' fluctuation theorems, gover…
We analyze wealth condensation for a wide class of stochastic economy models on the basis of the economic analog of thermodynamic potentials, termed transfer potentials. The economy model is based on three common transfers modes of wealth: random transfer, profit proportional to wealth and motivation of poor agents to …
Modified Gibbs-Helmholtz equation geometric models for thermodynamics.
problem Geometric interpretation of Gibbs-Helmholtz equation in thermodynamics.
method Developed new holonomic and non-holonomic geometric models associated to Gibbs-Helmholtz equation.
result Characterized equivalence between Gibbs-Helmholtz entropy and other entropies.
The formal structure of geometrical thermodynamics is reviewed with particular emphasis on the geometry of equilibria submanifolds. On these submanifolds thermodynamic metrics are defined as the Hessian of thermodynamic potentials. Links between geometry and thermodynamics are explored for single and multiple component…
The thermodynamics of markets is analyzed, revealing parallels with thermodynamic laws.
problem Understanding economic systems through thermodynamic principles.
method Derivation and analysis of thermodynamic laws applied to economic systems.
result Deep analogy between thermodynamic and economic parameters established.
New framework reveals thermodynamic principles for LLM training.
problem Understanding the training dynamics of large language models.
method Introducing Neural Thermodynamic Laws (NTL) under river-valley loss landscape assumptions.
result Key thermodynamic quantities and principles naturally emerge in LLM training.
This study compares different thermodynamic structure-informed neural networks for solving differential equations.
problem Improving the accuracy and physical consistency of neural network solutions to differential equations.
method Comprehensive evaluation of various thermodynamic formulations in physics-informed neural networks.
result Newtonian-residual-based PINNs fail to reliably recover physical quantities, while structure-preserving formulations enhance accuracy and robustness.
Geometric approach to quantum thermodynamics models state spaces and processes.
problem Quantum thermodynamics in the regime of non-equilibrium states.
method Contact geometry and principal fiber bundles to model quantum state spaces and processes.
result Geometric formulation reveals the fundamental thermodynamic relations and unattainability of the third law.
Paper uses referenced thermodynamic integration for Bayesian model selection in a complex COVID-19 transmission model.
problem Bayesian model selection with uncertainty and misleading metrics.
method Referenced thermodynamic integration for intractable high-dimensional distributions.
result Favourable convergence performance in model selection for COVID-19 transmission.
By treating the financial market as a thermodynamic system, we establish a one-to-one correspondence between thermodynamic variables and economic quantities. Measured by the expected loss under the worst-case scenario, financial risk caused by model uncertainty is regarded as a result of the interaction between financi…
Geometric structures help in understanding thermodynamics.
problem Understanding thermodynamic systems using geometric methods.
method Using almost cosymplectic structures and variational arguments.
result Evolution equations are derived and discussed.
New limits found for training deep learning models efficiently.
problem Optimizing the training speed of deep learning models without sacrificing accuracy.
method Applied stochastic thermodynamics to set speed limits for neural network training.
result Training neural networks is optimal within certain scaling assumptions.
PA reinterpreted as SB problem, unifying thermodynamics and optimal transport.
problem Optimizing discrete-time paths between probability distributions.
method Schrödinger Bridge theory, solving variational problems.
result PA's reweighting step derived from Schrödinger system.
A stochastic theory for the toppling activity in sandpile models is developed, based on a simple mean-field assumption about the toppling process. The theory describes the process as an anti-persistent Gaussian walk, where the diffusion coefficient is proportional to the activity. It is formulated as a generalization o…
With the advent of GPU-assisted hardware and maturing high-efficiency software platforms such as TensorFlow and PyTorch, Bayesian posterior sampling for neural networks becomes plausible. In this article we discuss Bayesian parametrization in machine learning based on Markov Chain Monte Carlo methods, specifically disc…
Unified framework for training diffusion and flow models to sample from target distributions.
problem Training diffusion and flow models to sample from target distributions defined by exponential tilting.
method Unified framework combining stochastic optimal control and non-equilibrium thermodynamics perspectives.
result Unified bias-variance decompositions and theoretical support for adjoint-based methods.
New analysis shows temperature guarantees generalization in stochastic training.
problem Analyzing the generalization gap in stochastic training algorithms.
method Analyzing Langevin dynamics with positive temperature and bounding the generalization gap.
result Bounding the generalization gap by √(βE[L(θ_0)] + log(1/δ))/√N with probability 1-δ.
Geometric study of thermodynamics using cotangent bundles.
problem Formulating classical thermodynamics using geometric structures.
method Contact geometry on cotangent bundles, homogeneous coordinates for intensive variables.
result Geometric formulation of thermodynamics with homogeneity properties.
Following the thermodynamic formulation of multifractal measure that was shown to be capable of detecting large fluctuations at an early stage, here we propose a new index which permits us to distinguish events like financial crisis in real time . We calculate the partition function from where we obtain thermodynamic q…
Generative diffusion models exhibit phase transitions in statistical mechanics, impacting their performance.
problem Understanding the performance and capabilities of generative diffusion models.
method Reformulating generative diffusion models using statistical mechanics, focusing on phase transitions and symmetry breaking.
result Generative diffusion models undergo second-order phase transitions with mean-field universality, critical instability, and mean-field critical exponents.
Financial market dynamics compared to thermodynamics.
problem Understanding the dynamics of financial markets through thermodynamic principles.
method Analogy with Szilárd information engine to derive market temperature and information extraction.
result Informed traders' gains are bounded by market temperature and information.
The author proposes a finance trading strategy named Entropy Oriented Trading and apply thermodynamics on the strategy. The state variables are chosen so that the strategy satisfies the second law of thermodynamics. Using the law, the author proves that the rate of investment (ROI) of the strategy is equal to or more t…
The paper analyzes how SGD visits different regions of a non-convex problem's state space.
problem Understanding the long-run distribution of stochastic gradient descent in non-convex problems.
method Large deviations theory and randomly perturbed dynamical systems.
result The long-run distribution of SGD resembles the Boltzmann-Gibbs distribution with temperature equal to the step-size.
Within the description of stochastic differential equations it is argued that the existence of Boltzmann-Gibbs type distribution in economy is independent of the time reversal symmetry in econodynamics. Both power law and exponential distributions can be accommodated by it. The demonstration is based on a mathematical …
Develops new techniques for learning from sequential data groups.
problem Learning from groups of inputs rather than individual inputs.
method Introduces feature-based and kernel-based learning techniques for sequential data.
result Achieves state-of-the-art performance on various real-world examples.
New Holder bounds improve variational inference by flattening thermodynamic curves.
problem Improving variational inference by addressing performance gaps between theory and practice.
method Generalizing thermodynamic integration to weighted Holder mean, introducing Holder bounds.
result Holder bounds promise a one-step approximation of exact marginal log-likelihood.
We propose a new method of valuation of portfolios and their respective investing strategies. To this end we define a canonical ensemble of portfolios that allows to use the formalism thermodynamics.
Upper bound derived for informed traders' gains in a model, akin to thermodynamics.
problem Informed traders' gains in a financial model with finite horizon.
method Bayesian inference and entropic inequality.
result Upper bound for expected gain, analogous to thermodynamics.
In this work we offer a framework for reasoning about a wide class of existing objectives in machine learning. We develop a formal correspondence between this work and thermodynamics and discuss its implications.
The paper provides a geometric framework for understanding non-equilibrium thermodynamics.
problem Unclear geometric structure of GENERIC in non-equilibrium thermodynamics.
method Cotangent lifts of dynamics, splitting into holonomic and vertical representatives, and formulation within contact geometry.
result Physical meaning and explicit formulation of the second law of thermodynamics within evolution equations.
Develops thermodynamic formalism for quasimorphisms on negatively curved spaces.
problem Analyzing quasimorphisms on negatively curved spaces.
method Thermodynamic formalism framework, Banach isomorphism, weak Livšic cohomology.
result Establishes Central Limit Theorem and invariance principle for unbounded quasimorphisms.
Dirac structures are geometric objects that generalize both Poisson structures and presymplectic structures on manifolds. They naturally appear in the formulation of constrained mechanical systems. In this paper, we show that the evolution equa- tions for nonequilibrium thermodynamics admit an intrinsic formulation in …
This work develops a machine learning approach to EOS models that accounts for thermodynamic constraints and model uncertainty.
problem Developing accurate equation of state models for high energy-density experiments with inherent uncertainties.
method Physics-informed Gaussian process regression (GPR) framework to capture model uncertainty and thermodynamic constraints.
result The proposed framework reduces prediction uncertainty by incorporating thermodynamic constraints, as demonstrated for diamond carbon EOS.
Develops neural networks for learning physics of complex systems by enforcing thermodynamics principles.
problem Learning physics of complex systems from incomplete experimental data.
method Integrates port-metriplectic formalism with neural networks to enforce thermodynamics principles.
result Neural networks can learn physics of complex systems by parts, reducing learning burden.
We study the indefinite metric G in the contact phase space (P,θ) of a homogeneous thermodynamical system introduced by R. Mrugala. We calculate the curvature tensor, Killing vector fields, second fundamental form of Legendre submanifolds of P - constitutive surfaces of different homogeneous thermodynamical syste…
We formulate thermodynamics of economic systems in terms of an arbitrary probability distribution for a conserved economic quantity. As in statistical physics, thermodynamic macroeconomic variables emerge as the mean value of microeconomic variables and their determination is reduced to the computation of the partition…
We show that classical thermodynamics has a formulation in terms of Hamilton-Jacobi theory, analogous to mechanics. Even though the thermodynamic variables come in conjugate pairs such as pressure/volume or temperature/entropy, the phase space is odd-dimensional. For a system with n thermodynamic degrees of freedom it …
Rate GENERIC extends thermodynamics principles to non-equilibrium systems.
problem Understanding non-equilibrium thermodynamics and its relation to equilibrium thermodynamics.
method Developed a geometrical framework for rate GENERIC, extending Onsager's variational principle.
result Rate GENERIC structure provides a new perspective on thermodynamics in non-equilibrium systems.
TANNs integrate thermodynamics into ANN models for accurate, consistent predictions.
problem Lack of rigorous physics-based approach in ANN constitutive modeling.
method TANNs encode thermodynamics principles in neural network architecture using automatic differentiation.
result TANNs produce thermodynamically consistent predictions without requiring large datasets.
A thermodynamic theory explains EU election vote distributions.
problem Describing EU election vote distributions.
method Tracing parallels between system energies of coupled nonlinear oscillators and party vote fractions.
result The Rayleigh-Jeans (RJ) theory well depicts EU vote results and candidate vote dispersion.
Proof of boundedness of quasimorphisms for certain Lie groups.
problem Bounding quasimorphisms on lattices in Lie groups.
method Thermodynamic formalism applied to bounded cohomology.
result Every π-quasimorphism on irreducible uniform lattices is bounded.