This work uses statistical mechanics to explain AI learning.
problem Understanding the statistical principles behind AI learning.
method Starting from sample concentration behaviors, the study applies statistical mechanics principles to AI and machine learning.
result Exponential families and statistical quantities are key in AI and machine learning.
Formulates mechanics for probability distributions on statistical manifold.
problem Formulating mechanics for probability distributions on statistical manifold.
method Information-geometric formulation of Classical Mechanics on statistical manifold, using dually-flat connection and Hilbert bundle structure.
result Provides coherent formalism for Lagrangian and Hamiltonian mechanics on statistical bundle.
The Yukawa term in statistical mechanics quantifies information generation.
problem Understanding information generation in statistical mechanics.
method Defining a statistical product and Yukawa term to quantify information generation.
result The Yukawa term diverges in the quantum case, indicating Bose-Einstein condensation.
A new mechanism for differentially private Fréchet mean on SPD matrices.
problem Privacy-preserving statistical summaries for SPD matrices.
method Tangent Gaussian mechanism for log-Euclidean metric.
result Significantly better utility and computational efficiency.
This paper analyzes MFVBI for GMM using statistical mechanics.
problem Approximate fast computation of Gaussian Mixture Model.
method Statistical mechanics and MFVBI applied to GMM.
result Rigorous analysis and mathematical foundation for MFVBI applied to GMM.
I introduce a new geometrical approach to thermo--statistical mechanics. Here I highlight the main physical ideas, and how do they translate into geometrical language. I contrast the present approach with previous thermo--statistical--geometrical formalisms, (pseudo-)Riemannian [Weinhold 1975; Ruppeiner 1979] as well a…
New method uses quantum annealing and VAN for better statistical mechanics calculations.
problem Difficulty in computing partition function in statistical mechanics.
method Combines quantum annealing samples with variational autoregressive networks.
result Enhanced accuracy in finite-size Sherrington-Kirkpatrick model.
Paper reviews algebraic research in machine learning theory.
problem Understanding phase transitions in machine learning models.
method Algebraic approaches in statistical mechanics.
result Algebraic methods are essential for analyzing machine learning models with singularities.
Financial asset markets are sociotechnical systems whose constituent agents are subject to evolutionary pressure as unprofitable agents exit the marketplace and more profitable agents continue to trade assets. Using a population of evolving zero-intelligence agents and a frequent batch auction price-discovery mechanism…
Combines public and private data for better statistical estimation.
problem Estimating aggregate statistics from mixed data with varying privacy needs.
method Mixed estimators optimized for minimizing variance or median, using differential privacy techniques.
result Our mechanisms often outperform baseline methods in empirical tests.
Statistical mechanics reveals phase transitions in ε-SVR error.
problem Understanding task precision in neural representations with variability.
method Statistical mechanics applied to ε-SVR. result Double-descent phenomenon in generalization error due to ε. Osborne's paradox explained via Bayesian inference and superstatistics.
problem Osborne's paradox regarding stock price changes.
method Bayesian inference and superstatistics.
result Stock market returns are locally described by equilibrium statistical mechanics, globally by superstatistics.
This paper bridges Kahler geometry and quantum mechanics in lognormal statistical models.
problem Evolution of spectral curves in Siegel Jacobi space through Schrodinger equation.
method Kahler geometry induced on lognormal statistical manifold, Dombrowski's construction.
result Time-dependent Schrodinger equation with varying energy.
Paper develops a framework to discover bioprocessing regulatory mechanisms using symbolic and statistical learning.
problem Challenges in modeling complex intracellular regulation, stochastic system behavior, and limited experimental data.
method Symbolic and statistical learning framework based on stochastic differential equations and Bayesian learning.
result Improved sample efficiency and robust model selection compared to state-of-the-art approaches.
Study introduces statistical mechanics for min-max problems.
problem Understanding the properties of min-max problems in high dimensions.
method Statistical mechanical formalism for analyzing min-max problems.
result Derives the relationship between training data and generalization error.
Ergodicity, this is to say, dynamics whose time averages coincide with ensemble averages, naturally leads to Boltzmann-Gibbs (BG) statistical mechanics, hence to standard thermodynamics. This formalism has been at the basis of an enormous success in describing, among others, the particular stationary state correspondin…
Develops new instance-optimality concepts in differential privacy.
problem Improving privacy guarantees in statistical estimation.
method Introduces local minimax risk and unbiased mechanisms, and develops inverse sensitivity mechanisms.
result Inverse sensitivity mechanisms are nearly instance optimal for a wide range of functions.
We propose an optimum mechanism for providing monetary incentives to the data sources of a statistical estimator such as linear regression, so that high quality data is provided at low cost, in the sense that the sum of payments and estimation error is minimized. The mechanism applies to a broad range of estimators, in…
Framework for efficient statistical estimation with privacy guarantees.
problem Statistical estimation problems with differential privacy constraints.
method High-dimensional Propose-Test-Release (HPTR) framework combining exponential mechanism, robust statistics, and resilience.
result Near-optimal utility guarantees and tight local sensitivity bounds for various statistical problems.
The study reveals how attention paths in Transformers influence learning outcomes.
problem Understanding the theoretical basis of Transformers' performance.
method Developed a statistical mechanics theory for a simplified attention network.
result The predictor statistics are influenced by the combination of attention paths.
Analyzes unsupervised neural networks using statistical mechanics and Monte Carlo simulations.
problem Understanding computational capabilities of unsupervised neural networks.
method Statistical mechanics approach and Monte Carlo simulations.
result Obtained a phase diagram summarizing network performance.
New statistical mechanics analysis shows edge pruning outperforms node pruning in neural networks.
problem Theoretical understanding of neural network pruning effectiveness is lacking.
method Statistical mechanics analysis of a teacher-student framework.
result DPP node pruning method is superior to other methods, but edge pruning is better overall.
Investigates financial and economic systems using statistical mechanics and information theory.
problem Complexity, asymmetry, stochasticity, and non-linearity in financial and economic systems.
method Model-based and empirical analyses using statistical mechanics and information theory.
result Derives probability distribution functions for better understanding of financial and economic dynamics.
New method for efficient matrix completion with nonignorable missing data.
problem Nonignorable missing data in matrix completion.
method Nuclear norm regularized U-statistic loss function and accelerated proximal gradient algorithm.
result Near minimax optimal statistical convergence rate for nonignorable missing data.
The exponential mechanism is a fundamental tool of Differential Privacy (DP) due to its strong privacy guarantees and flexibility. We study its extension to settings with summaries based on infinite dimensional outputs such as with functional data analysis, shape analysis, and nonparametric statistics. We show that one…
The Ising model replicates financial asset statistical features.
problem Replicating statistical characteristics of financial markets.
method Employed the Ising model and Monte Carlo simulations.
result The Ising model can replicate most financial asset statistical features.
This paper provides a construction of a quantum statistical mechanical system associated to knots in the 3-sphere and cyclic branched coverings of the 3-sphere, which is an analog, in the sense of arithmetic topology, of the Bost-Connes system, with knots replacing primes, and cyclic branched coverings of the 3-sphere …
A theory of deep learning is emerging, focusing on training dynamics and statistics.
problem Develop a scientific theory to understand deep learning.
method Synthesize research into five areas: idealized settings, tractable limits, mathematical laws, hyperparameters, and universal behaviors.
result The emerging theory is a mechanics of the learning process, named learning mechanics.
This paper presents a new mechanism for producing sanitized statistical summaries that achieve \emph{differential privacy}, called the \emph{K-Norm Gradient} Mechanism, or KNG. This new approach maintains the strong flexibility of the exponential mechanism, while achieving the powerful utility performance of objective …
I present in this paper some tools in Symplectic and Poisson Geometry in view of their applications in Geometric mechanics and Mathematical Physics. After a short discussion of the Lagrangian and Hamiltonian formalisms, including the use of symmetry groups, and a presentation of the Tulczyjew's isomorphisms (which expl…
The paper provides stability guarantees for non-parametric maximum likelihood estimation using statistical mechanics.
problem Non-parametric maximum likelihood estimation and Gaussian mixture models.
method Statistical mechanics analysis to establish stability guarantees for NPMLE.
result High probability upper bounds on the Kullback-Leibler divergence between NPMLE estimators and the true density.
The paper studies scaling laws for associative memory mechanisms.
problem Understanding and optimizing learning and memorization processes.
method High-dimensional matrices of outer products of embeddings, relating to transformer models. Derived scaling laws with sample and parameter sizes. Extensive numerical experiments.
result Precise scaling laws and statistical efficiency of estimators.
Paper connects RL and non-equilibrium statistical mechanics for entropy-regularized RL.
problem Obtaining analytical solutions for entropy-regularized RL.
method Mapping RL to non-equilibrium statistical mechanics, applying large deviation theory.
result Derives exact analytical results for optimal policy and dynamics in MDPs.
Study proposes a statistical test for Vision Transformer's attention mechanisms.
problem ViT's attention mechanisms may focus on irrelevant regions, leading to unreliable evidence.
method Selective inference framework to quantify statistical significance of attentions as p-values.
result Proposed method enables reliable quantification of false positive detection probability of attentions.
The paper analyzes ℓ1-LinR for Ising model selection using statistical mechanics.
problem Model selection consistency of ℓ1-LinR for Ising models. method Replica method from statistical mechanics, ℓ1-regularized linear regression (ℓ1-LinR). result Model selection consistency with sample complexity $M=\mathcal{O}\left(\log N
ight)$.
Privacy improves robustness in statistical estimation.
problem Sparse mean estimation under privacy constraints.
method Sum-of-Squares method and exponential-time mechanisms.
result Private algorithms matching optimal tradeoffs are not known, but achieved via Sum-of-Squares.
This paper enhances privacy in statistical model checking of cyber-physical systems.
problem Privacy concerns in consumer-level applications due to statistical model checking.
method Proposes expected differential privacy and a new exponential mechanism for sequential algorithms.
result Demonstrates a novel mechanism to preserve privacy in statistical model checking.
Statistical mechanics explains learning in large neural networks near interpolation.
problem Understanding the learning dynamics of large neural networks near interpolation.
method Statistical physics analysis of a two-layer network with generic weight distribution and activation function.
result Learning transitions and feature learning emerge as the number of data increases.
Exponential families are a particular class of statistical manifolds which are particularly important in statistical inference, and which appear very frequently in statistics. For example, the set of normal distributions, with mean μ and deviation σ, form a 2-dimensional exponential family. In this paper, we show that …
Geometrically decomposes Kähler functions on toric manifolds.
problem Decomposing Kähler functions on Kähler toric manifolds.
method Defining spectrum of Kähler functions and proving spectral decomposition theorem.
result Geometric spectral theory for Kähler functions established.
This paper proposes a statistical mechanics approach to the analysis of income distribution and inequality. A new distribution function, having its roots in the framework of k-generalized statistics, is derived that is particularly suitable to describe the whole spectrum of incomes, from the low-middle income region up…
In this comment on "Solving Statistical Mechanics Using Variational Autoregressive Networks" by Wu et al., we propose a subtle yet powerful modification of their approach. We show that the inherent sampling error of their method can be corrected by using neural network-based MCMC or importance sampling which leads to a…
Privacy preserving mechanisms such as differential privacy inject additional randomness in the form of noise in the data, beyond the sampling mechanism. Ignoring this additional noise can lead to inaccurate and invalid inferences. In this paper, we incorporate the privacy mechanism explicitly into the likelihood functi…
Paper introduces data-dependent SSP for private linear and logistic regression.
problem Private linear and logistic regression with better performance.
method Data-dependent sufficient statistic perturbation (SSP) for linear and logistic regression.
result Data-dependent SSP outperforms state-of-the-art methods for linear and logistic regression.
We describe an approach to understand the peculiar and counterintuitive generalization properties of deep neural networks. The approach involves going beyond worst-case theoretical capacity control frameworks that have been popular in machine learning in recent years to revisit old ideas in the statistical mechanics of…
Method quantifies uncertainties in complex MRF models.
problem Uncertainties in MRF predictions due to data, modeling, and approximations.
method Information-based uncertainty quantification using MRF graphical structure.
result Tight bounds on predictions for quantities of interest in MRFs.
Mapping the economy to the some statistical physics models we get strong indications that, in contrary to the pure stock market, the stock market with derivatives could not self-regulate.
This is a review article for Encyclopedia of Complexity and System Science, to be published by Springer http://refworks.springer.com/complexity/. The paper reviews statistical models for money, wealth, and income distributions developed in the econophysics literature since late 1990s.