Paper analyzes the free energy of CNNs with skip connections in Bayesian learning.
problem Dependency of CNNs with skip connections on the number of parameters.
method Examines the Bayesian free energy of CNNs with and without skip connections.
result The upper bound of free energy of Bayesian CNN with skip connections does not depend on overparametrization.
Active inference minimizes expected free energy for optimal behavior.
problem Understanding and optimizing behavior in complex systems.
method Combines Bayesian decision theory, optimal Bayesian design, and the free energy principle.
result Active inference emerges as a unified framework for information-seeking, utility maximization, and goal-directed behavior.
Bayesian free energy remains bounded for deep ReLU networks in overparametrized cases.
problem Understanding the generalization performance of deep ReLU neural networks.
method Analyzes Bayesian free energy in overparametrized deep ReLU neural networks.
result Bayesian free energy is bounded even in overparametrized deep ReLU networks.
Bayesian inference learns free energy landscapes from experimental data.
problem Characterize the free energy landscape of classical many-body systems from experimental data.
method Combines non-parametric Bayesian inference with physically-motivated constraints to automate the construction of approximate free energy functionals.
result Inference algorithms yield a probability distribution over free energy functionals, leading to highly accurate analytic expressions.
Enhances predictive models against misspecification and outliers.
problem Suboptimal generalization under misspecification and outliers.
method Combines PACm ensemble bounds with a generalized logarithm score function. result Produces predictive distributions resistant to both misspecification and outliers.
Study shows TAP free energy minimization provides better posterior inference in high-dimensional linear models.
problem Deviation from true posterior mean and underestimation of posterior uncertainty in variational inference.
method Minimization of TAP free energy in a high-dimensional asymptotic framework, showing geometric and statistical properties.
result Local minimizer of TAP free energy provides consistent estimate of posterior marginals and correctly calibrated posterior inference.
Loopy and generalized belief propagation are popular algorithms for approximate inference in Markov random fields and Bayesian networks. Fixed points of these algorithms correspond to extrema of the Bethe and Kikuchi free energy. However, belief propagation does not always converge, which explains the need for approach…
Bayesian models' singular fluctuation is shown to be akin to specific heat, influencing model complexity and generalization.
problem Understanding the thermodynamic interpretation of singular fluctuation in Bayesian models.
method Showed singular fluctuation as the curvature of Bayesian free energy and variance of log-likelihood observable under a Gibbs posterior.
result Singular fluctuation is the statistical analogue of specific heat, controlling model complexity and generalization.
Agents learn and control complex mechanical systems through shared memories.
problem Controlling multi-joint dynamical systems.
method Coupled autoregressive active inference agents using Bayesian filtering and minimizing expected free energy.
result Demonstrated learning and control of a double mass-spring-damper system.
A statistical model or a learning machine is called regular if the map taking a parameter to a probability distribution is one-to-one and if its Fisher information matrix is always positive definite. If otherwise, it is called singular. In regular statistical models, the Bayes free energy, which is defined by the minus…
New method estimates Nishimori temperature for node classification in weighted graphs.
problem Estimating Nishimori temperature for Bayesian inference.
method Spectral method using eigenvalues of Bethe Hessian matrix.
result Spectral method outperforms existing approaches in node classification.
Proposes a new method to estimate Bayesian neural network depth.
problem Estimating the depth of Bayesian neural networks.
method Uses a discrete truncated normal distribution to learn depth mean and variance, inferring posterior distributions by minimizing variational free energy.
result Improves test accuracy and reduces posterior depth variance on the spiral dataset.
The heuristic identification of peaks from noisy complex spectra often leads to misunderstanding of the physical and chemical properties of matter. In this paper, we propose a framework based on Bayesian inference, which enables us to separate multipeak spectra into single peaks statistically and consists of two steps.…
Inference problems in graphical models can be represented as a constrained optimization of a free energy function. It is known that when the Bethe free energy is used, the fixedpoints of the belief propagation (BP) algorithm correspond to the local minima of the free energy. However BP fails to converge in many cases o…
FEAT estimates free energy using adaptive transports.
problem Estimating free energy across scientific domains.
method Uses learned transports and stochastic interpolants.
result Provides consistent, minimum-variance estimators.
Derives TAP approximation for Bayesian linear regression.
problem Log-normalizing constant of posterior distribution in high-dimensional linear regression.
method Variational representation and Thouless-Anderson-Palmer approximation.
result Proves TAP approximation for spherical prior in proportional asymptotic regime.
AIF improves physical AI agents' performance in dynamic environments.
problem Physical AI agents are less capable than biological agents in open-ended real-world environments.
method Developed from probability theory, Bayesian machine learning, variational inference, and Active Inference (AIF), grounded in the Free Energy Principle.
result AIF minimizes variational free energy and is well-suited to physical constraints.
mAIS improves free energy evaluation efficiency.
problem Computational infeasibility of exact free energy evaluation.
method mAIS, a marginalized version of AIS.
result mAIS is more efficient under certain conditions.
This paper develops efficient federated learning and unlearning methods in Bayesian models.
problem Managing epistemic uncertainty and legal right to be forgotten in decentralized networks.
method Develops federated variational inference solutions based on decentralized local free energy minimization.
result Demonstrates efficient unlearning mechanisms in federated learning and unlearning.
We introduce thermodynamic response functions for singular Bayesian models.
problem Singular Bayesian models violate regular asymptotics due to non-identifiability and degenerate Fisher geometry.
method Posterior tempering induces thermodynamic response functions, linking WAIC, WBIC, and singular fluctuation.
result WAIC, WBIC, and singular fluctuation are unified within a thermodynamic response framework.
Density estimation is a fundamental problem in statistical learning. This problem is especially challenging for complex high-dimensional data due to the curse of dimensionality. A promising solution to this problem is given here in an inference-free hierarchical framework that is built on score matching. We revisit the…
Deep neural networks improve free energy calculations for peptide conformations.
problem Challenges in developing suitable mappings for free energy perturbation.
method Adapted machine learning approach to train deep neural networks for mapping between Boltzmann distributions.
result Accurate free energy differences calculated between thermodynamic states with spring centers separated by 1 Å and sometimes 2 Å.
We use matricial free energy to regularize autoencoders, producing Gaussian-like codes.
problem Generating Gaussian-like codes for autoencoders.
method Define a differentiable loss function based on singular values of the code matrix, minimizing matricial free energy.
result Minimizing matricial free energy results in Gaussian-like codes that generalize.
Study on existence of ground states for free energy on hyperbolic space.
problem Existence of ground states for a free energy functional on hyperbolic space.
method Derived HLS-type inequalities on Cartan-Hadamard manifolds to prove existence.
result Established conditions for the existence of ground states on hyperbolic space.
New result on critical points of Bethe free energy under deformation retracts.
problem Characterizing critical points of Bethe free energy for complex graphs.
method Analyzing homotopy types and deformation retracts of factor graphs.
result Critical points of Bethe free energy are invariant under deformation retracts.
New model estimates Gibbs free energies using machine learning and isobaric-isothermal flows.
problem Estimating Gibbs free energies for complex systems.
method Normalizing flows trained to sample isobaric-isothermal ensemble.
result Excellent agreement with established baselines for water phases.
Study of phase separation and geometry on a closed elastic curve, including dynamics and free energy minimization.
problem Free energy and dynamics of a closed elastic filament coupled to a scalar concentration field.
method Analytical and numerical simulations of coupled Willmore flow and Cahn--Hilliard gradient flow on differential geometry.
result Qualitative changes in free energy landscape due to closure constraint, leading to metastable and stable multi-domain morphologies.
SBO uses dual voting to build consensus in noisy feedback settings.
problem Collective decision-making under social influence.
method Dual voting system with noisy public votes and private interviews.
result Efficient consensus-building through noisy public votes, debiased by estimated social graph.
Bayesian calibration for BCP self-assembly models using image data and measure transport.
problem Calibrating models of BCP self-assembly from image data with aleatory uncertainty.
method Likelihood-free inference via measure transport and summary statistics.
result Expected information gains can be computed efficiently for model calibration.
DSVGD improves federated learning with fewer communication rounds.
problem Federated learning scalability and trustworthiness.
method Distributed Stein Variational Gradient Descent (DSVGD) for non-parametric Bayesian inference.
result DSVGD achieves comparable accuracy and scalability to other methods, with well-calibrated predictions.
A neural network model minimizes region-based free energy for faster inference in MRFs.
problem Efficient inference in complex Markov random fields (MRFs).
method Region-based Energy Neural Network (RENN) that directly minimizes region-based free energy.
result RENN outperforms other methods in marginal distribution estimation, partition function estimation, and MRF learning.
We address the problem of computing approximate marginals in Gaussian probabilistic models by using mean field and fractional Bethe approximations. We define the Gaussian fractional Bethe free energy in terms of the moment parameters of the approximate marginals, derive a lower and an upper bound on the fractional Beth…
This work improves Gaussian process regression for large, non-stationary data.
problem Scalability issues and performance degradation for non-stationary data.
method Combines variational free energy approximations with online expectation propagation and local splitting steps.
result Incremental adaptation to locality, heterogeneity, and non-stationarity in training data.
Non-negative matrix factorization (NMF) is a knowledge discovery method that is used in many fields. Variational inference and Gibbs sampling methods for it are also wellknown. However, the variational approximation error has not been clarified yet, because NMF is not statistically regular and the prior distribution us…
Study finds surfaces in spherical caps that maximize modified energy.
problem Geometry of surfaces with free boundaries and capillary conditions.
method Monotonicity formulae and energy maximization analysis.
result Capillary minimal surfaces maximize a modified energy in their conformal orbit.
Bayesian Predictive Coding improves deep learning uncertainty quantification.
problem Limitations of maximum a posteriori and maximum likelihood estimates in predictive coding.
method Developed Bayesian Predictive Coding (BPC) that estimates a posterior distribution over network parameters.
result BPC offers comparable uncertainty quantification to existing methods in Bayesian deep learning and improves convergence properties.
Paper proposes energy objective for training normalizing flows without determinants.
problem Challenges in training normalizing flows due to Jacobian determinants.
method Introduces energy objective based on proper scoring rules, determinant-free.
result Energy objective supports novel model families and competitive performance.
Study free energy in spherical spin glasses, proving universality dichotomy.
problem Analyzing free energy in spherical spin glass models with different tail exponents.
method Introduced a tail-adapted normalization and used universality dichotomy.
result Sharp universality dichotomy for free energy across different tail exponents.
Paper improves zero-shot protein stability prediction by clarifying free-energy foundations.
problem Improving zero-shot protein stability prediction using inverse folding models.
method Clarifying the free-energy foundations of inverse folding models and proposing better estimates of relative stability.
result Significant gains in zero-shot performance can be achieved with simple methods.
Unified framework for planning under uncertainty using variational inference.
problem Planning under uncertainty with separate objectives for exploration and exploitation.
method Variational inference on a generative model augmented with priors.
result EFE-based planning emerges as variational inference, enabling scalable, resource-aware policies.
This paper generalizes neural transport learning for free energy estimation in arbitrary state spaces.
problem Efficient estimation of free energy in various state spaces.
method Generalized neural transport learning approach for arbitrary state spaces.
result Validation of the proposed method's effectiveness and efficiency in diverse settings.
New algorithms learn latent variable models without tuning, outperforming existing methods.
problem Learning latent variable models without manual tuning.
method Two particle-based algorithms using free energy minimization and coin betting.
result Learning algorithms are entirely tuning-free and competitive with existing methods.
New approach infers object type from physical data and functional requirements.
problem Classical design theory treats object type as a given, but this paper argues it should be inferred.
method C-DMBD, a constrained extension of Dynamic Markov Blanket Detection algorithm, models product surface as a Markov blanket.
result Requirement-steered inference is computationally tractable and models different object types with different parameterizations.
The free energy of a closed 3-manifold is a 2-parameter formal power series which encodes the perturbative Chern-Simons invariant (also known as the LMO invariant) of a closed 3-manifold with gauge group U(N) for arbitrary N. We prove that the free energy of an arbitrary closed 3-manifold is uniformly Gevrey-1. As a …
The free energy functional has recently been proposed as a variational principle for bounded rational decision-making, since it instantiates a natural trade-off between utility gains and information processing costs that can be axiomatically derived. Here we apply the free energy principle to general decision trees tha…
Advances variational Bayesian neural networks using singular learning theory.
problem Discrepancies between predictive performance and variational objective in BNNs.
method Corrected asymptotic form of singular posterior distributions to inform variational family design.
result Improvements in variational free energy and generalization error with proposed normalizing flow.
Straight lines are a basin of attraction for the elastic flow at least to level 1.9615π.
problem Understanding the basin of attraction for the free boundary free elastic flow.
method Steepest descent gradient flow for elastic energy, numerical evidence.
result Straight lines have a basin of attraction at least to level 1.9615π.
Bayesian model for energy consumption helps electric vehicles navigate efficiently.
problem Limited battery capacity in electric vehicles makes energy efficient navigation challenging.
method Developed an online learning framework using Bayesian models and exploration strategies like Thompson Sampling.
result Established rigorous regret bounds for Thompson Sampling in both single-agent and multi-agent settings.