Online distributional prediction with latent cluster geometry
problem Predicting the full data-generating distribution in non-stationary streams
method Representing candidate laws as latent cluster geometry and using Gibbs quasi-posterior
result Achieving sublinear cumulative Wasserstein regret under bounded support and stable latent geometry
Hamiltonian Monte Carlo is a powerful algorithm for sampling from difficult-to-normalize posterior distributions. However, when the geometry of the posterior is unfavorable, it may take many expensive evaluations of the target distribution and its gradient to converge and mix. We propose neural transport (NeuTra) HMC, …
This work explores how overparametrization and priors affect Bayesian neural network posteriors.
problem Symmetries, non-identifiabilities, and weight-space priors fragment and inflate BNN posteriors.
method We study the interplay between overparametrization and priors in BNN posteriors, deriving key phenomena and validating through experiments.
result Overparametrization induces structured, prior-aligned weight posterior distributions.
Geometric framework analyzes bias in variational inference for posterior functionals.
problem Analyzing the bias of posterior functionals under variational approximations.
method Developed a geometric framework to evaluate the bias of posterior functionals using the variational tangent space.
result The leading-order bias of a posterior functional is determined by its component orthogonal to the variational tangent space.
This paper introduces a scalable benchmark for evaluating local posterior sampling in neural networks.
problem Degeneracy in neural network loss landscapes and its impact on SGMCMC algorithms.
method Development of a scalable benchmark for local posterior sampling.
result RMSProp-preconditioned SGLD is most effective at representing the local geometry of the posterior distribution.
In this paper we investigate the geometry of the likelihood of the unknown parameters in a simple class of Bayesian directed graphs with hidden variables. This enables us, before any numerical algorithms are employed, to obtain certain insights in the nature of the unidentifiability inherent in such models, the way pos…
We introduce thermodynamic response functions for singular Bayesian models.
problem Singular Bayesian models violate regular asymptotics due to non-identifiability and degenerate Fisher geometry.
method Posterior tempering induces thermodynamic response functions, linking WAIC, WBIC, and singular fluctuation.
result WAIC, WBIC, and singular fluctuation are unified within a thermodynamic response framework.
Bayesian neural networks approximate Gaussian, this method adapts to non-Gaussian posteriors.
problem Bayesian neural networks struggle with non-Gaussian posteriors, leading to poor performance.
method Proposes a Riemannian Laplace approximation to adapt to the shape of the true posterior.
result Consistently improves over conventional Laplace approximation across tasks.
This paper proposes a new geometric framework for asset pricing.
problem The asymmetry between risk-neutral and physical measures in asset pricing.
method Information geometry, focusing on the relativity of probabilistic reference frames.
result Unified explanation for price fluctuations, event-driven behavior, and risk premia.
GeoFunFlow tackles inverse problems on complex geometries with efficient learning.
problem Challenges in inverse problems governed by PDEs, especially on irregular geometries.
method Combines geometric function autoencoder and latent diffusion model trained via rectified flow.
result Achieves state-of-the-art reconstruction accuracy and efficient inference.
Bayesian models for networks are often misspecified, leading to overconfident inference.
problem Real-world networks violate assumptions of geometry and link function in latent space models.
method Proposes a generalized posterior framework for random geometric graphs, using Link-Sequential R-SafeBayes to adaptively tune posterior regularization.
result Improved calibration and better link prediction performance demonstrated on synthetic and real-world networks.
Geometry-aware KDE model improves multiclass quantification.
problem Accurately estimating class prevalence for label shift adaptation.
method Log-ratio representations and Aitchison geometry for compositional data, shrinkage regularization.
result Competitive with state-of-the-art quantifiers, often improving over standard KDE-based baselines.
Bayesian neural networks improve uncertainty calibration without sacrificing accuracy.
problem Bayesian neural networks struggle with uncertainty calibration and high-dimensional geometry.
method Model uncertainty only in weight directions using a von Mises-Fisher posterior on the unit sphere, deriving a compact KL term.
result A lightweight, dimension-aware variational unit improves calibration without sacrificing accuracy.
Flexible VAEs using FIFs improve model likelihood on image datasets.
problem Limitations of diagonal Gaussian posteriors in VAEs.
method Regularized Free-form Injective Flow (FIF) for flexible posterior.
result Full covariance VAEs outperform diagonal Gaussian posteriors.
GP-PCA reduces infinite-dimensional GP posteriors to a finite space for meta-learning.
problem How to define a structure for a set of Gaussian process posteriors.
method Information geometric framework and variational inference.
result GP-PCA improves meta-learning performance through reduced GP posteriors.
New method speeds up Bayesian inverse problem solving with neural operators.
problem Solving infinite-dimensional Bayesian inverse problems with high computational cost.
method Delayed-acceptance geometric MCMC driven by derivative-informed neural operator surrogates.
result Significant speedup in generating posterior samples (3-9 times faster).
Unified framework for isotropic SG noise in posterior sampling.
problem Bayesian posterior sampling with practical and robust methods.
method Designing a novel, isotropic SG noise approach with fixed learning rate.
result Competitive and practical method compared to state-of-the-art.
New MIF architecture improves posterior approximations in Bayesian models.
problem Challenges in variational inference for complex hierarchical models.
method Combines VIP and autoregressive flow with prior information and hierarchical ordering.
result Empirically, MIF delivers tighter posterior approximations and state-of-the-art performance.
The paper analyzes and proposes a new stopping criterion for recursive Bayesian classification.
problem Limitations of conventional stopping criteria in recursive Bayesian classification.
method Geometric interpretation of state posterior progression and analysis of conventional criteria.
result Proposes a new stopping criterion to overcome limitations of conventional methods.
Transformers mimic Bayesian reasoning in controlled settings, revealing geometric mechanisms.
problem Verifying if transformers perform Bayesian reasoning rigorously in natural data.
method Constructing Bayesian wind tunnels with known posteriors and proving memorization impossibility.
result Transformers achieve 10−3-10−4 bit accuracy in Bayesian posteriors, while MLPs fail. SoftCVI uses contrastive estimation to infer complex posteriors.
problem Estimating complex posteriors in Bayesian inference.
method Contrastive variational inference with self-generated soft labels.
result SoftCVI outperforms other variational approaches in stability and coverage.
Improves Laplace approximation for Bayesian inference on Riemannian manifolds.
problem Inaccurate Gaussian approximations for complex targets and finite-data posteriors.
method Develops alternative variants of the Laplace approximation using a Riemannian metric.
result Exact approximations at the limit of infinite data, improving practical performance.
Boundary effects inflate variance in Gaussian processes, leading to acquisition bias.
problem Boundary-induced acquisition bias in Gaussian processes.
method Traced root cause to geometric mechanism of kernel truncation at domain boundaries.
result Boundary effects create distortion that worsens with dimensionality, affecting acquisition behavior.
Variational Autoencoder (VAE), a simple and effective deep generative model, has led to a number of impressive empirical successes and spawned many advanced variants and theoretical investigations. However, recent studies demonstrate that, when equipped with expressive generative distributions (aka. decoders), VAE suff…
Enhances graph modeling with hyperbolic geometry and variational inference.
problem Challenges in modeling relational data with complex dependencies.
method Semi-implicit hierarchical variational Bayes with Poincaré embedding and mutual information regularization.
result Improves graph representation quality and flexibility in edge prediction and node classification.
New method uses quotient predictor space for better PAC-Bayes bounds, reducing KL divergence and improving model performance.
problem Overparameterized models with continuous symmetries can lead to biased predictions.
method Perform PAC-Bayesian analysis on quotient predictor space, constructing a canonical prior that reflects model's implicit bias.
result The new prior reduces KL divergence and improves model performance in experiments.
New samplers minimize KL divergence for constrained and non-Euclidean geometries.
problem Efficient sampling from constrained and non-Euclidean distributions.
method Stein Variational Mirror Descent and Mirrored Stein Variational Gradient Descent.
result New samplers converge more rapidly and accurately than prior methods.
A new method normalizes flow mixtures for better inference across different data types.
problem Inference failure across diverse posterior geometries in normalizing flows.
method Introduces a two-stage framework with a stable global weighting mechanism based on sEMA.
result Achieves consistent NLL improvements and stable weight trajectories over baselines.
Paper proposes a probabilistic alignment method for domain adaptation.
problem Latent distribution mismatch and miscalibrated uncertainty in adapting large-scale models.
method Bayesian latent transport framework with PAC-Bayesian regularization.
result Reduction in latent manifold discrepancy and improved uncertainty calibration.
A new geometric concept, the dead direction, bridges singular learning theory and information geometry.
problem The gap between singular learning theory and information geometry.
method Introducing the dead direction, a unit vector along degenerating Fisher metric, and showing its KL order can be recovered.
result The KL order of the dead direction can be recovered as the decay rate of the directional Fisher curvature, providing a handle on singular geometry.
A method to automatically and symbolically detect and resolve degenerate parameter combinations from parameter-data pairs.
problem Identifying degenerate parameter combinations in physical models or real-world datasets.
method The degeneracy distillery method detects and resolves degenerate parameter combinations from parameter-data pairs.
result The method reduces the simulation budget required for downstream neural posterior estimation.
Paper analyzes latent space geometry in generative models using Fisher information.
problem Understanding the structure of latent spaces in generative models.
method Reconstructs Fisher information metric from generated samples and posterior distribution.
result Reveals fractal structure and abrupt changes in Fisher metric at phase boundaries.
State space models (SSM) have been widely applied for the analysis and visualization of large sequential datasets. Sequential Monte Carlo (SMC) is a very popular particle-based method to sample latent states from intractable posteriors. However, SSM is significantly influenced by the choice of the proposal. Recently Ha…
GABI learns geometry from diverse systems to improve Bayesian inference.
problem Bayesian inversion of physical systems with varying geometries.
method Geometric Autoencoders for Bayesian Inversion (GABI) learns geometry-aware priors from large datasets.
result GABI yields comparable predictive accuracy to deterministic methods and well-calibrated uncertainty quantification.
FP-BMA improves generalization by encouraging flat posteriors in Bayesian Model Averaging.
problem Lack of flat posterior in approximate Bayesian inference methods hinders effective Bayesian Model Averaging.
method Proposes Flat Posterior-aware Bayesian Model Averaging (FP-BMA) and Flat Posterior-aware Bayesian Transfer Learning schemes.
result FP-BMA successfully captures flat posteriors, improving generalization performance.
A new method reduces complexity of normalizing flows for MCMC preconditioning.
problem Improving sampling efficiency in MCMC algorithms for complex target distributions.
method Factorized preconditioning architecture combining a linear component and a conditional NF.
result Significantly better tail samples and higher effective sample sizes on various distributions.
Unified PAC-Bayesian framework for deep learning generalization.
problem Limitations of existing PAC-Bayesian norm-based bounds for deep neural networks.
method Unified framework using anisotropic Gaussian posteriors and sensitivity matrix.
result Comparable or tighter generalization bounds compared to state-of-the-art approaches.
New research shows CPE only occurs when Bayesian posterior underfits.
problem Model misspecification leading to CPE under perfect model specification.
method Theoretical analysis of Bayesian posterior and underfitting.
result No CPE if there is no underfitting of the Bayesian posterior.
Differential privacy of Gaussian process posterior sampling
problem Privacy of posterior sample paths from Gaussian process
method Intrinsic randomness yields DP guarantees
result Intrinsic randomness yields DP guarantees
GS-B3SE improves label shift estimation by smoothing priors on a graph.
problem Label shift adaptation when source and target distributions share conditional but not marginal probabilities.
method Graph-Smoothed Bayesian Black-Box Shift Estimator (GS-B3SE) places Laplacian-Gaussian priors on log-priors and confusion-matrix columns tied by a label-similarity graph. result GS-B3SE produces a tractable posterior with HMC or Newton-CG schemes, proving identifiability, contraction, and robustness. Theoretical framework for M-posteriors connects Bayesian and frequentist statistics.
problem Connecting Bayesian and frequentist approaches in statistical inference.
method Developed a theoretical framework for M-posteriors, showing asymptotic normality and frequentist consistency.
result M-posteriors are robust and contract around M-estimators under mild conditions.
New method improves generative model performance by fully conditioning variational posteriors.
problem Inaccurate inference due to partial conditioning of variational posteriors in sequential LVMs.
method Introduces fully-conditioned approximate posteriors to improve generative model performance.
result Improves generative modelling and multi-step prediction performance.
PVI seeks a posterior that makes predictions closer to true data, not approximating the Bayesian posterior.
problem Finding meaningful posterior distributions under model misspecification.
method Predictive variational inference (PVI) seeks an optimal posterior density for close predictive matching to true data.
result PVI learns a posterior that is not the same as the Bayesian posterior, but is closer to the true data generating process.
Optimized α-posteriors reduce KL divergence from true posterior in parametric misspecification.
problem Reduction of KL divergence from true posterior in parametric model misspecification.
method Derivation of Bernstein-von Mises theorem and optimization of α-posteriors. result Optimized α-posteriors minimize KL divergence from true posterior, especially in severe misspecification. New priors can update posteriors without re-estimating likelihoods.
problem Degradation of classification approaches when class priors change.
method Recompute posteriors using recovered likelihoods from original posteriors and new priors.
result Dynamic update of original posteriors is possible without re-estimating likelihoods.
Bayesian learning made scalable with posteriors library.
problem Computational challenges in Bayesian learning with modern models.
method Introducing posteriors library and tempered MCMC.
result Bayesian approximations are useful and scalable.
The representation of the approximate posterior is a critical aspect of effective variational autoencoders (VAEs). Poor choices for the approximate posterior have a detrimental impact on the generative performance of VAEs due to the mismatch with the true posterior. We extend the class of posterior models that may be l…
New decision-theoretic characterization separates belief and decision posteriors.
problem Understanding the conditions under which loss-based updating coincides with Bayesian updating.
method Decision-theoretic approach to distinguish belief and decision posteriors.
result Generalized Bayes coincides with ordinary Bayesian updating only if the loss is proportional to negative log-likelihood.