We consider the inference of the structure of an undirected graphical model in an exact Bayesian framework. More specifically we aim at achieving the inference with close-form posteriors, avoiding any sampling step. This task would be intractable without any restriction on the considered graphs, so we limit our explora…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “closed-form posterior”
We propose a general algorithm for approximating nonstandard Bayesian posterior distributions. The algorithm minimizes the Kullback-Leibler divergence of an approximating distribution to the intractable posterior distribution. Our method can be used to approximate any posterior distribution, provided that it is given i…
A new algorithm optimizes Gaussian process posterior mean functions efficiently.
problem Optimizing Gaussian process posterior mean functions over hyperrectangles is challenging due to nonlinearity and nonconvexity.
method PALM-Mean, a piecewise-analytic lower-bounding framework embedded in reduced-space spatial branch-and-bound.
result PALM-Mean improves scalability for large datasets compared to general-purpose solvers.
Bayesian Learning of Conditional Kernel Mean Embeddings for Automatic Likelihood-Free Inferencestat.ML
KELFI improves inference accuracy in likelihood-free settings with limited simulations.
problem Intractable likelihood evaluations in likelihood-free inference.
method Kernel embedding likelihood-free inference (KELFI) learns model hyperparameters to balance accuracy and efficiency.
result Improved accuracy and efficiency on challenging inference problems in ecology.
Improves posterior approximation speed for Dirichlet process mixture models.
problem Inefficiency of stochastic variational inference in large datasets.
method Uses stochastic gradient ascent with adaptive stepsize optimization.
result Adaptive stepsize improves speed and performance of posterior approximation.
CNR uses convex optimization to estimate conditional distributions.
problem Estimating uncertainty in predictions and posterior conditional distributions.
method Convex optimization of a posterior defined via non-linear transformations on Gaussians.
result CNR can fit arbitrary conditional distributions, including multimodal and non-symmetric ones.
Latent variable time-series models are among the most heavily used tools from machine learning and applied statistics. These models have the advantage of learning latent structure both from noisy observations and from the temporal ordering in the data, where it is assumed that meaningful correlation structure exists ac…
Transfer learning has recently attracted significant research attention, as it simultaneously learns from different source domains, which have plenty of labeled data, and transfers the relevant knowledge to the target domain with limited labeled data to improve the prediction performance. We propose a Bayesian transfer…
LF-IBIS learns optimal policies online without explicit likelihood.
problem Bayesian RL challenges due to intractable likelihood functions.
method Combines ABC with IBIS for online belief updates.
result Approximates posterior distributions for policies and parameters.
Bayesian neural networks improve uncertainty calibration without sacrificing accuracy.
problem Bayesian neural networks struggle with uncertainty calibration and high-dimensional geometry.
method Model uncertainty only in weight directions using a von Mises-Fisher posterior on the unit sphere, deriving a compact KL term.
result A lightweight, dimension-aware variational unit improves calibration without sacrificing accuracy.
Bayesian framework for sphere regression using Gaussian fields.
problem Nonparametric regression on the sphere with Gaussian priors.
method Isotropic Gaussian field priors, harmonic structure, exact posterior distributions, optimal spectral truncation, posterior contraction rates.
result Sharp posterior contraction rates for Gaussian priors with polynomially decaying angular power spectra.
We examine an analytic variational inference scheme for the Gaussian Process State Space Model (GPSSM) - a probabilistic model for system identification and time-series modelling. Our approach performs variational inference over both the system states and the transition function. We exploit Markov structure in the true…
Dead-Direction Signatures (DDS) provide a cheap, closed-form spectral reading of a network's singular complexity.
problem Estimating the complexity of deep networks through their loss singularities.
method DDS replaces the SGLD posterior chain with spectral linear algebra.
result DDS observables rank-track the network's singular complexity at the framework-predicted sign.
Focal loss improves classification but not class-posterior probability estimation.
problem Improving class-posterior probability estimation from focal loss.
method Proved classification-calibration and derived a transformation to recover true class-posterior probabilities.
result A transformation of the confidence score from focal loss minimization allows recovery of true class-posterior probabilities.
Mixed Variational Inferencestat.ML
The Laplace approximation has been one of the workhorses of Bayesian inference. It often delivers good approximations in practice despite the fact that it does not strictly take into account where the volume of posterior density lies. Variational approaches avoid this issue by explicitly minimising the Kullback-Leibler…
CIFs improve VI by providing flexible posteriors for complex topologies.
problem Complex posterior distributions in VI problems.
method Using CIFs as part of an auxiliary VI scheme, exploiting conditional independence.
result CIFs yield low-variance estimators of model evidence and improve VI results.
Kernel methods are one of the mainstays of machine learning, but the problem of kernel learning remains challenging, with only a few heuristics and very little theory. This is of particular importance in methods based on estimation of kernel mean embeddings of probability measures. For characteristic kernels, which inc…
Exact posterior score estimation for solving linear inverse problems
problem Solving linear inverse problems
method Derive the exact posterior score and use it as a denoising training objective
result EPS outperforms training-free and training-based baselines on various metrics
Unified view on GP transfer learning for Bayesian optimization.
problem Improving data efficiency in Bayesian optimization with scarce data.
method Unified hierarchical GP models for transfer learning, including a novel boosted GP transfer model.
result Unified analysis and comparison of transfer learning methods for GP models.
This paper introduces NPR, a technique to improve Bayesian inference for multi-modal, high-dimensional simulations.
problem Challenges in Bayesian inference for multi-modal, high-dimensional simulations.
method Introduces Neural Posterior Regularization (NPR) to enforce exploration of input parameter space.
result Empirically validated that NPR significantly improves performance on various simulation tasks.
VSE estimates complex processes from noisy measurements without a model.
problem Estimating states of complex, model-free processes from noisy data.
method Variational state estimation using recurrent neural networks (RNNs) in both learning and inference phases.
result VSE provides a competitive state estimate for a benchmark process (Lorenz system) compared to known and data-driven methods.
New GP-based method improves uncertainty quantification for causal functions.
problem Challenges in quantifying uncertainty for causal effects, especially for entire functions.
method GP-based approach using inner-product of observational functions in RKHS, with tractable posterior moments and calibration.
result Improves uncertainty quantification while maintaining causal effect estimation performance.
FTIP uses normalizing flows to improve posterior inference in function space.
problem Challenges in posterior inference with implicit-process priors.
method FTIP uses normalizing flows to define a richer variational distribution over combination weights.
result FTIP captures asymmetric and multimodal posterior structure better than Gaussian coefficient approximations.
Proposes robust ITRs integrating multiple datasets to handle posterior shift.
problem Posterior shift in conditional outcome distributions between source and target populations.
method Distributionally robust approach with closed-form solution and adaptive uncertainty tuning.
result Achieves superior performance compared to existing methods in simulations and real-data applications.
New method optimises worst-case risk under model uncertainty.
problem Minimizing expected risk under posterior beliefs leads to sub-optimal decisions due to model uncertainty.
method Distributionally Robust Optimisation with Bayesian Ambiguity Sets (DRO-BAS)
result Improved out-of-sample robustness in the Newsvendor problem.
Unified Skew-Gaussian process framework for various regression and classification tasks.
problem Handling multiple types of regression and classification problems.
method Generalization of Skew-Gaussian processes to handle various types of data and likelihoods.
result Closed-form posterior distributions for multiple tasks.
We define and discuss the first sparse coding algorithm based on closed-form EM updates and continuous latent variables. The underlying generative model consists of a standard `spike-and-slab' prior and a Gaussian noise model. Closed-form solutions for E- and M-step equations are derived by generalizing probabilistic P…
Deep models memorize training data in geophysical inversion, leading to biased posterior distributions.
problem Memorization of training data biases learned priors in geophysical inverse problems.
method Casting generative models' training as maximum likelihood, we show memorization results in a reweighted empirical distribution for diffusion models, leading to Gaussian mixture priors and posteriors.
result Memorization leads to posterior distributions that are likelihood-weighted lookup among stored training examples, affecting full waveform inversion outcomes.
MC Dropout is re-evaluated as not Bayesian, affecting predictive posterior and multimodality.
problem MC Dropout's Bayesian properties and predictive posterior are questioned.
method Re-evaluation of MC Dropout's properties, including a new VI engine in pytorch.
result MC Dropout does not produce a faithful Bayesian predictive posterior.
Adaptive querying learns user psychometrics with AI personas.
problem Learning user psychometrics within query budgets.
method Persona-induced latent variable model with AI personas and large language model response distributions.
result Persona-based posteriors deliver accurate probabilistic predictions.
Study compares chi-squared divergence and KL-divergence posteriors for PAC-Bayesian bounds.
problem Investigates optimal posteriors for PAC-Bayesian bounds using chi-squared divergence.
method Analyzes bounds for three distance functions, derives FP equations for computation.
result Chi-squared divergence based posteriors have weaker bounds and worse test errors.
Improved Thompson Sampling using fractional posteriors achieves better regret bounds.
problem Optimizing regret in stochastic multi-armed bandit problems.
method Using -posterior distributions, derived frequentist regret bounds.
result Instance-dependent and instance-independent regret bounds established.
C-DPS improves diffusion posterior sampling for inverse problems without projection or likelihood approximation.
problem Inaccurate and unstable solutions in inverse problems due to complex or high-noise conditions.
method C-DPS introduces a forward stochastic process in measurement space evolving in parallel with data-space diffusion, leading to a closed-form posterior.
result C-DPS consistently outperforms existing methods across multiple inverse problem benchmarks.
Proposes a method to adapt DNNs to drift in data distribution.
problem Adapting to out-of-distribution data and shifting objectives.
method Bayesian Inference, Variational Density Propagation, Evidence Lower Bound (ELBO), Minimum Description Length (MDL) Principle.
result Minimizes catastrophic forgetting by approximating MDL principle.
The variational autoencoder (VAE) is a powerful generative model that can estimate the probability of a data point by using latent variables. In the VAE, the posterior of the latent variable given the data point is regularized by the prior of the latent variable using Kullback Leibler (KL) divergence. Although the stan…
Sparse Gaussian Processes simplify GP inference for large datasets.
problem Efficiently handling large datasets in Gaussian Process models.
method Sparse Gaussian Processes combined with variational inference.
result Sparse GPs enable approximate inference with reduced memory and computational requirements.
We revisit Rahimi and Recht (2007)'s kernel random Fourier features (RFF) method through the lens of the PAC-Bayesian theory. While the primary goal of RFF is to approximate a kernel, we look at the Fourier transform as a prior distribution over trigonometric hypotheses. It naturally suggests learning a posterior on th…
Bayesian methods are appealing in their flexibility in modeling complex data and ability in capturing uncertainty in parameters. However, when Bayes' rule does not result in tractable closed-form, most approximate inference algorithms lack either scalability or rigorous guarantees. To tackle this challenge, we propose …
Transformers mimic Bayesian reasoning in controlled settings, revealing geometric mechanisms.
problem Verifying if transformers perform Bayesian reasoning rigorously in natural data.
method Constructing Bayesian wind tunnels with known posteriors and proving memorization impossibility.
result Transformers achieve - bit accuracy in Bayesian posteriors, while MLPs fail.
Bayesian method classifies actin cytoskeleton networks using topological data.
problem Classifying the structure of biological networks, especially actin cytoskeleton networks.
method Transform actin cytoskeleton networks into persistence diagrams, quantify variability with Bayesian framework, estimate posterior distributions.
result Bayesian framework successfully classifies actin filament networks, outperforming state-of-the-art methods.
Mean-field variational inference is a method for approximate Bayesian posterior inference. It approximates a full posterior distribution with a factorized set of distributions by maximizing a lower bound on the marginal likelihood. This requires the ability to integrate a sum of terms in the log joint likelihood using …
Paper develops robust Bayesian models for linear regression under adversarial perturbations.
problem Ensuring reliable machine learning models under data perturbations.
method Formulates adversarial Bregman divergence loss, computes adversarial perturbation, introduces adversarially robust posteriors, derives generalization certificates.
result Derives first rigorous generalization certificates for adversarially robust Bayesian linear regression.
Improved VAE model with autoregressive posterior for better image generation.
problem Improving the VAE model to better capture correlated natural signals like images.
method Replaced the standard white Gaussian approximate posterior with a first-order autoregressive Gaussian.
result Significantly improved image generation quality with minimal parameter increase.
Novel MBRL method for large-scale RL with reduced posterior complexity.
problem Theoretical guarantees for MBRL in large spaces with complex models.
method Kernelized Stein Discrepancy for compression of posterior estimate.
result Sublinear Bayesian regret and up to 50% reduction in training time.
PostNet predicts uncertainty without OOD data, improving OOD detection and calibration.
problem Accurate uncertainty estimation for safe systems.
method PostNet uses Normalizing Flows to learn individual posterior distributions over predicted probabilities.
result PostNet achieves state-of-the-art results in OOD detection and uncertainty calibration.
Paper presents a reparameterized DP-DLGMM for clustering.
problem Non-parametric DP priors in DLGMM are hard to couple with variational inference.
method Closed-form updates for DP-DLGMM's variational posterior.
result Model generates realistic samples and performs competitively in semi-supervised settings.
The EM training algorithm of the classical i-vector extractor is often incorrectly described as a maximum-likelihood method. The i-vector model is however intractable: the likelihood itself and the hidden-variable posteriors needed for the EM algorithm cannot be computed in closed form. We show here that the classical …
Optimal posterior distributions improve SVM classifiers and parameter selection.
problem Improving SVM classifiers and selecting optimal regularization parameters.
method PAC-Bayesian approach with optimal posterior identification for stochastic classifiers.
result Optimal posteriors yield tight risk bounds and improved SVM performance.