The paper analyzes distributed Bayesian inference and its Frequentist guarantees.
problem Analyzing large decentralized datasets with distributed Bayesian inference.
method Establishes Frequentist properties for distributed (non-)Bayesian inference.
result Distributed Bayesian inference retains parametric efficiency and enhances robustness.
Paper develops a scalable distributed inference algorithm for sensor networks.
problem Efficient inference in intelligent sensor networks for location, tracking, and mapping.
method Distributed variational inference algorithm for continuous variables and large-scale data.
result Derives a separable lower bound for distributed variational inference with one-hop communication.
Inference for normal and Monte Carlo distributions using minimum relative entropy.
problem Inference from partial information on expectations and covariances.
method Minimum relative entropy sub-manifolds, analytical formulas, Monte Carlo simulations.
result Improved numerical implementation for inference from partial information.
Improved variational inference for geophysical inverse problems with data correction.
problem High computational cost and accuracy issues in Bayesian inference for geophysical inverse problems.
method Amortized variational inference with latent distribution correction using physics-based priors.
result Improved robustness of amortized variational inference under data distribution shifts.
Fast Bayesian inference with adaptable priors for real-time applications.
problem Intractable exact posterior computation limits Bayesian inference's adoption.
method Distribution Transformer architecture that learns mappings between priors and posteriors.
result Significant reduction in computation time from minutes to milliseconds.
A new method learns posterior and predictive distributions together, reducing computational cost.
problem Sequential two-stage Bayesian inference is computationally expensive.
method Amortized variational inference targeting posterior-predictive distribution.
result Efficient online inference with more accurate predictive distributions.
SIXO improves inference by learning smoothing distributions from all observations.
problem Inference limitations due to ignoring future observations in filtering distributions.
method Density ratio estimation to warp filtering distributions into smoothing distributions, then use SMC with learned targets.
result Proves tighter log marginal lower bounds and more accurate inferences and estimates.
The paper introduces a DRM for causal inference, offering a flexible method to analyze counterfactual distributions.
problem Estimating mean causal effects is limited; a distributional perspective is needed for a more thorough understanding.
method The paper employs a semiparametric density ratio model (DRM) with an empirical likelihood (EL) approach to estimate counterfactual distribution functions.
result The DRM framework enables direct and transparent causal inference from a distributional perspective, validated by numerical studies.
Meta-learn Bayesian inference for task-specific BNNs using amortised inference.
problem Efficiently learning Bayesian inference for small-scale probabilistic meta-learning.
method Replace global inducing points with actual data to create a set of approximate likelihoods, train a meta-model to learn these parameters across related datasets.
result Meta-learned inference can be applied to task-specific BNNs, improving efficiency and scalability.
This paper speeds up inference in large hierarchical models.
problem Inference in large hierarchical models is slow and difficult.
method Amortized variational inference with shared parameters.
result Amortized inference is as accurate as a full-rank Gaussian but much faster.
SNVI combines likelihood estimation with variational inference for efficient Bayesian inference.
problem Bayesian inference in models with intractable likelihoods.
method Sequential Neural Variational Inference (SNVI) that combines likelihood-estimation with variational inference.
result SNVI is more computationally efficient than previous algorithms without sacrificing accuracy.
Paper develops online statistical inference methods for stochastic optimization using Kiefer-Wolfowitz algorithms.
problem Online statistical inference of model parameters in stochastic optimization problems.
method Kiefer-Wolfowitz algorithm with random search directions, asymptotic distribution analysis.
result Developed valid confidence intervals for online statistical inference.
Deep Gaussian Processes improve likelihood-free inference for complex distributions.
problem Limited flexibility of Bayesian Optimization with GPs for multimodal distributions.
method Proposes Deep Gaussian Processes (DGPs) as a surrogate model for likelihood-free inference.
result DGPs outperform GPs on multimodal distributions while maintaining comparable performance on unimodal cases.
Adaptive method for prediction sets under changing data distributions.
problem Forming prediction sets in an online setting with varying data distributions.
method Adaptive conformal inference that re-estimates the distribution shift parameter over time.
result Adaptive method achieves desired coverage frequency over long-time intervals.
A new method combines scores of individual observations to efficiently approximate posterior distributions.
problem Handling posterior distributions conditioned on multiple observations with neural methods.
method Conditional score modeling to combine learned scores from individual observations.
result Sample-efficient method that can aggregate multiple observations at inference time.
Federated Learning aims to train distributed deep models without sharing the raw data with the centralized server. Similarly, in distributed inference of neural networks, by partitioning the network and distributing it across several physical nodes, activations and gradients are exchanged between physical nodes, rather…
Improved inference for models with continuous latent variables.
problem Inference accuracy with traditional variational methods is limited.
method Reparameterized Variational Rejection Sampling (RVRS) using a proposal distribution with a reparameterized gradient estimator.
result RVRS offers a better trade-off between computational cost and inference fidelity.
SFSVI uses Gaussian mixtures to approximate neural network outputs for continual learning.
problem Learning new tasks without forgetting old ones in neural networks.
method Sequential function-space variational inference with Gaussian mixture approximation.
result Gaussian mixture SFSVI outperforms other methods in continual learning.
New distribution simplifies covariance matrix inference.
problem Efficient inference for covariance matrices in large models.
method Incorporates Inverse G-Wishart distribution for variational message passing.
result Elegant and succinct expression of variational message passing fragments.
Paper improves statistical efficiency of median-of-means estimator for Byzantine robust distributed inference.
problem Byzantine robustness in distributed learning systems.
method Variance reduced median-of-means (VRMOM) estimator for Byzantine robust distributed inference.
result Achieves a fast convergence rate with only a constant number of rounds of communications.
This paper offers a distribution-free method for post-detection changepoint localization.
problem Locating the exact time of a change in distribution after a sequential detection procedure.
method A distribution-free framework using conformal test martingales for sequential change detection and post-detection inference.
result Valid post-detection coverage guarantees and non-asymptotic bounds on confidence set size.
ALFI improves likelihood-free inference for black-box generators.
problem Limitations of likelihood-free inference on black-box generators.
method Adversarial Likelihood-Free Inference (ALFI) to estimate posterior distributions.
result ALFI achieves best parameter estimation accuracy with limited simulation.
Paper constructs unfaithful probability distributions in binary causal graphs.
problem Unfaithful probability distributions in binary causal graphs.
method Constructs unfaithful probability distributions in binary causal graphs.
result Examples of unfaithful probability distributions in binary causal graphs.
This work improves variational inference by reducing gradient variance.
problem Hard optimization of flexible variational distributions.
method Control variate based on quadratic approximation of the model's mean and covariance.
result Significant improvement in gradient variance and optimization convergence.
In this letter we borrow from the inference techniques developed for unbounded state-cardinality (nonparametric) variants of the HMM and use them to develop a tuning-parameter free, black-box inference procedure for Explicit-state-duration hidden Markov models (EDHMM). EDHMMs are HMMs that have latent states consisting…
Transformers infer tasks from context via two modes, geometrically shaped task vectors explain their behavior.
problem Understanding how transformers infer tasks from context and the geometric properties of task vectors.
method Synthetic setting to train small transformers, mathematical characterization of task-vector geometry and inference modes.
result Task-vector geometry shapes in-distribution and out-of-distribution behavior of transformers.
Neural Empirical Bayes estimates source distributions from noisy simulations.
problem Estimating source distributions from noisy, simulated data.
method Uses neural density estimators to estimate a prior or source distribution over uncorrupted samples, then performs posterior inference.
result Recovering ground truth source distributions up to symmetries.
SymmPI predicts unobserved values under group symmetries, improving over existing methods.
problem Quantifying uncertainty in predictions under group symmetries.
method Distributional equivariant transformations to preserve symmetries.
result SymmPI provides valid coverage and performs favorably in simulations and empirical data.
Two novel distributed VB algorithms improve Bayesian inference in sensor networks.
problem Efficient inference in Bayesian frameworks for sensor networks.
method Two novel distributed VB algorithms for general Bayesian inference, using stochastic natural gradient and ADMM.
result Distributed algorithms perform nearly as well as centralized ones, demonstrating excellent performance.
Mixture modelling involves explaining some observed evidence using a combination of probability distributions. The crux of the problem is the inference of an optimal number of mixture components and their corresponding parameters. This paper discusses unsupervised learning of mixture models using the Bayesian Minimum M…
This work improves the accuracy of OEF and DBV maps from qBOLD MRI data.
problem Noisy and inaccurate inference of OEF and DBV from qBOLD MRI data.
method Flexible amortized variational inference using a scaled multivariate logit-Normal distribution.
result Smooth and physiologically plausible OEF and DBV maps inferred with fewer training data.
Unified framework for simulation-based inference learns a single model for multiple tasks.
problem Simulation-based inference for multiple tasks with limited model retraining.
method Unified flow-matching generative model with query-aware masking distribution.
result Competitive performance on various inference tasks and real-world problems.
As inductive inference and machine learning methods in computer science see continued success, researchers are aiming to describe ever more complex probabilistic models and inference algorithms. It is natural to ask whether there is a universal computational procedure for probabilistic inference. We investigate the com…
PIVID infers DAG structures from data using variational inference and permutations.
problem Estimating the structure of Bayesian networks from observational data.
method PIVID uses variational inference and continuous relaxations of discrete distributions to infer a distribution over permutations and DAGs.
result PIVID outperforms deterministic and Bayesian approaches in estimating DAG structures from data.
Variational Prediction simplifies Bayesian inference without test time costs.
problem Bayesian inference's computational costs and posterior predictive distribution marginalization.
method Variational Prediction learns a variational approximation to the posterior predictive distribution using a variational bound.
result Directly learns a variational approximation to the posterior predictive distribution without test time marginalization costs.
To train an inference network jointly with a deep generative topic model, making it both scalable to big corpora and fast in out-of-sample prediction, we develop Weibull hybrid autoencoding inference (WHAI) for deep latent Dirichlet allocation, which infers posterior samples via a hybrid of stochastic-gradient MCMC and…
The mean field methods, which entail approximating intractable probability distributions variationally with distributions from a tractable family, enjoy high efficiency, guaranteed convergence, and provide lower bounds on the true likelihood. But due to requirement for model-specific derivation of the optimization equa…
Electrostatics method samples complex distributions deterministically.
problem Sampling and inference of complex, high-dimensional distributions.
method Electrostatics-based particle system with Newton mechanics principles.
result Method achieves comparable performance to other methods in benchmark tasks.
Paper simplifies data carving inference with a parametric distribution.
problem Valid inference after selection with data carving.
method Developed a parametric distribution for data carving inference.
result Exact inference for data carving can be computed trivially.
Proposes two-stage robust and sparse distributed inference for large-scale data.
problem Statistical inference in large-scale, high-dimensional, and outlier-contaminated data.
method Two-stage approach: model selection with robust Lasso, fusion of local selections, and bootstrap methods for inference.
result Robust and computationally efficient inference procedures for variable selection, confidence intervals, and standard deviation approximations.
We develop a general variational inference method that preserves dependency among the latent variables. Our method uses copulas to augment the families of distributions used in mean-field and structured approximations. Copulas model the dependency that is not captured by the original variational distribution, and thus …
The paper proposes a method to improve Bayesian inference for periodic data using data-driven priors.
problem Efficiency in approximating posterior distribution in models with periodicity.
method Construct a prior distribution from data using a Gaussian process with a periodic kernel, approximated using adaptive importance sampling.
result The proposed method improves the marginal posterior distribution of the period parameter.
Stacking improves inference for multimodal Bayesian posterior distributions.
problem Difficulty of MCMC in moving between modes and underestimation of posterior uncertainty.
method Parallel runs of MCMC, variational, or mode-based inference, combined using Bayesian stacking.
result Stacking efficiently samples from multimodal posterior distributions and represents uncertainty better than variational inference.
Bayesian inference is used extensively to infer and to quantify the uncertainty in a field of interest from a measurement of a related field when the two are linked by a physical model. Despite its many applications, Bayesian inference faces challenges when inferring fields that have discrete representations of large d…
SoftCVI uses contrastive estimation to infer complex posteriors.
problem Estimating complex posteriors in Bayesian inference.
method Contrastive variational inference with self-generated soft labels.
result SoftCVI outperforms other variational approaches in stability and coverage.
Gradient-based approximate inference methods, such as Stein variational gradient descent (SVGD), provide simple and general-purpose inference engines for differentiable continuous distributions. However, existing forms of SVGD cannot be directly applied to discrete distributions. In this work, we fill this gap by propo…
The paper proposes a method for better uncertainty estimation in neural networks.
problem Estimating predictive uncertainty in neural networks is crucial but challenging.
method The paper proposes a function-space variational inference method to infer a posterior distribution over functions.
result The proposed method leads to state-of-the-art uncertainty estimation and predictive performance.
A new method improves SVI for high-dimensional, poorly-conditioned distributions.
problem Challenges in existing SVI methods for high-dimensional, poorly-conditioned distributions.
method Trust-region optimization approach leveraging conditional independences and second-order information.
result Superior numerical performance and better scalability in high-dimensional distributions.