A new particle algorithm improves mean-field variational inference.
problem Efficiently approximating nonparametric posterior distributions in machine learning.
method Introduces PArticle VI (PAVI), a novel particle-based algorithm for nonparametric mean-field approximation.
result Obtains non-asymptotic error bounds for PArticle VI, providing the first end-to-end guarantee for particle-based MFVI.
Selective inference controls Type I error in k-means clustering tests.
problem Inflated Type I error in classical hypothesis tests for k-means clusters.
method Selective inference approach to control Type I error.
result Proposes a computable finite-sample p-value for selective inference.
The mean field algorithm is a widely used approximate inference algorithm for graphical models whose exact inference is intractable. In each iteration of mean field, the approximate marginals for each variable are updated by getting information from the neighbors. This process can be equivalently converted into a feedf…
Bayesian deconditional embeddings solve complex function recovery.
problem Recovering original functions from conditional mean observations.
method Formalizes deconditional kernel mean embeddings as Bayesian inference, connects to task-transformed Gaussian processes.
result Establishes deconditional kernel means as posterior predictive mean, providing Bayesian interpretations and uncertainty.
KELFI improves inference accuracy in likelihood-free settings with limited simulations.
problem Intractable likelihood evaluations in likelihood-free inference.
method Kernel embedding likelihood-free inference (KELFI) learns model hyperparameters to balance accuracy and efficiency.
result Improved accuracy and efficiency on challenging inference problems in ecology.
A spiking neural network model for probabilistic inference of binary Markov random fields.
problem Implementing probabilistic inference in spiking neural networks.
method Designing a spiking recurrent neural network and proving its equivalence to mean-field inference of binary Markov random fields.
result The spiking neural network model can implement inference of arbitrary binary Markov random fields.
New algorithm improves mean field inference in probabilistic models.
problem Improving mean field inference in probabilistic models.
method DR-DoubleGreedy algorithm for continuous DR-submodular maximization with box-constraints.
result Achieves optimal 1/2 approximation ratio for continuous DR-submodular maximization.
Paper derives two inference algorithms for Gaussian RBMs.
problem Inference in Gaussian RBMs.
method Two mean-field approximation algorithms.
result Second method outperforms first method.
TS-K-means improves financial data clustering with dynamic time warping.
problem Inadequate handling of temporal dependencies in financial time series data.
method Integrates Dynamic Time Warping into Time Series K-means for financial data.
result TS-K-means outperforms traditional K-means in financial data analysis.
Study shows mean field method's effectiveness in community detection.
problem Theoretical and practical guarantees for community detection using mean field variational inference.
method Iterative Coordinate Ascent Variational Inference algorithm for the Stochastic Block Model.
result The algorithm converges linearly to the minimax rate within log n iterations.
Paper proposes an effective mean-field inference method for NNBMs.
problem Inference in NNBMs is challenging due to their complex structure.
method Uses mean-field method and diagonal consistency method.
result Effective inference method for NNBMs is proposed.
Mean field Gaussian inference limits mutual information to regularize neural networks.
problem Understanding and quantifying the regularization effect of mean field Gaussian inference.
method Empirically observed and theoretically quantified mutual information limitation through noise.
result Bounding mutual information between parameters and data effectively regularizes neural networks.
New methods for Bayesian inference using mean shift particle systems.
problem Approximating expectations with unnormalized densities in Bayesian inference.
method Mean shift interacting particle systems that minimize maximum mean discrepancy (MMD).
result Mean shift interacting particle systems converge quickly and capture complex distributions.
New method improves approximate inference for Bayesian models.
problem Approximate inference for high-dimensional Bayesian models.
method Entropic regularization of mean-field variational inference.
result Improved recovery of true posterior dependency.
Bayesian neural networks ignore data in infinite units limit.
problem Pathological behavior of posterior in over-parameterized networks.
method Mean-field variational inference in infinite hidden units limit.
result Posterior mean converges to zero, ignoring data.
A method for rank verification in multivariate Gaussian data, improving on existing approaches.
problem Determining the top K means in multivariate Gaussian data with any covariance structure. method Selective inference tools to generalize the two-sided difference-of-means test for any K and covariance structure. result The method provides a generalization for rank verification in multivariate Gaussian data with any covariance structure.
Paper improves statistical efficiency of median-of-means estimator for Byzantine robust distributed inference.
problem Byzantine robustness in distributed learning systems.
method Variance reduced median-of-means (VRMOM) estimator for Byzantine robust distributed inference.
result Achieves a fast convergence rate with only a constant number of rounds of communications.
Review of mean-field methods for neural network inference.
problem Understanding neural network learning from a theoretical perspective.
method Mean-field methods, high-temperature expansions, replica method, message passing algorithms.
result Equivalences and complementarities of mean-field methods.
Improved inference of mean outcome with less data.
problem Improving inference of mean outcome with limited labeled data.
method k-fold, cross-fitted, double robust estimator for root-n inference.
result Root-n inference of mean outcome possible with less labeled data.
The paper studies stability of mean-field variational inference for log-concave distributions.
problem Stability of mean-field variational inference for log-concave distributions.
method Novel approach via linearized optimal transport, lifting non-convex problem to convex optimization over transport maps.
result Dimension-free Lipschitz continuity of the MFVI optimizer with respect to the target distribution, measured in 2-Wasserstein distance.
Beta process is the standard nonparametric Bayesian prior for latent factor model. In this paper, we derive a structured mean-field variational inference algorithm for a beta process non-negative matrix factorization (NMF) model with Poisson likelihood. Unlike the linear Gaussian model, which is well-studied in the non…
Paper generalizes EC inference for loopy models.
problem Probabilistic inference in loopy models.
method Generalized Expectation Consistency (GEC) method.
result GEC can be applied to MAP and MMSE estimation.
Unified approach for estimating quantiles of potential outcomes using inverse estimating equations.
problem Estimating quantiles of potential outcomes for causal inference.
method Inverse estimating equations and moment function.
result Unified approach to estimate mean and quantiles of potential outcomes.
New method for estimating counterfactual means in adaptive experiments.
problem Inference for counterfactual means in sequentially designed experiments with adaptive treatment policies.
method Latent factor model and nearest neighbors method for estimation.
result Asymptotically valid confidence intervals for counterfactual means established.
The paper uses deep neural networks to estimate and infer ATE without needing to know the dimension of the data.
problem Estimating and inferring the average treatment effect (ATE) in complex data settings.
method The paper uses deep neural networks to estimate the mean regression function and then calculates the ATE. It establishes consistency and asymptotic normality of the estimators.
result The deep neural network estimates of ATE are consistent and asymptotically normal, providing dimension-free rates.
Study shows TAP free energy minimization provides better posterior inference in high-dimensional linear models.
problem Deviation from true posterior mean and underestimation of posterior uncertainty in variational inference.
method Minimization of TAP free energy in a high-dimensional asymptotic framework, showing geometric and statistical properties.
result Local minimizer of TAP free energy provides consistent estimate of posterior marginals and correctly calibrated posterior inference.
New method uses Fokker-Planck equation for sampling and inference.
problem Intractability of evaluating probability density in practical applications.
method Reformulates Fokker-Planck equation as a particle flow method, using velocity field.
result Turns intractable density evaluation into an advantage for variational inference, kernel mean embeddings, and sequential Monte Carlo.
Copula variational inference improves variational approximations by modeling latent dependencies.
problem Improving variational approximations of latent variable distributions.
method Uses copulas to augment variational families, preserving latent dependencies.
result Enhanced variational approximations leading to better posterior characterizations.
ALO-CV approximates leave-one-out error in proportional regime.
problem Estimating generalization error in high-dimensional settings.
method Developed new analysis for ALO-CV, showed consistency under strong convexity.
result ALO-CV approximates leave-one-out error up to negligible error.
The mean field methods, which entail approximating intractable probability distributions variationally with distributions from a tractable family, enjoy high efficiency, guaranteed convergence, and provide lower bounds on the true likelihood. But due to requirement for model-specific derivation of the optimization equa…
ConvMMD improves inference in noisy data.
problem Inference degradation due to measurement error in noisy data.
method Convolutional Maximum Mean Discrepancy (convMMD) for inference with noisy, heteroscedastic observations.
result Established consistency and asymptotic normality of the convMMD-based estimator.
Proposes selective inference for testing differences in means between clusters.
problem Inflated type I error rate when testing differences in means between clusters.
method Selective inference approach to control selective type I error rate.
result Controls selective type I error rate by accounting for data-driven cluster definition.
New method for estimating mean in SS inference with selection bias and decaying overlap.
problem Estimating mean in SS inference with selection bias and decaying overlap.
method Double Robust Semi-Supervised (DRSS) mean estimator.
result Consistent estimation of mean with correct specification of outcome or propensity score model.
Improved MMD estimator for likelihood-free inference.
problem Computational challenges in estimating MMD for likelihood-free inference.
method Optimally-weighted MMD estimator with improved sample complexity.
result Significantly improved sample complexity for accurate MMD estimation.
Develops a new framework for analyzing MFVI algorithms.
problem Analyzes mean field variational inference (MFVI) formulations.
method Inspired by variational Bayesian formulations, represents MFVI problem in three ways: gradient flow, Fokker-Planck-like equations, and diffusion process.
result Establishes rigorous guarantees for convergence of time-discretized coordinate ascent variational inference algorithms.
New algorithm speeds up large-scale statistical inference.
problem Efficiently solving large-scale mean-field variational inference problems.
method Developed a novel primal-dual algorithm (PD-VI) and a block-preconditioned extension (P2D-VI) for mean-field variational inference. result PD-VI and P2D-VI achieve faster convergence and better solution quality compared to existing methods. CoRMF uses RNNs to solve Ising models efficiently by ordering critical edges.
problem Solving Ising models efficiently and accurately.
method Criticality-ordered spin sequence and RNNs for mean-field factorization.
result Proves tighter error bounds than naive mean-field.
MGVI improves variational inference by accounting for correlations without mean-field simplifications.
problem Approximating Bayesian inference problems with variational methods, especially for high-dimensional models.
method Metric Gaussian Variational Inference (MGVI) iteratively approximates the posterior with Gaussian distributions, optimizing the KL-divergence and using natural gradient descent.
result MGVI achieves higher accuracy and significant speedup compared to traditional methods, scaling linearly in computational time and memory.
New method for faster, scalable inference in coupled Gaussian Processes.
problem Coupled Gaussian Processes require scalable inference methods for posterior uncertainty.
method Structured variational inference for multi-Gaussian Processes.
result Fast and scalable inference capturing posterior dependencies.
Variational inference simplifies Bayesian model approximations.
problem Approximating complex Bayesian posterior distributions.
method Solving optimization problems to approximate posterior distributions with simpler variational distributions.
result Variational inference has been successfully applied in various models and large-scale applications.
New algorithm improves CRF inference and learning.
problem Efficient inference and learning for dense CRFs.
method Regularized Frank-Wolfe algorithm for nonconvex CRF optimization.
result Regularized Frank-Wolfe outperforms mean field and CNN baselines.
The paper analyzes mean-field variational Bayes for complex models and proposes new uncertainty quantification methods.
problem Approximating posterior distributions in complex Bayesian models with latent variables.
method Non-asymptotic analysis on mean-field variational inference, showing that a normal distribution with the MLE center approximates the posterior well.
result The mean-field approximation matches the MLE up to higher-order terms and is essentially efficient for regular parametric models.
An autonomous variational inference algorithm for arbitrary graphical models requires the ability to optimize variational approximations over the space of model parameters as well as over the choice of tractable families used for the variational approximation. In this paper, we present a novel combination of graph part…
New method infers population dynamics from snapshots using path space optimization.
problem Recover dynamics of a population from its temporal marginals.
method Grid-free algorithm using Schrödinger bridges coupled via noisy gradient descent in mean-field limit.
result Global convergence to min-entropy estimator with end-to-end theoretical guarantees.
This paper provides a dictionary of closed-form kernel mean embeddings.
problem Challenges in deriving closed-form kernel mean embeddings.
method Comprehensive dictionary and practical tools for deriving new embeddings.
result Provides a Python library with minimal implementations of embeddings.
Proposes a method to evaluate generalizability in causal inference models.
problem Lack of formal procedures to statistically evaluate generalizability in causal inference.
method Frugal parameterization to simulate from causal benchmarks, using mean and distributional regression methods.
result Ensures more realistic evaluations of causal inference models, avoiding over-reliance on conventional metrics.
Ensemble Kalman Filter improves GPSSM inference for online learning.
problem Non-mean-field variational inference issues in GPSSM.
method Combining EnKF with NMF variational inference.
result Improved online learning performance and data-fitting accuracy.
Rotates MFVI for better Gaussian approximations.
problem Improving variational approximations for complex distributions.
method Rotated coordinate system, PCA-based rotation, iterative Gaussianization.
result Significantly more accurate approximations with lower computational cost.