New perspective on federated learning as posterior inference, improving optimization.
problem Optimizing global models in distributed learning settings.
method Formulated as posterior inference problem, using MCMC for approximate inference and federated averaging for refinement.
result Federated posterior averaging (FedPA) outperforms existing methods on benchmarks.
Variational inference methods for latent variable statistical models have gained popularity because they are relatively fast, can handle large data sets, and have deterministic convergence guarantees. However, in practice it is unclear whether the fixed point identified by the variational inference algorithm is a local…
Global optimization in Bayesian inference yields little additional benefit.
problem Improving psychometric parameter estimation using global optimization strategies.
method Experimental simulations comparing myopic and global strategies in multiple models.
result Global optimization strategies provide negligible additional utility improvement beyond the immediate next steps.
We consider the problem of analyzing the heterogeneity of clustering distributions for multiple groups of observed data, each of which is indexed by a covariate value, and inferring global clusters arising from observations aggregated over the covariate domain. We propose a novel Bayesian nonparametric method reposing …
New algorithms improve likelihood of finding global optima in Bayesian inference.
problem Finding global optima in Bayesian inference is difficult due to nonconvexity.
method Developed two algorithms: consistent Laplace approximation (CLA) and consistent stochastic variational inference (CSVI).
result Both CSVI and CLA improve likelihood of obtaining global optima compared to standard methods.
Paper proposes Meta Label Learning to infer global labels for robust few-shot models.
problem Few-shot learning with limited training data.
method Meta Label Learning (MeLa) framework that infers global labels.
result MeLa framework is competitive with existing methods and robust for few-shot learning.
Combines multi-layer graphs to infer global network structure.
problem Leveraging domain knowledge in structure inference for multi-layer graphs.
method Mask combination of multi-layer graphs using optimization.
result Enhanced structure inference through multi-layer graph integration.
Graph cuts find global optima for Potts models in slight perturbations.
problem Finding optimal solutions in Potts models with graph cuts.
method α-expansion algorithm for MAP inference, with certification for perturbations.
result All local minima are global minima in slight perturbations, and solutions are close to original.
Boosting Variational Inference improves posterior approximations with adaptive step-sizes.
problem Limited resources hinder the widespread adoption of Boosting Variational Inference.
method Characterized global curvature impact, introduced local curvature, and developed an approximate backtracking algorithm.
result New theoretical convergence rates and experimental validation demonstrate improved performance.
Mixup inference improves adversarial robustness by mixing inputs with clean samples.
problem Adversarial examples can fool deep networks due to local non-linearity.
method Develops mixup inference, which mixes inputs with clean samples to shrink adversarial perturbations.
result Mixup inference enhances adversarial robustness for mixup-trained models.
This article studies local and global inference for smoothing spline estimation in a unified asymptotic framework. We first introduce a new technical tool called functional Bahadur representation, which significantly generalizes the traditional Bahadur representation in parametric models, that is, Bahadur [Ann. Inst. S…
OCEAN infers online task identities from context variables.
problem Online task inference for compositional tasks with context adaptation.
method Variational inference framework OCEAN models global and local context variables in a joint latent space.
result OCEAN provides more effective task inference with sequential context adaptation.
HNPE uses auxiliary data to estimate parameters in uncertain models.
problem Uncertain models with identical observations.
method Exploits global parameters from auxiliary data to estimate parameters.
result Validated on a motivating example and applied to neuroscience.
GAAVI offers anytime-valid tests for CMF global null and contrasts.
problem Inference on the conditional mean function for high confidence decisions.
method Asymptotic anytime-valid tests for CMF global null and contrasts.
result Achieves asymptotic type-I error guarantees, power one, and optimal sample complexity.
New method guarantees global convergence in variational inference.
problem Limited convergence to local optima in variational inference.
method Minimizes inclusive KL divergence using neural networks and neural tangent kernel.
result Gradient descent dynamics converge to a unique solution in function space.
MeLa learns task relations by inferring global labels for robust FSL.
problem Few-shot learning with limited global labels.
method Meta Label Learning (MeLa) and augmented pre-training.
result MeLa outperforms existing methods across diverse benchmarks.
LOAD discovers optimal adjustments locally for scalable causal inference.
problem Scalable causal inference for unknown causal graphs.
method Local Optimal Adjustments Discovery (LOAD) method.
result LOAD combines local and global approaches for efficient and accurate causal effect estimation.
Enhances sequence labeling with embedded-state latent CRFs.
problem Complex non-local constraints between sequence labels.
method Integrates multiple hidden states with low-rank log-potential scoring matrices.
result Model outperforms baseline CRF+RNN models with global constraints.
Improves diffusion model performance and efficiency through classical search.
problem Tackles inference-time control in diffusion models.
method Proposes a framework combining local and global search for efficient navigation.
result Significant gains in performance and efficiency across various domains.
Inter-domain Deep Gaussian Processes improve inference for non-stationary data.
problem Inference limitations in Gaussian processes for non-stationary data.
method Combines inter-domain and deep Gaussian processes for scalable approximate inference.
result Outperforms inter-domain shallow GPs and conventional DGPs on non-stationary data.
FedBE aggregates local models into a robust global model via Bayesian inference.
problem Challenges in aggregating non-i.i.d. local models into a global model in federated learning.
method FedBE uses Bayesian inference to sample and combine higher-quality global models from local models.
result FedBE leads to more robust aggregation of local models into a global model, especially when data is non-i.i.d.
Federated learning for Bayesian clustering of large datasets.
problem Bayesian model-based clustering of large-scale binary and categorical data.
method Federated variational inference with local merge and delete moves in parallel batches, followed by global merge moves.
result Empirical validation shows superior performance compared to existing algorithms.
This paper addresses problematic global optima in VAEs, proposing a new inference method.
problem VAEs often yield solutions that violate modeling desiderata, leading to unrealistic data generation.
method The paper presents LiBI, a novel inference method to mitigate these issues.
result LiBI can learn better generative and inference models on synthetic datasets.
Probabilistic programming languages (PPLs) are a powerful modeling tool, able to represent any computable probability distribution. Unfortunately, probabilistic program inference is often intractable, and existing PPLs mostly rely on expensive, approximate sampling-based methods. To alleviate this problem, one could tr…
Enhances belief propagation to find global optima without increasing computational burden.
problem Improving probabilistic inference accuracy on graphical models.
method Homotopy continuation method that gradually incorporates pairwise potentials.
result SBP finds the global optimum of the Bethe approximation for attractive models.
Develops a flexible deep autoencoding topic model with scalable hybrid Bayesian inference.
problem Flexible and interpretable document analysis models.
method DATM with hybrid Bayesian inference, including topic-layer-adaptive stochastic gradient Riemannian MCMC and Weibull variational encoder.
result Demonstrates scalability and efficacy on big corpora in unsupervised and supervised learning tasks.
Develops a method to infer cell trajectories from RNA sequencing data.
problem Inferring cell trajectories from single cell RNA-sequencing data.
method Entropy-regularized optimal transport for global optimization.
result Proves and implements a method to recover ground truth trajectories from limited samples.
New pruning method captures global correlations for efficient neural network inference.
problem Efficiently pruning neural networks for faster inference and reduced memory usage.
method Second-order structured pruning (SOSP-H) with innovative saliency-based approaches.
result SOSP-H scales to large-scale vision tasks and improves accuracy without compromising efficiency.
DSVNP uses global and local latent variables for improved neural process predictions.
problem Limited expressiveness of vanilla neural processes in capturing target-specific local variation.
method Introduces DSVNP combining global and local latent variables for prediction.
result Competitive prediction performance in multi-output regression and uncertainty estimation.
Bayesian inference on structured models typically relies on the ability to infer posterior distributions of underlying hidden variables. However, inference in implicit models or complex posterior distributions is hard. A popular tool for learning implicit models are generative adversarial networks (GANs) which learn pa…
Inference in general Markov random fields (MRFs) is NP-hard, though identifying the maximum a posteriori (MAP) configuration of pairwise MRFs with submodular cost functions is efficiently solvable using graph cuts. Marginal inference, however, even for this restricted class, is in #P. We prove new formulations of deriv…
Meta-learn Bayesian inference for task-specific BNNs using amortised inference.
problem Efficiently learning Bayesian inference for small-scale probabilistic meta-learning.
method Replace global inducing points with actual data to create a set of approximate likelihoods, train a meta-model to learn these parameters across related datasets.
result Meta-learned inference can be applied to task-specific BNNs, improving efficiency and scalability.
The mean field variational Bayes method is becoming increasingly popular in statistics and machine learning. Its iterative Coordinate Ascent Variational Inference algorithm has been widely applied to large scale Bayesian inference. See Blei et al. (2017) for a recent comprehensive review. Despite the popularity of the …
Unified framework improves robust causal inference, overcoming Gaussian barriers and optimization issues.
problem Improving robust causal inference in non-Gaussian settings.
method Combines gamma-Divergence, GNC, and Gatekeeper mechanism.
result Enhanced robustness and global optimization in causal effect estimation.
Global inducing points improve Bayesian neural network performance.
problem Improving Bayesian neural network performance.
method Adapting correlated approximate posterior to all layers in a Bayesian neural network and deep Gaussian processes using learned global inducing points.
result State-of-the-art performance on CIFAR-10 (86.7%) without data augmentation or tempering.
Networks are ubiquitous in biology and computational approaches have been largely investigated for their inference. In particular, supervised machine learning methods can be used to complete a partially known network by integrating various measurements. Two main supervised frameworks have been proposed: the local appro…
A new method for privacy-preserving Bayesian learning in federated learning.
problem Privacy-preserving learning of models from distributed sensitive data.
method Differentially private partitioned variational inference (DPVI) for federated learning.
result First general framework for federated Bayesian learning with differential privacy.
Contemporary global optimization algorithms are based on local measures of utility, rather than a probability measure over location and value of the optimum. They thus attempt to collect low function values, not to learn about the optimum. The reason for the absence of probabilistic global optimizers is that the corres…
New method bypasses global fit for LISA's Galactic binaries, extracting population parameters directly.
problem Disentangling LISA's Galactic binary sources from backgrounds in a computationally intensive process.
method Simulation-based approach using normalizing flow to infer population parameters.
result Direct inference of population parameters from LISA's frequency strain series.
Local mass perspective on Bayesian inference
problem Measuring distributional discrepancy in Bayesian inference
method Introducing Mass Index and Regularised Extended KL
result Proving inequalities for comparing local small-ball masses
AM-PPI uses multiple predictors to reduce label cost in healthcare AI.
problem Reduces label cost in post-deployment monitoring of healthcare AI.
method Combines model predictions with a small labeled sample, routing each instance to a cost-appropriate subset of predictors.
result Produces narrower confidence intervals than single-predictor methods.
Sobol method applied to probabilistic networks for sensitivity analysis.
problem Measuring influence of probabilistic network nodes on a quantity of interest.
method Transforms global sensitivity analysis into marginalization inference exploiting network structure.
result Efficient computation of sensitivity indices for complex networks.
Amortized variational inference (AVI) replaces instance-specific local inference with a global inference network. While AVI has enabled efficient training of deep generative models such as variational autoencoders (VAE), recent empirical work suggests that inference networks can produce suboptimal variational parameter…
GAMs combine autoregressive and log-linear components for data-efficient sequence learning.
problem Poor performance of standard autoregressive models under small-data conditions.
method Introduce Global Autoregressive Models (GAMs) combining autoregressive and log-linear components, trained in two steps.
result GAMs show a strong perplexity reduction over standard models in language modelling.
A deep model learns to infer fluorescence labels from unlabeled microscopy images.
problem Challenges in obtaining high quality images of cellular structures due to complex environments and label staining limitations.
method Developed a novel deep model using global pixel transformer layers and dense blocks, incorporating multi-scale input strategy.
result Significantly outperforms state-of-the-art methods in fluorescence image prediction tasks.
Structured prediction requires searching over a combinatorial number of structures. To tackle it, we introduce SparseMAP: a new method for sparse structured inference, and its natural loss function. SparseMAP automatically selects only a few global structures: it is situated between MAP inference, which picks a single …
Contrastive divergence (CD) is a promising method of inference in high dimensional distributions with intractable normalizing constants, however, the theoretical foundations justifying its use are somewhat shaky. This document proposes a framework for understanding CD inference, how/when it works, and provides multiple…
Optimal inference in distributed quantile regression without stringent scaling conditions.
problem Challenges in achieving optimal inference in distributed quantile regression due to the non-smooth nature of the QR loss function.
method Double-smoothing approach applied to local and global objective functions, with a trade-off between communication cost and statistical error.
result Established a finite-sample theoretical framework for distributed QR estimators, showing a trade-off between communication cost and statistical error.