BOED improves SBI by optimizing experimental designs and inference functions.
problem Efficiently use experimental resources for better inference on complex models.
method Link mutual information bounds between SBI and BOED, optimizing both design and inference.
result BOED improves inference in real-world simulators in epidemiology and biology.
Inference models are a key component in scaling variational inference to deep latent variable models, most notably as encoder networks in variational auto-encoders (VAEs). By replacing conventional optimization-based inference with a learned model, inference is amortized over data examples and therefore more computatio…
Active inference minimizes expected free energy for optimal behavior.
problem Understanding and optimizing behavior in complex systems.
method Combines Bayesian decision theory, optimal Bayesian design, and the free energy principle.
result Active inference emerges as a unified framework for information-seeking, utility maximization, and goal-directed behavior.
This work frames active inference through control as inference, offering robust control algorithms.
problem Active inference framework lacks practical sensorimotor control algorithms.
method Frame active inference through control as inference, presenting trajectory optimization as inference.
result AI may be framed as partially-observed CaI when the cost function is defined in observation states.
Quantized Variational Inference improves ELBO optimization with fast convergence.
problem Maximizing Evidence Lower Bound (ELBO) for variational inference.
method Optimal Voronoi Tesselation for variance-free gradients, Richardson extrapolation for asymptotic improvement.
result Quantized Variational Inference leads to fast convergence with comparable computational cost.
Derives time-averaged active inference from control principles.
problem Finite-horizon or discounted-surprise problems in active inference.
method Derives infinite-horizon, average-surprise active inference from optimal control principles.
result Unified objective functional for sensorimotor control.
Unified approach for sequence design combining likelihood-free inference and black-box optimization.
problem Designing biological sequences efficiently and accurately.
method Unified probabilistic framework integrating likelihood-free inference and black-box optimization.
result Previous optimization methods can be adapted and new algorithms proposed within this framework.
Variational inference is an umbrella term for algorithms which cast Bayesian inference as optimization. Classically, variational inference uses the Kullback-Leibler divergence to define the optimization. Though this divergence has been widely used, the resultant posterior approximation can suffer from undesirable stati…
Membership inference determines, given a sample and trained parameters of a machine learning model, whether the sample was part of the training set. In this paper, we derive the optimal strategy for membership inference with a few assumptions on the distribution of the parameters. We show that optimal attacks only depe…
Kernel semi-implicit variational inference improves variational inference without additional optimization.
problem Intractability of hierarchical semi-implicit distributions in variational inference.
method Kernel semi-implicit variational inference (KSIVI) using kernel methods to eliminate lower-level optimization.
result KSIVI reduces variational inference to kernel Stein discrepancy (KSD) optimization, improving expressiveness and tractability.
We empirically evaluate a stochastic annealing strategy for Bayesian posterior optimization with variational inference. Variational inference is a deterministic approach to approximate posterior inference in Bayesian models in which a typically non-convex objective function is locally optimized over the parameters of t…
New method for online inference of constrained optimization problems.
problem Online inference of constrained stochastic optimization problems.
method Random scaling of Sketched Stochastic Sequential Quadratic Programming (SSQP).
result Asymptotically valid confidence intervals and matrix-free computation.
Paper shows how to infer hidden states in neural networks analytically.
problem Intractability of Bayesian inference for neural networks.
method Leverage tractable approximate Gaussian inference (TAGI) for hidden states inference.
result Demonstrates inference of hidden states through constraints for various applications.
Bayesian coresets improve scalable Bayesian inference.
problem Efficiently approximating posterior inference with a subset of data.
method Sparsity constrained optimization and accelerated optimization methods.
result Explicit convergence rate guarantees and superior performance compared to state-of-the-art.
EVODiff optimizes DM inference by reducing conditional entropy, improving image generation.
problem Slow and inaccurate inference in diffusion models.
method Entropy-aware variance optimization for efficient inference.
result Significant improvement in image generation quality and efficiency.
To model modern large-scale datasets, we need efficient algorithms to infer a set of P unknown model parameters from N noisy measurements. What are fundamental limits on the accuracy of parameter inference, given finite signal-to-noise ratios, limited measurements, prior information, and computational tractability …
Inference-aware meta-alignment of LLMs reduces computational cost.
problem Aligning LLMs to diverse human preferences is challenging due to conflicting criteria.
method IAMA trains a base model to be aligned to multiple tasks via different inference-time alignment algorithms, using non-linear GRPO for optimization.
result IAMA enables effective alignment of LLMs to multiple criteria with limited computational budget.
New method speeds up Bayesian inference for complex simulators.
problem Challenges in Bayesian inference for complex stochastic simulators with intractable likelihood functions.
method Optimization Monte Carlo framework reformulated as deterministic optimization problems with gradient-based methods.
result Accurate posterior inference with reduced runtimes compared to existing methods.
New perspective on federated learning as posterior inference, improving optimization.
problem Optimizing global models in distributed learning settings.
method Formulated as posterior inference problem, using MCMC for approximate inference and federated averaging for refinement.
result Federated posterior averaging (FedPA) outperforms existing methods on benchmarks.
Optimism stabilizes Thompson Sampling for adaptive inference in multi-armed bandits.
problem Subtle inferential properties of Thompson Sampling under adaptive data collection.
method Introduced optimism as a key mechanism to restore stability and validity of inference.
result Suitably implemented optimism stabilizes Thompson Sampling and enables asymptotically valid Wald inference.
Enhances optimization and sampling methods using ensemble-based gradient inference.
problem Improving ensemble-based methods for optimization and sampling.
method Ensemble-based gradient inference (EGI) to extract higher-order derivatives from particle ensembles.
result Augmented algorithms outperform gradient-free variants, especially in multimodal and non-Gaussian settings.
AutoBayes simplifies variational inference by composing models and optimizing them.
problem Generalized variational inference complexities and optimization challenges.
method Compositional framework exploiting chain rules for automatic differentiation.
result Optimized models and parameterized statistical games can be locally optimized.
BBVI relies on adaptive stochastic optimization algorithms for posterior approximation, but these require extensive tuning.
problem BBVI posterior approximation
method Adaptive stochastic optimization algorithms
result No single method dominates, but a selection of 5 suffices.
New algorithms improve likelihood of finding global optima in Bayesian inference.
problem Finding global optima in Bayesian inference is difficult due to nonconvexity.
method Developed two algorithms: consistent Laplace approximation (CLA) and consistent stochastic variational inference (CSVI).
result Both CSVI and CLA improve likelihood of obtaining global optima compared to standard methods.
A new method improves SVI for high-dimensional, poorly-conditioned distributions.
problem Challenges in existing SVI methods for high-dimensional, poorly-conditioned distributions.
method Trust-region optimization approach leveraging conditional independences and second-order information.
result Superior numerical performance and better scalability in high-dimensional distributions.
Paper develops online statistical inference methods for stochastic optimization using Kiefer-Wolfowitz algorithms.
problem Online statistical inference of model parameters in stochastic optimization problems.
method Kiefer-Wolfowitz algorithm with random search directions, asymptotic distribution analysis.
result Developed valid confidence intervals for online statistical inference.
Energy savings for DNN inference on resource-constrained devices.
problem Energy efficiency in deep learning inference for constrained devices.
method Efficiently searches through equivalent DNN graphs to find the one with the least execution cost.
result Achieves 24% energy savings with minimal performance impact.
Efficient method for tensor linear form inference with noisy incomplete data.
problem Statistical inference of tensor linear forms with incomplete and noisy observations.
method Initial estimate + debiasing + one-step power iteration.
result Optimal uncertainty quantification and statistical-to-computational gaps examined.
Develops theory for conditional optimal transport in infinite-dimensional spaces.
problem Bayesian inference with functional parameters in infinite-dimensional spaces.
method Theory of constrained optimal transport for block-triangular maps.
result Regularity estimates on conditioning maps from prior to posterior.
Deep Gaussian Processes improve likelihood-free inference for complex distributions.
problem Limited flexibility of Bayesian Optimization with GPs for multimodal distributions.
method Proposes Deep Gaussian Processes (DGPs) as a surrogate model for likelihood-free inference.
result DGPs outperform GPs on multimodal distributions while maintaining comparable performance on unimodal cases.
Particle-based variational inference offers a flexible way of approximating complex posterior distributions with a set of particles. In this paper we introduce a new particle-based variational inference method based on the theory of semi-discrete optimal transport. Instead of minimizing the KL divergence between the po…
Bayesian models quantify uncertainty and facilitate optimal decision-making in downstream applications. For most models, however, practitioners are forced to use approximate inference techniques that lead to sub-optimal decisions due to incorrect posterior predictive distributions. We present a novel approach that corr…
New guarantees for black-box variational inference methods.
problem Insufficient theoretical guarantees for black-box variational inference.
method Novel convergence guarantees for stochastic optimization of variational inference.
result Provable convergence of proximal and projected stochastic gradient descent for variational inference.
Improves learning of spectral mixture kernels with approximate Bayesian inference.
problem Difficult optimization of large number of SM kernel parameters.
method Approximate Bayesian inference using variational distribution of spectral points and random Fourier features.
result Accelerates convergence and leads to better optimal parameters.
Accelerates pulsar light curve inference with learned representations and optimization.
problem Computational expense of Markov chain Monte Carlo methods for posterior inference.
method Combining U-Net latent representations with local simulator-guided optimization.
result 120x reduction in inference time (24 hours to 12 minutes) with accuracy preserved.
A new method combines VI and IS to improve Bayesian inference accuracy.
problem Bayesian inference often underestimates posterior tails, leading to miscalibration and degeneracy.
method Proposes a novel combination of optimization and sampling techniques using the forward KL divergence.
result The method guarantees asymptotic consistency and fast convergence to optimal IS and variational approximations.
PVI seeks a posterior that makes predictions closer to true data, not approximating the Bayesian posterior.
problem Finding meaningful posterior distributions under model misspecification.
method Predictive variational inference (PVI) seeks an optimal posterior density for close predictive matching to true data.
result PVI learns a posterior that is not the same as the Bayesian posterior, but is closer to the true data generating process.
Algorithm optimizes and infers performance online, improving reliability.
problem Balancing optimization and statistical inference in complex systems.
method Online algorithm that provides consistent performance variance and confidence intervals.
result Online consistent estimator for performance variance and asymptotic confidence intervals.
Variational inference methods for latent variable statistical models have gained popularity because they are relatively fast, can handle large data sets, and have deterministic convergence guarantees. However, in practice it is unclear whether the fixed point identified by the variational inference algorithm is a local…
A new method uses Coordinate Descent to optimize ResNet networks for private inference.
problem Reducing ReLU count in ResNet networks for private inference.
method Directly optimizing in the discrete domain using Coordinate Descent.
result Our method yields a sparse solution and is state-of-the-art.
EHR-MPC optimizes sepsis treatment using digital twins and inference-time control.
problem Optimal sepsis treatment policies are contested and difficult to adapt during inference.
method EHR-MPC decouples learning patient dynamics from treatment optimization, enabling inference-time control over learned digital twins.
result EHR-MPC achieves comparable off-policy performance and improved simulation performance compared to RL baselines.
Post-ADC inference corrects bias in statistical inference after active data collection.
problem Bias in inference after active data collection.
method Post-ADC inference framework that corrects bias from both ADC process and data-driven target construction.
result Valid inference for data collected by SMBO methods like GP-UCB and TPE.
Paper proposes a second-order method for faster SVI convergence.
problem Poor convergence rate of first-order SVI algorithms.
method Derives Hessian matrix and implements two numerical schemes for efficient second-order SVI.
result Proposed approach achieves faster convergence compared to first-order SVI.
Optimizes variational inference for dynamic network models.
problem Estimating pairwise inner products and intercepts in dynamic latent space models.
method Structured mean-field variational inference with block coordinate ascent algorithm.
result Variational risk attains minimax optimal rate with logarithmic factor under certain conditions.
Method reformulates constrained optimization as latent space inference.
problem Optimizing black-box functions with hard constraints.
method Posterior inference in latent space using flow-based models and diffusion models.
result Method achieves superior performance across various tasks.
Unified framework improves robust causal inference, overcoming Gaussian barriers and optimization issues.
problem Improving robust causal inference in non-Gaussian settings.
method Combines gamma-Divergence, GNC, and Gatekeeper mechanism.
result Enhanced robustness and global optimization in causal effect estimation.
A new method for efficient inference and model selection in SBMs using OT.
problem Efficient inference and model selection in stochastic block models.
method Interpreting MLVI as srGW with entropic regularization, then unregularizing for sparse solutions, and adding a sparsity-promoting regularizer.
result The method consistently recovers SBM parameters and selects the number of clusters in finite samples.
VERA uses variational inference to jailbreak LLMs without manual optimization.
problem Lack of principled objective for gradient-based optimization in jailbreaking LLMs.
method VERA casts black-box jailbreak prompting as a variational inference problem, training a small attacker LLM to approximate the target LLM's posterior over adversarial prompts.
result VERA achieves strong performance across various target LLMs, demonstrating the value of probabilistic inference for adversarial prompt generation.