Improved analysis of extragradient methods for structured VIPs.
problem Efficiently solving large-scale VIPs with weaker conditions.
method Single-call stochastic extragradient methods with expected residual condition.
result Convergence guarantees for quasi-strongly monotone and weak Minty VIPs.
We introduce the implicit processes (IPs), a stochastic process that places implicitly defined multivariate distributions over any finite collections of random variables. IPs are therefore highly flexible implicit priors over functions, with examples including data simulators, Bayesian neural networks and non-linear tr…
New insights into stochastic methods for solving variational inequalities.
problem Understanding convergence behaviors of stochastic algorithms in variational inequalities.
method Re-casting SEG/SGDA as Markov Chains to analyze their probabilistic structures.
result The average iterate is asymptotically normal with a unique invariant distribution for various VIPs.
Improved convergence for VIPs with SEG-RR, a variant of SEG with random reshuffling.
problem Solving variational inequality problems (VIPs) in machine learning.
method Stochastic Extragradient with Random Reshuffling (SEG-RR).
result SEG-RR achieves faster convergence rates than with-replacement variants for certain VIP classes.
New MIF architecture improves posterior approximations in Bayesian models.
problem Challenges in variational inference for complex hierarchical models.
method Combines VIP and autoregressive flow with prior information and hierarchical ordering.
result Empirically, MIF delivers tighter posterior approximations and state-of-the-art performance.
Unified perspective on natural gradient methods for GMMs, improving variational inference.
problem Efficiently learning multi-modal approximations of complex distributions.
method Comparison and optimization of VIPS and iBayes-GMM methods for Gaussian mixture models.
result Hybrid approach significantly outperforms both VIPS and iBayes-GMM.
Unified analysis of efficient local training methods for distributed variational inequalities.
problem Efficient distributed/federated learning for variational inequality problems.
method Unified convergence analysis of communication-efficient local training methods.
result First local gradient descent-accent algorithms with improved communication complexity.
Many applications of Bayesian data analysis involve sensitive information, motivating methods which ensure that privacy is protected. We introduce a general privacy-preserving framework for Variational Bayes (VB), a widely used optimization-based Bayesian inference method. Our framework respects differential privacy, t…