arXiv research

New algorithms accelerate SVGD convergence using deep unfolding.

New method accelerates energetic variational inference using particle dynamics.

Paper accelerates Bayesian few-shot classification using mirror descent.

Accelerates MMLE using SVGD with Nesterov acceleration.

New method tightens variational representations of divergences for faster learning.

ASVGD accelerates SVGD for efficient sampling.

The variational principle and the corresponding differential equation for geodesic circles in two dimensional (pseudo)-Riemannian space are being discovered. The relationship with the physical notion of uniformly accelerated relativistic particle is emphasized. The known form of spin-curvature interaction emerges due t…

New adaptive methods for constrained convex optimization and variational inequalities.

SympFormer accelerates attention blocks using inertial dynamics on density spaces.

Develops an accelerated algorithm for solving nonmonotone generalized equations.

New method speeds up diffusion models without requiring complex assumptions.

This paper analyzes speculative decoding, a method to speed up large language model inferences.

Derives new optimization methods using variational integrators.

Accelerated gradient methods play a central role in optimization, achieving optimal rates in many settings. While many generalizations and extensions of Nesterov's original acceleration method have been proposed, it is not yet clear what is the natural scope of the acceleration concept. In this paper, we study accelera…

This paper presents a methodology and numerical algorithms for constructing accelerated gradient flows on the space of probability distributions. In particular, we extend the recent variational formulation of accelerated gradient methods in (wibisono, et. al. 2016) from vector valued variables to probability distributi…

Particle-based variational inference methods (ParVIs) have gained attention in the Bayesian inference literature, for their capacity to yield flexible and accurate approximations. We explore ParVIs from the perspective of Wasserstein gradient flows, and make both theoretical and practical contributions. We unify variou…

We derive a variational model to fit a composite Bézier curve to a set of data points on a Riemannian manifold. The resulting curve is obtained in such a way that its mean squared acceleration is minimal in addition to remaining close the data points. We approximate the acceleration by discretizing the squared second o…

ASVGD accelerates SVGD for efficient sampling from Gaussian targets.

NIFTy.re accelerates imaging models and expands Gaussian processes and variational inference.

New study on No-U-Turn Sampler for accelerated mixing in Hamiltonian Monte Carlo.

It is proved that the set of geodesic circles in two dimensions may be given a variational description and the explicit form of it is presented. In the limit case of the Euclidean geometry a certain claim of uniqueness of such description is proved. A formal notion of 'spin' force is discovered as a by-product of the v…

The Hawkes process (HP) has been widely applied to modeling self-exciting events including neuron spikes, earthquakes and tweets. To avoid designing parametric triggering kernel and to be able to quantify the prediction confidence, the non-parametric Bayesian HP has been proposed. However, the inference of such models …

Purpose: To introduce a combined machine learning (ML) and physics-based image reconstruction framework that enables navigator-free, highly accelerated multishot echo planar imaging (msEPI), and demonstrate its application in high-resolution structural and diffusion imaging. Methods: Singleshot EPI is an efficient enco…

A Python package for GPU-accelerated signature kernel computation.

We accelerate CNF by reducing ODE truncation errors with polynomial regularization.

RVI accelerates encoderless VI for faster convergence.

Variational inference offers scalable and flexible tools to tackle intractable Bayesian inference of modern statistical models like Bayesian neural networks and Gaussian processes. For largely over-parameterized models, however, the over-regularization property of the variational objective makes the application of vari…

ASBART accelerates Soft BART for faster Bayesian regression.

A new transformer model accelerates training with optimization techniques.

Improves learning of spectral mixture kernels with approximate Bayesian inference.

A parameter-invariant variational problem with a manifestly covariant Lagrangian function of second order is considered, which covers the case of the free relativistic top at constraint manifold of constant acceleration

MER algorithm speeds up VI solving with Markovian data.

Improved optimization guarantees for deep learning models with Nesterov acceleration.

Flow-VQE uses generative flows to optimize VQE efficiently.

We introduce a new convex formulation for stable principal component pursuit (SPCP) to decompose noisy signals into low-rank and sparse representations. For numerical solutions of our SPCP formulation, we first develop a convex variational framework and then accelerate it with quasi-Newton methods. We show, via synthet…

The paper proposes an efficient method to scale Bayesian inference for mixed multinomial logit models to very large datasets.

Paper accelerates diffusion models without retraining, reducing evaluations.

New algorithm improves convergence of Bayesian inference.

Stein variational gradient descent (SVGD) was recently proposed as a general purpose nonparametric variational inference algorithm [Liu & Wang, NIPS 2016]: it minimizes the Kullback-Leibler divergence between the target distribution and its approximation by implementing a form of functional gradient descent on a reprod…

LDTA expands LDA's topic modeling capacity with tree-structured priors.

Continuous semi-implicit models enable faster training and better performance in generative modeling.

Bayesian Neural Networks (BNNs) have been proposed to address the problem of model uncertainty in training and inference. By introducing weights associated with conditioned probability distributions, BNNs are capable of resolving the overfitting issue commonly seen in conventional neural networks and allow for small-da…

A quantum walk-based method for generating precise probability distributions efficiently.

Reduces necessary conditions for collision avoidance on curved spaces.

Paper proposes a second-order method for faster SVI convergence.

Negative momentum accelerates convergence in minimax games but at a suboptimal rate.

Proposes momentum methods for Lie groups, improving on classical algorithms.

This paper analyzes two Lie group momentum optimization algorithms and their convergence rates.