Variational reduction simplifies Lagrangian systems with scaling symmetries.
problem Simplifying Lagrangian systems with scaling symmetries.
method Defining a variational reduction procedure for homogenous Lagrangian systems.
result Reconstructing trajectories from critical points of reduced variational principle.
This work proposes using zero-variance control variates to reduce variance in pathwise gradient estimators for variational inference.
problem Pathwise gradient estimators in variational inference have high variance, leading to inefficient optimization.
method Apply zero-variance control variates to pathwise gradient estimators.
result Zero-variance control variates can significantly reduce the variance of pathwise gradient estimators without requiring complex assumptions.
Optimizes MCMC chains with neural control variates.
problem Reducing variance in Markov Chain Monte Carlo (MCMC) simulations.
method Uses neural networks as control variates to minimize asymptotic variance.
result Derives optimal convergence rate under various ergodicity assumptions.
Study nonholonomic systems with collisions using variational principles.
problem Variational problems on nonholonomic systems with collisions.
method Extended variational principle, introduced connection on principal bundles, applied Lagrange–Poincaré–Pontryagin reduction.
result Implicit Lagrange–d'Alembert–Pontryagin equations for nonholonomic systems with collisions.
Neural SDEs reduce variance in stochastic simulations.
problem Efficiency of Monte Carlo simulations in finance.
method Use neural SDEs with control variates parameterized by neural networks.
result Prove optimality conditions for variance reduction in SDEs with infinite activity.
New algorithms reduce variance in solving complex mathematical problems.
problem Solving convex-concave saddle point problems, variational inequalities, and inclusions.
method Stochastic variance reduction for extragradient, forward-backward-forward, and forward-reflected-backward methods.
result All proposed methods converge with complexities matching or improving deterministic counterparts.
Improves gradient estimation for discrete distributions with variance reduction techniques.
problem Excessive variance in gradient estimation for discrete distributions.
method Stein operators for discrete distributions and control variates.
result Substantially lower variance in gradient estimation.
U-statistics improve gradient estimation in importance-weighted variational inference.
problem High variance in gradient estimation for importance-weighted variational inference.
method Use U-statistics to average base gradient estimators on overlapping batches of size m, achieving lower variance.
result U-statistic variance reduction leads to modest to significant improvements in inference performance.
The process of un-reduction, a sort of reversal of reduction by the Lie group symmetries of a variational problem, is explored in the setting of field theories. This process is applied to the problem of curve matching in the plane, when the curves depend on more than one independent variable. This situation occurs in a…
Meta-CVs leverage task similarity to reduce variance with limited data.
problem Reducing variance in Monte Carlo estimators with few samples.
method Meta-learning control variates for related tasks.
result Meta-CVs lead to significant variance reduction in settings with limited data.
This paper develops scalable control variates for Monte Carlo methods using stochastic optimization.
problem Reducing variance in Monte Carlo estimators for large-scale problems.
method Control variates based on Stein operators, optimized through stochastic optimization.
result Novel theoretical results and empirical validations show effective variance reduction.
Improved GPLVM model for single-cell RNA-seq data.
problem Lack of effective scalable models for clustering cell types in large-scale single-cell RNA-seq data.
method Introduces amortized stochastic variational Bayesian GPLVM (BGPLVM) tailored for single-cell RNA-seq.
result Matches the performance of scVI on synthetic and real-world datasets and reveals more interpretable latent structures.
BasisVAE combines VAE and clustering for tabular data analysis.
problem Lack of insights in tabular high-dimensional data analysis.
method Combines VAE with probabilistic clustering prior for joint dimensionality reduction and clustering.
result Learned one-hot basis function representation for translation-invariant features.
Survey of factor analysis, PCA, variational inference, and VAE.
problem Dimensionality reduction and generative modeling of data.
method Variational inference, factor analysis, probabilistic PCA, and VAE.
result Derivation and explanation of ELBO, EM, and closed-form solutions.
Over the years data has become increasingly higher dimensional, which has prompted an increased need for dimension reduction techniques. This is perhaps especially true for clustering (unsupervised classification) as well as semi-supervised and supervised classification. Although dimension reduction in the area of clus…
We consider the problem of sufficient dimensionality reduction (SDR), where the high-dimensional observation is transformed to a low-dimensional sub-space in which the information of the observations regarding the label variable is preserved. We propose DVSDR, a deep variational approach for sufficient dimensionality r…
A new method reduces variance in training discrete latent variable models.
problem High variance in stochastic gradient estimators for discrete latent variable models.
method Double control variates for score function estimators using Taylor expansions.
result Our method can have lower variance compared to other estimators.
Paper optimizes classification of distributions using Wasserstein metric.
problem Classifying instances represented by distributions on a vector space.
method Maximizing Fisher's ratio in the Wasserstein metric space through iterative algorithm.
result The method enhances classification performance and is robust to variations in distribution summaries.
A reductive structure is associated here with Lagrangian canonically defined conserved quantities on gauge-natural bundles. Parametrized transformations defined by the gauge-natural lift of infinitesimal principal automorphisms induce a variational sequence such that the generalized Jacobi morphism is naturally self-ad…
Balanced Neural ODEs combine VAEs and Neural ODEs for efficient time series modeling.
problem Efficiently modeling systems with time-varying inputs and varying complexity.
method Combines VAEs for dimensionality reduction and Neural ODEs for dynamics, using variational parameters to adaptively learn.
result Balanced Neural ODEs (B-NODE) efficiently approximate Koopman operator without predefined dimensionality.
Reduces necessary conditions for collision avoidance on curved spaces.
problem Finding non-intersecting trajectories for multiple agents on curved spaces.
method Reduction by Lie group symmetries of variational collision avoidance problems.
result Derives necessary conditions for reduced extremals.
New method reduces variance in complex probabilistic model optimization.
problem High variance in stochastic optimisation of complex models.
method Use recognition network to approximate optimal control variate for each mini-batch.
result Sub-optimal variance reduction is improved with new approach.
A new method reduces complexity and uncertainty in neural networks.
problem Uncertainty quantification in complex neural networks.
method Condensed Stein Variational Gradient Descent (cSVGD) method.
result Condensed SVGD provides uncertainty quantification on parameters.
Paper improves variance control in importance weighted variational bounds.
problem Improving the variance of gradient estimators for IWAE.
method Develops a novel control variate that grows SNR as √K for large K.
result Empirically, the method yields superior variance reduction for generative models.
Evolution Strategies (ES) are a powerful class of blackbox optimization techniques that recently became a competitive alternative to state-of-the-art policy gradient (PG) algorithms for reinforcement learning (RL). We propose a new method for improving accuracy of the ES algorithms, that as opposed to recent approaches…
We consider a family of tight contact structures on the three-dimensional torus and we compute the relative Contact Homology by using the variational theory of critical points at infinity. We will also show some algebraic equivariant homology reductions.
MSFA clusters high-dimensional spatial data using spline-based covariance structures.
problem Clustering high-dimensional spatial data with flexible covariance structures.
method Mixture of spatial factor analyzers with spline-based covariance and matrix variate factor analyzers for dimensionality reduction.
result Proposed models accurately infer and differentiate distinct spatial patterns in tensor-variate data.
Reinforcement learning mimics expert behavior.
problem Learning from expert demonstrations in reinforcement learning.
method Reduction to reinforcement learning with a stationary reward.
result Expert reward can be recovered and imitation learning is bounded.
Proposes variance reduction techniques for sliced Wasserstein distance estimation.
problem Intractability of estimating sliced Wasserstein distances.
method Uses control variates based on Gaussian approximations of projected measures.
result Significant reduction in variance of SW distance estimators.
The paper studies symmetry reduction and optimal control on Riemannian manifolds.
problem Symmetry reduction and optimal control on Riemannian manifolds.
method Derivation of reduced equations of motion for variational problems on Lie groups and Riemannian homogeneous spaces.
result Derivation of geodesic equations and reduced equations of motion for various applications.
The paper simplifies conditions for optimal paths on manifolds avoiding obstacles.
problem Finding optimal paths on manifolds avoiding obstacles.
method Study of sufficient conditions for optimality on Riemannian manifolds and Lie groups.
result New conditions for optimality are provided in terms of matrix invertibility.
In statistics and machine learning, approximation of an intractable integration is often achieved by using the unbiased Monte Carlo estimator, but the variances of the estimation are generally high in many applications. Control variates approaches are well-known to reduce the variance of the estimation. These control v…
This work improves variational inference by reducing gradient variance.
problem Hard optimization of flexible variational distributions.
method Control variate based on quadratic approximation of the model's mean and covariance.
result Significant improvement in gradient variance and optimization convergence.
There are no known exact formulas for the valuation of a number of exotic options, and this is particularly true for options under discrete monitoring and for American style options. Therefore, one usually recourses to a Monte Carlo Simulation approach, amongst other numerical methods, to estimate the value of these op…
In this note, we study the relationship between the variational gap and the variance of the (log) likelihood ratio. We show that the gap can be upper bounded by some form of dispersion measure of the likelihood ratio, which suggests the bias of variational inference can be reduced by making the distribution of the like…
The Black Box Variational Inference (Ranganath et al. (2014)) algorithm provides a universal method for Variational Inference, but taking advantage of special properties of the approximation family or of the target can improve the convergence speed significantly. For example, if the approximation family is a transforma…
We reduce variance in Bures-Wasserstein variational inference.
problem High variance in Monte Carlo approximations of Bures-Wasserstein gradients.
method Control variates to reduce variance in the forward step.
result Proposed estimator reduces variance by orders of magnitude.
We discuss the use of Dirac structures to obtain a better understanding of the geometry of a class of optimal control problems and their reduction by symmetries. In particular we will show how to extend the reduction of Dirac structures recently proposed by Yoshimura and Marsden [Yo09] to describe the reduction of a cl…
SDR outperforms IDR in multimodal data analysis, especially with fewer samples.
problem Understanding and optimizing data efficiency in multimodal representation learning.
method Generative linear model to synthesize multimodal data, comparing IDR and SDR methods.
result Linear SDR methods yield higher-quality, more succinct reduced-dimensional representations with smaller datasets.
The paper presents the application of Variational Autoencoders (VAE) for data dimensionality reduction and explorative analysis of mass spectrometry imaging data (MSI). The results confirm that VAEs are capable of detecting the patterns associated with the different tissue sub-types with performance than standard appro…
LMMVAE improves VAE for correlated data by separating latent variables into fixed and random parts.
problem Correlated data in tabular and image datasets.
method Integrates random effects into VAE architecture, separating latent variables into fixed and random parts.
result Significant improvement in reconstruction error and likelihood loss on unseen data.
GD-VAEs learn dynamics from observations using geometric and topological information.
problem Learning parsimonious representations of nonlinear dynamics from observations.
method Develops data-driven methods incorporating geometric and topological information using Variational Autoencoders (VAEs).
result GD-VAEs provide methods for learning reduced dimensional representations of nonlinear dynamics.
NCV uses neural networks to improve Monte Carlo integration.
problem Improving variance reduction in parametric Monte Carlo integration.
method NCV combines a normalizing flow and a neural network to approximate the integrand and solve the integral equation, with a neural importance sampler to estimate the difference.
result NCV achieves state-of-the-art performance in light transport simulation with reduced noise and negligible bias.
We introduce a dimension reduction method for visualizing the clustering structure obtained from a finite mixture of Gaussian densities. Information on the dimension reduction subspace is obtained from the variation on group means and, depending on the estimated mixture model, on the variation on group covariances. The…
The purpose of this paper is twofold. One is to give a survey of our study on the reductions of harmonic bundles, and the other is to explain a simple application in the study of TERP structure. In particular, we investigate the asymptotic behaviour of the "new supersymmetric index" for variation of pure polarized TERP…
A new method for SVGD reduces variance in high dimensions.
problem High-dimensional variance in SVGD.
method Grassmann Stein Variational Gradient Descent (GSVGD) projects onto arbitrary subspaces and uses coupled Grassmann-valued diffusion.
result GSVGD explores high-dimensional problems with intrinsic low-dimensional structure efficiently.
New method reduces variance in Bayesian inverse problems.
problem High variance in Monte Carlo estimates for inverse problems.
method Conditional neural control variates based on Stein's identity.
result Substantial variance reduction across different inverse problems.
In this article we study constrained variational problems in one independent variable defined on the space of integral curves of a Frenet system in a homogeneous space G/H. We prove that if the Lagrangian is G-invariant and coisotropic then the extremal curves can be found by quadratures. Our proof is constructive and …