We introduce a new algorithm for approximate inference that combines reparametrization, Markov chain Monte Carlo and variational methods. We construct a very flexible implicit variational distribution synthesized by an arbitrary Markov chain Monte Carlo operation and a deterministic transformation that can be optimized…
Improved diffusion bridge sampling with rKL-LD loss.
problem Improving sampling from unnormalized distributions using diffusion bridges.
method Employing the rKL-LD loss instead of the Log Variance (LV) loss for diffusion bridges.
result rKL-LD consistently outperforms LV loss in diffusion bridges.
Proposes spred for solving L1 penalty with SGD.
problem Solving L1 penalty in optimization problems. method Reparametrization and SGD approach.
result Proves spred as an exact differentiable solver of L1. MetFlow combines MCMC and VI efficiently for better inference.
problem Combining MCMC and VI for efficient inference.
method Introduces MetFlow, a novel MCMC algorithm with Normalizing Flows, and a new method to combine it with VI.
result MetFlow produces expressive variational families with improved computational efficiency.
In the original version of the Variational Autoencoder, Kingma et al. assume Gaussian distributions for the approximate posterior during the inference and for the output during the generative process. This assumptions are good for computational reasons, e.g. we can easily optimize the parameters of a neural network usi…
SCORE technique reduces BO's high-dimensional search costs.
problem Bayesian optimization's high computational costs in high-dimensional spaces.
method 1D reparametrization trick to maintain linear time complexity.
result Successfully finds global minimum in high-dimensional optimization.
AEVB improves understanding of latent variable models.
problem Training latent variable models efficiently and understanding their limitations.
method Motivates AEVB from EM, emphasizing approximate E-step and M-step.
result AEVB tightens ELBO, improving model training.
Reparameterization of variational auto-encoders with continuous random variables is an effective method for reducing the variance of their gradient estimates. In the discrete case, one can perform reparametrization using the Gumbel-Max trick, but the resulting objective relies on an argmax operation and is non-dif…
We propose a second-order (Hessian or Hessian-free) based optimization method for variational inference inspired by Gaussian backpropagation, and argue that quasi-Newton optimization can be developed as well. This is accomplished by generalizing the gradient computation in stochastic backpropagation via a reparametriza…
A standard Variational Autoencoder, with a Euclidean latent space, is structurally incapable of capturing topological properties of certain datasets. To remove topological obstructions, we introduce Diffusion Variational Autoencoders with arbitrary manifolds as a latent space. A Diffusion Variational Autoencoder uses t…
Understanding generalization in reinforcement learning (RL) is a significant challenge, as many common assumptions of traditional supervised learning theory do not apply. We focus on the special class of reparameterizable RL problems, where the trajectory distribution can be decomposed using the reparametrization trick…
Study on how reparametrization affects neural nets' parameter spaces from a geometric perspective.
problem Inconsistencies in flatness measures, optimization, and probability densities under reparametrization.
method Riemannian geometry to study invariance of neural nets under reparametrization.
result Invariance of neural nets is an inherent property if the metric is explicitly represented and transformation rules are correct.
A new method for 3D surface registration using dynamic programming.
problem Elastic shape registration of 3D surfaces.
method Optimization over a subset of reparametrizations using dynamic programming.
result Proposes an algorithm that produces a solution closer to optimal than gradient-based methods.
DualVDT improves time-series forecasting with a novel dual reparametrized structure.
problem Time-series forecasting with improved performance and analytical rigor.
method Dual reparametrized variational mechanisms on VAE, latent score based generative model, reverse time stochastic differential equation, variational ancestral sampling, KL divergence reduction.
result Advanced performance in time-series forecasting with reduced KL divergence.
New topological quantum gravity theories linked to Ricci flow.
problem Quantum gravity and geometric flows on manifolds.
method BRST quantization, gauging symmetries, localization.
result Path integral localized to Ricci flow solutions.
New approach uses SPG for semantic communication without a known channel model.
problem Designing efficient semantic communication systems without a known channel model.
method Applying Stochastic Policy Gradient (SPG) for reinforcement learning.
result Achieves comparable performance to model-aware approaches with a decreased convergence rate.
We investigate the use of alternative divergences to Kullback-Leibler (KL) in variational inference(VI), based on the Variational Dropout \cite{kingma2015}. Stochastic gradient variational Bayes (SGVB) \cite{aevb} is a general framework for estimating the evidence lower bound (ELBO) in Variational Bayes. In this work, …
The inference of deep hierarchical models is problematic due to strong dependencies between the hierarchies. We investigate a specific transformation of the model parameters based on the multivariate distributional transform. This transformation is a special form of the reparametrization trick, flattens the hierarchy a…
New findings show a balance between data fit and complexity in kernel hyperparameters.
problem Overcorrelation due to reparametrization of kernel hyperparameters.
method Reparametrization of kernel hyperparameters and analysis of marginal likelihood.
result Data fit term influences all other kernel hyperparameters, not just the complexity penalty.
We prove an optimal control on the time-dependent measure of a measurable set under a reparametrized Lagrangian mean curvature flow of almost calibrated submanifolds in a Calabi-Yau manifold. Moreover we give a classification of those Lagrangian translating solitons in Cm that evolve by this reparametrized …
The paper analyzes Bayesian neural networks trained with VI, proving a law of large numbers for different schemes.
problem Training Bayesian neural networks with variational inference.
method Analyzes three training schemes: exact estimation, Bayes by Backprop, and Minimal VI.
result All training schemes converge to the same mean-field limit.
Monge SAM improves deep learning by making sharpness-aware minimization invariant to reparametrizations.
problem Non-invariance of sharpness-aware minimization (SAM) to reparametrizations.
method Introduces Monge SAM, a reparametrization-invariant version of SAM using a Riemannian metric.
result Monge SAM enhances robustness and generalization compared to previous methods.
We consider pairs of a non-empty compact connected and locally connected Hausdorff space and a real-valued continuous function. Our aim is to measure the difference between this kind of the pairs. In this notes we introduce new pseudodistances between pairs associated with reparametrization invariant seminorms. We fini…
Paper derives CLT for Bayesian neural networks trained with variational inference.
problem Analyzing the fluctuation behavior of Bayesian neural networks trained with different variational inference schemes.
method Rigorous derivation of CLT for three variational inference schemes: idealized, Bayes-by-Backprop, and Minimal VI.
result Minimal VI scheme has larger variances but is more computationally efficient.
Paper bridges VAEs and KDEs for more flexible posterior estimation.
problem Limitations of Gaussian latent space in VAEs and challenges in KL-divergence estimation.
method Approximate posterior with KDEs and derive upper bound of KL-divergence in ELBO.
result Epanechnikov kernel minimizes KL-divergence upper bound asymptotically.
Study of skateboard flips as continuous curves in SO(3) group.
problem Characterize skateboard flip tricks as continuous motions.
method Model flips as curves in SO(3), analyze lifts to S3, derive formulas. result There are only four distinct flip tricks up to continuous deformation.
Nash's theorem proved with Günther's trick
problem Proving Nash's smooth embedding theorem
method Using Günther's trick
result Nash's theorem proved
Paper proves trajectories of Chaplygin systems are reparametrized geodesics.
problem Understanding trajectories of Chaplygin systems.
method Constructive proof using modified Riemannian metrics.
result Reparametrized geodesics of Chaplygin systems.
Explains Conway's tangle trick and its mathematical origins.
problem Understanding the relationship between braids and elliptic curves.
method Discusses the tangle trick, its mathematical underpinnings, and historical context.
result Establishes the connection between braids and elliptic curves.
Completeness of surface metrics established for Sobolev spaces.
problem Ensuring completeness of reparametrization-invariant Sobolev metrics on surface spaces.
method Recasting completeness criteria for infinite-dimensional Riemannian manifolds and applying geometric estimates based on the Michael--Simon--Sobolev inequality.
result Established metric and geodesic completeness for specific Sobolev metrics on immersed surfaces, validating Mumford's conjecture.
Unified framework for gradient estimation in combinatorial spaces.
problem Scaling relaxed gradient estimators to large combinatorial distributions.
method Introducing stochastic softmax tricks within the perturbation model framework.
result Stochastic softmax tricks improve model performance and discover more latent structure.
We prove all knots can be transformed into a trefoil using special diagrams.
problem Transforming any knot into a trefoil using magic tricks.
method Introducing knotholder diagrams to encode transformations.
result All knots can be transformed into a trefoil.
Geometric trick simplifies link homotopy and concordance.
problem Homotopy and concordance of links in homology spheres.
method Relative Whitney trick to remove double points.
result Links in homology spheres can be simplified to topologically slice links.
Outlier based Robust Principal Component Analysis (RPCA) requires centering of the non-outliers. We show a "bias trick" that automatically centers these non-outliers. Using this bias trick we obtain the first RPCA algorithm that is optimal with respect to centering.
The Gumbel-max trick and its extensions simplify sampling from categorical distributions in machine learning.
problem Sampling from categorical distributions with unnormalized probabilities.
method Extensions of the Gumbel-max trick for various applications.
result Simplified and efficient methods for sampling and gradient estimation.
Tricks improve retail product image classification accuracy.
problem Retail Product Image Classification
method Various tricks including a new LCA layer, Instagram-pretrained Convnet, and Maximum Entropy loss.
result Increased accuracy of fine-tuned convnets by a large margin.
A new numerical framework simplifies elastic surface matching and comparison.
problem Challenging problem in surface comparison and matching in computer vision.
method Relaxing the geodesic boundary constraint using a varifold fidelity metric.
result Flexibility to deal with arbitrary topologies and sampling patterns, scalability to large meshes.
A new gradient estimator for categorical distributions reduces bias and variance.
problem Intractability of gradients for categorical distributions in discrete latent variable models.
method CatLog-Derivative trick and IndeCateR gradient estimator.
result IndeCateR reduces bias and variance of gradients for categorical distributions.
Establishes necessary and sufficient conditions for smooth triviality of Lie subalgebras and Lie ideals, and proves Moser's trick for foliations.
problem Smooth triviality of Lie subalgebras and Lie ideals
method Establishing necessary and sufficient conditions and proving Moser's trick for foliations
result Direct proof of Moser's trick for foliations
Efficiently infers latent SDEs with scalable memory and time costs.
problem Inference of latent SDEs with high time and memory complexity.
method Amortized reparametrization of expectations under linear SDEs, coupled with efficient gradient approximation.
result Achieves similar performance to adjoint sensitivities with fewer model evaluations.
We simplify SVI volatility smile constraints for three sub-SVIs without numerical methods.
problem No arbitrage constraints for SVI volatility smiles.
method Explicit domain derivation for sub-SVIs without numerical procedures.
result Explicit no arbitrage domains for Symmetric SVI, Vanishing Upward/Downward SVI, and SSVI.
Motivated by Demailly's strategy towards the Kobayashi hyperbolicity conjecture, we study the action on the k-jets of germs of holomorphic discs in a complex manifold X of the reparametrization group of k-jets of germs of biholomorphisms of the source. This reparametrization group is a subgroup of the general linear gr…
Triple-point Whitney trick classifies ornaments of 3-manifolds.
problem Classifying ornaments of 3-manifolds in high dimensions.
method Triple-point Whitney trick applied to orientable manifolds.
result Classification of ornaments by the μ-invariant.
Alexander trick applied to homology spheres for manifold homeomorphisms.
problem Group of homeomorphisms of contractible manifolds.
method Strong uniqueness statement for one-sided h-cobordisms.
result Group of homeomorphisms is contractible for d≥6. Embolic volume of compact manifolds is defined in terms of Berger's embolic inequality. In this paper, we show a result of relating embolic volume to the first Betti number. The proof relies on Gromov's covering argument appeared in systolic geometry. Berger called this method covering trick. We exploit and present mor…
The Square Root Normal Field (SRNF), introduced by Jermyn et al. in [3], provides a way of representing immersed surfaces in R3, and equipping the set of these immersions with a "distance function" (to be precise, a pseudometric) that is easy to compute. Importantly, this distance function is invariant under…
Expands Bredon's trick for applications in geometry and topology.
problem Local-to-global extension principles in geometric and topological contexts.
method Novel applications and frameworks for stratified pseudomanifolds, Ricci flow, and persistent homology.
result Establishes Bredon's trick as a unifying framework.
We analyze a notion of multiple valued sections of a vector bundle over an abstract smooth Riemannian manifold, which was suggested by W. Allard in the unpublished note "Some useful techniques for dealing with multiple valued functions" and generalizes Almgren's Q-valued functions. We study some relevant properties o…