New algorithm combines MCMC and variational methods for flexible implicit distributions.
problem Approximate inference for complex continuous models.
method Combines reparametrization, MCMC, and variational methods to construct flexible implicit distributions.
result Easily applicable to arbitrary continuous models without computing log density ratios.
Improved diffusion bridge sampling with rKL-LD loss.
problem Improving sampling from unnormalized distributions using diffusion bridges.
method Employing the rKL-LD loss instead of the Log Variance (LV) loss for diffusion bridges.
result rKL-LD consistently outperforms LV loss in diffusion bridges.
Proposes spred for solving L 1 L_1 L 1 penalty with SGD.
problem Solving L 1 L_1 L 1 penalty in optimization problems. method Reparametrization and SGD approach.
result Proves spred as an exact differentiable solver of L 1 L_1 L 1 . Transforms hierarchical model parameters to decouple dependencies and improve inference.
problem Problematic dependencies between hierarchical model parameters.
method Transformation of model parameters using multivariate distributional transform.
result Decouples transformed parameters a priori, leading to faster inference.
MetFlow combines MCMC and VI efficiently for better inference.
problem Combining MCMC and VI for efficient inference.
method Introduces MetFlow, a novel MCMC algorithm with Normalizing Flows, and a new method to combine it with VI.
result MetFlow produces expressive variational families with improved computational efficiency.
SCORE technique reduces BO's high-dimensional search costs.
problem Bayesian optimization's high computational costs in high-dimensional spaces.
method 1D reparametrization trick to maintain linear time complexity.
result Successfully finds global minimum in high-dimensional optimization.
AEVB improves understanding of latent variable models.
problem Training latent variable models efficiently and understanding their limitations.
method Motivates AEVB from EM, emphasizing approximate E-step and M-step.
result AEVB tightens ELBO, improving model training.
New method uses adversarial networks to improve image quality in autoencoders.
problem Blurriness in autoencoder-generated images due to Gaussian assumptions.
method Integrates adversarial networks to optimize parameters without Gaussian assumptions.
result Improves image quality by allowing better representation of multimodal distributions.
We propose a second-order (Hessian or Hessian-free) based optimization method for variational inference inspired by Gaussian backpropagation, and argue that quasi-Newton optimization can be developed as well. This is accomplished by generalizing the gradient computation in stochastic backpropagation via a reparametriza…
Direct optimization of discrete variational auto-encoders using arg max.
problem Optimizing discrete latent variables in variational auto-encoders.
method Direct optimization through arg max without softmax relaxations.
result Empirical effectiveness of direct loss minimization in discrete latent variables.
Diffusion Variational Autoencoders capture topological properties of datasets.
problem Standard VAEs struggle with topological properties of certain datasets.
method Introduces Diffusion VAEs with transition kernels of Brownian motion on arbitrary manifolds.
result Diffusion VAEs can capture topological properties of synthetic datasets.
Study on how reparametrization affects neural nets' parameter spaces from a geometric perspective.
problem Inconsistencies in flatness measures, optimization, and probability densities under reparametrization.
method Riemannian geometry to study invariance of neural nets under reparametrization.
result Invariance of neural nets is an inherent property if the metric is explicitly represented and transformation rules are correct.
A new method for 3D surface registration using dynamic programming.
problem Elastic shape registration of 3D surfaces.
method Optimization over a subset of reparametrizations using dynamic programming.
result Proposes an algorithm that produces a solution closer to optimal than gradient-based methods.
New topological quantum gravity theories linked to Ricci flow.
problem Quantum gravity and geometric flows on manifolds.
method BRST quantization, gauging symmetries, localization.
result Path integral localized to Ricci flow solutions.
DualVDT improves time-series forecasting with a novel dual reparametrized structure.
problem Time-series forecasting with improved performance and analytical rigor.
method Dual reparametrized variational mechanisms on VAE, latent score based generative model, reverse time stochastic differential equation, variational ancestral sampling, KL divergence reduction.
result Advanced performance in time-series forecasting with reduced KL divergence.
New approach uses SPG for semantic communication without a known channel model.
problem Designing efficient semantic communication systems without a known channel model.
method Applying Stochastic Policy Gradient (SPG) for reinforcement learning.
result Achieves comparable performance to model-aware approaches with a decreased convergence rate.
Study on generalization in reparameterizable RL, deriving new guarantees.
problem Understanding generalization in reparameterizable RL.
method Using supervised learning and transfer learning theory, derived guarantees on the gap between expected and empirical return.
result Generalization capability of reparameterizable RL is related to multiple factors including smoothness of the environment transition, reward, and policy function class.
Proves properness of action on map space for complex reparametrization group.
problem Properness of action of $PSL(n+1, {f C})$ on L k p L_k^p L k p -maps. method Proof of properness using v v v -stability and reparametrization group. result Proved properness of action of $PSL(n+1, {f C})$ on L k p L_k^p L k p -maps. Revisits and proves a reparametrization theorem for multi-valued graphs in higher codimension.
problem Analyzing multi-valued sections of vector bundles and proving a reparametrization theorem.
method Develops properties of Q Q Q -multisections and provides a geometric proof. result Elementary and purely geometric proof of a reparametrization theorem for multi-valued graphs.
The paper analyzes Bayesian neural networks trained with VI, proving a law of large numbers for different schemes.
problem Training Bayesian neural networks with variational inference.
method Analyzes three training schemes: exact estimation, Bayes by Backprop, and Minimal VI.
result All training schemes converge to the same mean-field limit.
New findings show a balance between data fit and complexity in kernel hyperparameters.
problem Overcorrelation due to reparametrization of kernel hyperparameters.
method Reparametrization of kernel hyperparameters and analysis of marginal likelihood.
result Data fit term influences all other kernel hyperparameters, not just the complexity penalty.
This research explores using Alpha-Divergences in variational dropout for better inference.
problem Improving variational inference methods using alternative divergences.
method Extending the Stochastic Gradient Variational Bayes (SGVB) framework with Alpha-Divergences.
result The α α α -divergence with α i g h t a r r o w 1 α
ightarrow 1 α i g h t a r r o w 1 yields the lowest training error and optimizes the ELBO. Paper derives CLT for Bayesian neural networks trained with variational inference.
problem Analyzing the fluctuation behavior of Bayesian neural networks trained with different variational inference schemes.
method Rigorous derivation of CLT for three variational inference schemes: idealized, Bayes-by-Backprop, and Minimal VI.
result Minimal VI scheme has larger variances but is more computationally efficient.
A new method uses signatures to classify shapes efficiently.
problem Classifying shapes succinctly and invariantly.
method Proposes a method using signatures for shape classification.
result Outperforms current methods like SRV transform and dynamic programming.
Monge SAM improves deep learning by making sharpness-aware minimization invariant to reparametrizations.
problem Non-invariance of sharpness-aware minimization (SAM) to reparametrizations.
method Introduces Monge SAM, a reparametrization-invariant version of SAM using a Riemannian metric.
result Monge SAM enhances robustness and generalization compared to previous methods.
We consider pairs of a non-empty compact connected and locally connected Hausdorff space and a real-valued continuous function. Our aim is to measure the difference between this kind of the pairs. In this notes we introduce new pseudodistances between pairs associated with reparametrization invariant seminorms. We fini…
Paper bridges VAEs and KDEs for more flexible posterior estimation.
problem Limitations of Gaussian latent space in VAEs and challenges in KL-divergence estimation.
method Approximate posterior with KDEs and derive upper bound of KL-divergence in ELBO.
result Epanechnikov kernel minimizes KL-divergence upper bound asymptotically.
Weight normalization and reparametrized gradient descent adaptively regularize weights and converge to minimum l2 norm solutions.
problem Adapting to non-convex weight normalization for convergence to minimum l2 norm solutions.
method Weight normalization and reparametrized projected gradient descent (rPGD) for overparametrized least-squares regression.
result rPGD converges close to the minimum l2 norm solution, even for far-from-zero initializations.
Paper defines embolic volume and relates it to Betti number using the covering trick.
problem Relating embolic volume to topological invariants.
method Covering trick from systolic geometry applied to Berger's inequality.
result Relates embolic volume to the first Betti number.
Study of skateboard flips as continuous curves in S O ( 3 ) SO(3) S O ( 3 ) group.
problem Characterize skateboard flip tricks as continuous motions.
method Model flips as curves in S O ( 3 ) SO(3) S O ( 3 ) , analyze lifts to S 3 S^3 S 3 , derive formulas. result There are only four distinct flip tricks up to continuous deformation.
Paper proves trajectories of Chaplygin systems are reparametrized geodesics.
problem Understanding trajectories of Chaplygin systems.
method Constructive proof using modified Riemannian metrics.
result Reparametrized geodesics of Chaplygin systems.
A new method centers outliers in robust PCA without manual intervention.
problem Outliers in robust PCA require manual centering, complicating the analysis.
method Introduces a 'bias trick' to automatically center non-outliers.
result First optimal RPCA algorithm with automatic centering.
New method for elastic curve and surface matching.
problem Elastic matching of unparametrized curves and surfaces.
method Combines square root normal fields and varifold fidelity metrics.
result Numerical examples demonstrate the approach's effectiveness.
Nash's theorem proved with Günther's trick
problem Proving Nash's smooth embedding theorem
method Using Günther's trick
result Nash's theorem proved
Explains Conway's tangle trick and its mathematical origins.
problem Understanding the relationship between braids and elliptic curves.
method Discusses the tangle trick, its mathematical underpinnings, and historical context.
result Establishes the connection between braids and elliptic curves.
Study controls volume measure for Lagrangian flows in Calabi-Yau manifolds.
problem Controlling volume measure for Lagrangian flows in Calabi-Yau manifolds.
method Optimal control on time-dependent measure of a measurable set under reparametrized Lagrangian mean curvature flow.
result Classification of Lagrangian translating solitons in C m \mathbb{C}^m C m that evolve by the reparametrized flow. Completeness of surface metrics established for Sobolev spaces.
problem Ensuring completeness of reparametrization-invariant Sobolev metrics on surface spaces.
method Recasting completeness criteria for infinite-dimensional Riemannian manifolds and applying geometric estimates based on the Michael--Simon--Sobolev inequality.
result Established metric and geodesic completeness for specific Sobolev metrics on immersed surfaces, validating Mumford's conjecture.
Unified framework for gradient estimation in combinatorial spaces.
problem Scaling relaxed gradient estimators to large combinatorial distributions.
method Introducing stochastic softmax tricks within the perturbation model framework.
result Stochastic softmax tricks improve model performance and discover more latent structure.
We prove all knots can be transformed into a trefoil using special diagrams.
problem Transforming any knot into a trefoil using magic tricks.
method Introducing knotholder diagrams to encode transformations.
result All knots can be transformed into a trefoil.
Geometric trick simplifies link homotopy and concordance.
problem Homotopy and concordance of links in homology spheres.
method Relative Whitney trick to remove double points.
result Links in homology spheres can be simplified to topologically slice links.
A new numerical framework simplifies elastic surface matching and comparison.
problem Challenging problem in surface comparison and matching in computer vision.
method Relaxing the geodesic boundary constraint using a varifold fidelity metric.
result Flexibility to deal with arbitrary topologies and sampling patterns, scalability to large meshes.
The Gumbel-max trick and its extensions simplify sampling from categorical distributions in machine learning.
problem Sampling from categorical distributions with unnormalized probabilities.
method Extensions of the Gumbel-max trick for various applications.
result Simplified and efficient methods for sampling and gradient estimation.
Tricks improve retail product image classification accuracy.
problem Retail Product Image Classification
method Various tricks including a new LCA layer, Instagram-pretrained Convnet, and Maximum Entropy loss.
result Increased accuracy of fine-tuned convnets by a large margin.
New methods improve on the Gumbel trick for sampling and estimating partition functions.
problem Improving sampling and partition function estimation for discrete distributions.
method Deriving a family of related methods, including low-rank perturbations.
result New methods provide superior properties with minimal additional computational cost.
Efficiently infers latent SDEs with scalable memory and time costs.
problem Inference of latent SDEs with high time and memory complexity.
method Amortized reparametrization of expectations under linear SDEs, coupled with efficient gradient approximation.
result Achieves similar performance to adjoint sensitivities with fewer model evaluations.
A new gradient estimator for categorical distributions reduces bias and variance.
problem Intractability of gradients for categorical distributions in discrete latent variable models.
method CatLog-Derivative trick and IndeCateR gradient estimator.
result IndeCateR reduces bias and variance of gradients for categorical distributions.
Establishes necessary and sufficient conditions for smooth triviality of Lie subalgebras and Lie ideals, and proves Moser's trick for foliations.
problem Smooth triviality of Lie subalgebras and Lie ideals
method Establishing necessary and sufficient conditions and proving Moser's trick for foliations
result Direct proof of Moser's trick for foliations
Doubling tricks help improve multi-armed bandit algorithms, but their effectiveness depends on the horizon length.
problem Improving the performance of multi-armed bandit algorithms using doubling tricks.
method Analyzed geometric and exponential doubling tricks for different horizon lengths.
result Geometric doubling tricks can conserve regret bounds in O ( T ) O(\sqrt{T}) O ( T ) , but not in O ( log T ) O(\log T) O ( log T ) . Exponential doubling tricks can conserve bounds in O ( log T ) O(\log T) O ( log T ) .