A new method for 3D surface registration using dynamic programming.
problem Elastic shape registration of 3D surfaces.
method Optimization over a subset of reparametrizations using dynamic programming.
result Proposes an algorithm that produces a solution closer to optimal than gradient-based methods.
Weight normalization and reparametrized gradient descent adaptively regularize weights and converge to minimum l2 norm solutions.
problem Adapting to non-convex weight normalization for convergence to minimum l2 norm solutions.
method Weight normalization and reparametrized projected gradient descent (rPGD) for overparametrized least-squares regression.
result rPGD converges close to the minimum l2 norm solution, even for far-from-zero initializations.
We classify compact Kähler surfaces with nonconstant Killing potentials such that all integral curves of their gradients are reparametrized geodesics.
Improved VI with Price's gradient estimator for target log-density.
problem Approximating target distributions from unnormalized log-densities.
method Stochastic gradient-based variational inference with Price's gradient estimator.
result Identifies Price's gradient as the key to WVI's superior performance.
Monge SAM improves deep learning by making sharpness-aware minimization invariant to reparametrizations.
problem Non-invariance of sharpness-aware minimization (SAM) to reparametrizations.
method Introduces Monge SAM, a reparametrization-invariant version of SAM using a Riemannian metric.
result Monge SAM enhances robustness and generalization compared to previous methods.
Quantized Variational Inference improves ELBO optimization with fast convergence.
problem Maximizing Evidence Lower Bound (ELBO) for variational inference.
method Optimal Voronoi Tesselation for variance-free gradients, Richardson extrapolation for asymptotic improvement.
result Quantized Variational Inference leads to fast convergence with comparable computational cost.
Efficiently infers latent SDEs with scalable memory and time costs.
problem Inference of latent SDEs with high time and memory complexity.
method Amortized reparametrization of expectations under linear SDEs, coupled with efficient gradient approximation.
result Achieves similar performance to adjoint sensitivities with fewer model evaluations.
Proposes spred for solving L1 penalty with SGD.
problem Solving L1 penalty in optimization problems. method Reparametrization and SGD approach.
result Proves spred as an exact differentiable solver of L1. A vector field on a Riemannian manifold is called geodesic if its integral curves are reparametrized geodesics. We classify compact Kähler manifolds admitting nontrivial real-holomorphic geodesic gradient vector fields that satisfy an additional integrability condition. They are all biholomorphic to bundles of complex …
BBVI converges nearly dimensionally independent for log-concave targets.
problem Efficiently optimizing variational parameters in high-dimensional spaces.
method Proved convergence rate of BBVI with reparametrization gradient for log-concave targets.
result BBVI converges with nearly independent dimension dependence for log-concave targets.
Develop gradient boosting for estimating covariate-dependent GP distributions in insurance.
problem Estimating covariate-dependent Generalized Pareto distributions in insurance.
method Developing a statistical learning theory for gradient boosting.
result Deriving non-asymptotic error bounds for the boosting estimator.
This paper puts forth a new formulation and algorithm for the elastic matching problem on unparametrized curves and surfaces. Our approach combines the frameworks of square root normal fields and varifold fidelity metrics into a novel framework, which has several potential advantages over previous works. First, our var…
We introduce Natural Neural Networks, a novel family of algorithms that speed up convergence by adapting their internal representation during training to improve conditioning of the Fisher matrix. In particular, we show a specific example that employs a simple and efficient reparametrization of the neural network weigh…
Study on how reparametrization affects neural nets' parameter spaces from a geometric perspective.
problem Inconsistencies in flatness measures, optimization, and probability densities under reparametrization.
method Riemannian geometry to study invariance of neural nets under reparametrization.
result Invariance of neural nets is an inherent property if the metric is explicitly represented and transformation rules are correct.
Two new proofs of Gromov's non-squeezing theorem using curve reparametrization and gradient bounds.
problem Gromov's non-squeezing theorem in symplectic geometry.
method Reparametrization of pseudo-holomorphic curves and application of mean value inequality or Gromov-Schwarz lemma.
result Uniform bounds on the gradient of pseudo-holomorphic curves leading to compactness of moduli space.
DualVDT improves time-series forecasting with a novel dual reparametrized structure.
problem Time-series forecasting with improved performance and analytical rigor.
method Dual reparametrized variational mechanisms on VAE, latent score based generative model, reverse time stochastic differential equation, variational ancestral sampling, KL divergence reduction.
result Advanced performance in time-series forecasting with reduced KL divergence.
Gradient flow in parameters equals linear interpolation in outputs.
problem Understanding and optimizing training algorithms in deep learning.
method Proving equivalence between gradient flow in parameter space and linear interpolation in output space, and deriving formulas for global minima.
result Gradient flow in parameters can be transformed into linear interpolation in outputs, leading to global minima.
Evolution Strategies (ES) are a powerful class of blackbox optimization techniques that recently became a competitive alternative to state-of-the-art policy gradient (PG) algorithms for reinforcement learning (RL). We propose a new method for improving accuracy of the ES algorithms, that as opposed to recent approaches…
Improved diffusion bridge sampling with rKL-LD loss.
problem Improving sampling from unnormalized distributions using diffusion bridges.
method Employing the rKL-LD loss instead of the Log Variance (LV) loss for diffusion bridges.
result rKL-LD consistently outperforms LV loss in diffusion bridges.
We introduce a new algorithm for approximate inference that combines reparametrization, Markov chain Monte Carlo and variational methods. We construct a very flexible implicit variational distribution synthesized by an arbitrary Markov chain Monte Carlo operation and a deterministic transformation that can be optimized…
New findings show a balance between data fit and complexity in kernel hyperparameters.
problem Overcorrelation due to reparametrization of kernel hyperparameters.
method Reparametrization of kernel hyperparameters and analysis of marginal likelihood.
result Data fit term influences all other kernel hyperparameters, not just the complexity penalty.
We prove an optimal control on the time-dependent measure of a measurable set under a reparametrized Lagrangian mean curvature flow of almost calibrated submanifolds in a Calabi-Yau manifold. Moreover we give a classification of those Lagrangian translating solitons in Cm that evolve by this reparametrized …
We consider pairs of a non-empty compact connected and locally connected Hausdorff space and a real-valued continuous function. Our aim is to measure the difference between this kind of the pairs. In this notes we introduce new pseudodistances between pairs associated with reparametrization invariant seminorms. We fini…
We propose a second-order (Hessian or Hessian-free) based optimization method for variational inference inspired by Gaussian backpropagation, and argue that quasi-Newton optimization can be developed as well. This is accomplished by generalizing the gradient computation in stochastic backpropagation via a reparametriza…
Paper proves trajectories of Chaplygin systems are reparametrized geodesics.
problem Understanding trajectories of Chaplygin systems.
method Constructive proof using modified Riemannian metrics.
result Reparametrized geodesics of Chaplygin systems.
Completeness of surface metrics established for Sobolev spaces.
problem Ensuring completeness of reparametrization-invariant Sobolev metrics on surface spaces.
method Recasting completeness criteria for infinite-dimensional Riemannian manifolds and applying geometric estimates based on the Michael--Simon--Sobolev inequality.
result Established metric and geodesic completeness for specific Sobolev metrics on immersed surfaces, validating Mumford's conjecture.
A new numerical framework simplifies elastic surface matching and comparison.
problem Challenging problem in surface comparison and matching in computer vision.
method Relaxing the geodesic boundary constraint using a varifold fidelity metric.
result Flexibility to deal with arbitrary topologies and sampling patterns, scalability to large meshes.
We simplify SVI volatility smile constraints for three sub-SVIs without numerical methods.
problem No arbitrage constraints for SVI volatility smiles.
method Explicit domain derivation for sub-SVIs without numerical procedures.
result Explicit no arbitrage domains for Symmetric SVI, Vanishing Upward/Downward SVI, and SSVI.
Motivated by Demailly's strategy towards the Kobayashi hyperbolicity conjecture, we study the action on the k-jets of germs of holomorphic discs in a complex manifold X of the reparametrization group of k-jets of germs of biholomorphisms of the source. This reparametrization group is a subgroup of the general linear gr…
New approach uses SPG for semantic communication without a known channel model.
problem Designing efficient semantic communication systems without a known channel model.
method Applying Stochastic Policy Gradient (SPG) for reinforcement learning.
result Achieves comparable performance to model-aware approaches with a decreased convergence rate.
Reparameterization of variational auto-encoders with continuous random variables is an effective method for reducing the variance of their gradient estimates. In the discrete case, one can perform reparametrization using the Gumbel-Max trick, but the resulting objective relies on an argmax operation and is non-dif…
We analyze a notion of multiple valued sections of a vector bundle over an abstract smooth Riemannian manifold, which was suggested by W. Allard in the unpublished note "Some useful techniques for dealing with multiple valued functions" and generalizes Almgren's Q-valued functions. We study some relevant properties o…
A new variational method improves deep neural network inference.
problem Overparametrized deep neural networks struggle with variational approximations.
method A novel variational family with two independent linear subspaces.
result State-of-the-art performance across various tasks and datasets.
The Ricci flow on the 2-sphere with marked points is shown to converge in all three stable, semi-stable, and unstable cases. In the stable case, the flow was known to converge without any reparametrization, and a new proof of this fact is given. The semi-stable and unstable cases are new, and it is shown that the flow …
Paper shows surfaces with same SRNF but different shapes.
problem SRNF degeneracy in shape space.
method Introduced Square Root Normal Field (SRNF) to represent and compare surfaces.
result Examples of surfaces with same SRNF but different shapes.
We reparametrize ReLU NNs as splines to understand their learning dynamics.
problem Understanding the learning dynamics and inductive bias of neural networks.
method Reparametrize ReLU NNs as continuous piecewise linear splines to study learning dynamics.
result Standard weight initializations yield very flat functions, leading to strength and type of implicit regularization.
In this article we investigate a first order reparametrization-invariant Sobolev metric on the space of immersed curves. Motivated by applications in shape analysis where discretizations of this infinite-dimensional space are needed, we extend this metric to the space of Lipschitz curves, establish the wellposedness of…
This study compares two methods for sampling with transport maps, finding flow-based proposals work better for multimodal distributions.
problem Sampling from distributions with complex geometries.
method Compares two approaches: (i) proposal draws from the flow and (ii) reparametrization.
result Flow-based proposals are more effective for multimodal distributions in high dimensions, while reparametrization methods are more robust in other scenarios.
In this paper we study the shape space of curves with values in a homogeneous space M=G/K, where G is a Lie group and K is a compact Lie subgroup. We generalize the square root velocity framework to obtain a reparametrization invariant metric on the space of curves in M. By identifying curves in M with thei…
Study on completeness of Sobolev metrics on manifold-valued curves.
problem Completeness of Sobolev metrics on spaces of manifold-valued curves.
method Analysis of reparametrization invariant Sobolev metrics of order n≥2. result Sobolev immersions are metrically and geodesically complete for several important cases of metrics.
The space of all immersed closed curves of rotation degree 0 in the plane modulo reparametrizations has the same homotopy groups as the circle times the 2-sphere.
Research shows deep generative models' likelihoods are unreliable for anomaly detection.
problem Anomaly detection using deep generative models' likelihoods is unreliable.
method Examined the behavior of distribution densities through reparametrization.
result The likelihoods used for anomaly detection rely on strong and implicit hypotheses.
The chapter reviews metrics for comparing curves, focusing on quotient elastic and square root velocity metrics.
problem Comparing and analyzing shapes of curves.
method Construction and theoretical properties of quotient elastic metrics, special case of square root velocity metric, numerical approaches for estimation.
result Simplified expression for the square root velocity metric distance.
Adjusting the learning rate schedule in stochastic gradient methods is an important unresolved problem which requires tuning in practice. If certain parameters of the loss function such as smoothness or strong convexity constants are known, theoretical learning rate schedules can be applied. However, in practice, such …
MetFlow combines MCMC and VI efficiently for better inference.
problem Combining MCMC and VI for efficient inference.
method Introduces MetFlow, a novel MCMC algorithm with Normalizing Flows, and a new method to combine it with VI.
result MetFlow produces expressive variational families with improved computational efficiency.
Study disproves conjecture about metric completion of curve spaces.
problem Completeness properties of spaces of immersed curves with reparametrization-invariant metrics.
method Examined Sobolev-type metrics on real-valued immersed curves, demonstrating multiple distinct limit points.
result Metric completion of spaces of immersed open curves includes multiple distinct limit points, not a single point as previously conjectured.
New connections share geodesics with superintegrable systems.
problem Understanding geodesics in affine connections related to superintegrable systems.
method Analyzing dual-geodesics and comparing them across different connections.
result Certain torsion-free affine connections associated with second order superintegrable systems share the same dual-geodesics.
Signatures provide a succinct description of certain features of paths in a reparametrization invariant way. We propose a method for classifying shapes based on signatures, and compare it to current approaches based on the SRV transform and dynamic programming.