arXiv research

This paper tackles non-vacuous generalization bounds in ReLU networks by resolving rescaling invariances.

The abstract discusses nonuniqueness results for specific Riemannian invariants.

New Lipschitz bound for ReLU networks resists weight rescaling.

It has been empirically observed that the flatness of minima obtained from training deep networks seems to correlate with better generalization. However, for deep networks with positively homogeneous activations, most measures of sharpness/flatness are not invariant to rescaling of the network parameters, corresponding…

New method extends invariant reduction to rescaled geometric structures.

Localizes Wodzicki residue for logarithm of differential operators.

We consider a one-parameter family of strictly convex hypersurfaces in $\mathbb{R}^{n+1}$ moving with speed $- K^αν$, where $ν$ denotes the outward-pointing unit normal vector and $α\geq \frac{1}{n+2}$. For $α> \frac{1}{n+2}$, we show that the flow converges to a round sphere after rescaling. In the affine invariant ca…

"Ends of hyperbolic 3-manifolds should support canonical Wick Rotations, so they realize effective interactions of their ending globally hyperbolic spacetimes of constant curvature." We develop a consistent sector of WR-rescaling theory in 3D gravity, that, in particular, concretizes the above guess for many geometrica…

Classifies self-dual almost-Kähler 4-manifolds, proving uniqueness up to rescaling.

We consider inverse curvature flows in hyperbolic space with starshaped initial hypersurface, driven by positive powers of a homogeneous curvature function. The solutions exist for all time and, after rescaling, converge to a sphere.

Estimates LLC for deep linear networks up to 100M parameters.

Study of mean curvature flows with conical singularities using mathematical techniques.

A new method to improve deep neural networks using weight rescaling.

The notion of flat minima has played a key role in the generalization studies of deep learning models. However, existing definitions of the flatness are known to be sensitive to the rescaling of parameters. The issue suggests that the previous definitions of the flatness might not be a good measure of generalization, b…

In the work of Ammann, Dahl and Humbert it has turned out that the Yamabe invariant on closed manifolds is a bordism invariant below a certain threshold constant. A similar result holds for a spinorial analogon. These threshold constants are characterized through Yamabe-type equations on products of spheres with rescal…

The study examines conditions that prevent null geodesic lines in spacetimes, impacting cosmological geometry.

Paper rigorously defines Feynman graph integrals on Kähler manifolds.

In this short notes, we discuss monotonicity formulas under various rescaled versions of Ricci flow. The main result is Theorem \ref{theo rescaled}.

The paper calculates spectral torsion for rescaled Dirac operators on manifolds.

The paper studies the free elastic flow of closed curves and finds their asymptotic shape converges to a circle.

New spectral torsion defined for rescaled Dirac operators.

For a Riemannian manifold $M$, we determine some curvature properties of a tangent bundle equipped with the rescaled metric.The main aim of this paper is to give explicit formulae for the rescaled metric on $TM$, and investigate the geodesics on the tangent bundle with respect to the rescaled Sasaki metric.

Study geometric characterization of asymptotic pseudodifferential calculus on spinor bundles.

The paper calculates the noncommutative residue for a rescaled Dirac operator on 6D manifolds.

Rescaling expansiveness proven for k*-expansive vector fields.

We demonstrate a family of Strichartz estimates for the conformally invariant Klein-Gordon equation on a class of asymptotically de Sitter spaces with C^2 metrics by using well-known local Strichartz estimates and a rescaling argument. This class of metrics includes de Sitter space. We also give an application of the e…

Fast algorithm for rescaling vectors with clipping, improving training efficiency.

A new method to rescale ReLU neural networks based on path-lifting.

We produce longtime solutions to the Kähler-Ricci flow for complete Kähler metrics on $\Bbb C ^n$ without assuming the initial metric has bounded curvature, thus extending results in [3]. We prove the existence of a longtime bounded curvature solution emerging from any complete $U(n)$-invariant Kähler metric with non-n…

We prove a local index theorem of Atiyah-Singer type for Dirac operators on manifolds with a Lie structure at infinity (Lie manifolds for short). With the help of a renormalized supertrace, defined on a suitable class of regularizing operators, the proof of the index theorem relies on a rescaling technique similar in s…

We exploit the spinor description of four-dimensional Walker geometry, and conformal rescalings of such, to describe the local geometry of four-dimensional neutral geometries with algebraically degenerate self-dual Weyl curvature and an integrable distribution of alpha-planes (algebraically special real alpha-geometry)…

New method learns optimal prediction strategies in adversarial games.

We consider the inverse curvature flows $\dot x=F^{-p}ν$ of closed star-shaped hypersurfaces in Euclidean space in case $0<p\not=1$ and prove that the flow exists for all time and converges to infinity, if $0<p<1$, while in case $p>1$, the flow blows up in finite time, and where we assume the initial hypersurface to be…

Consider an asymptotically flat Riemannian manifold $(M,g)$ of dimension $n \geq 3$ with nonempty compact boundary. We recall the harmonic conformal class $[g]_h$ of the metric, which consists of all conformal rescalings given by a harmonic function raised to an appropriate power. The geometric significance is that eve…

The paper constructs bundles and recovers Kirillov character formula.

We revisit the choice of SGD for training deep neural networks by reconsidering the appropriate geometry in which to optimize the weights. We argue for a geometry invariant to rescaling of weights that does not affect the output of the network, and suggest Path-SGD, which is an approximate steepest descent method with …

We consider a variant of online convex optimization in which both the instances (input vectors) and the comparator (weight vector) are unconstrained. We exploit a natural scale invariance symmetry in our unconstrained setting: the predictions of the optimal comparator are invariant under any linear transformation of th…

Adaptive stepsizing improves sampling in Bayesian neural networks.

We prove that the Ricci flow g(t) starting at any metric on the euclidean space that is invariant by a transitive nilpotent Lie group N, can be obtained by solving an ODE for a curve of nilpotent Lie brackets. By using that this ODE is the negative gradient flow of a homogeneous polynomial, we obtain that g(t) is type-…

Study proves existence and uniqueness of ancient flows from cones.

We define risk-free portfolios using three gauge invariant differential operators that require such portfolios to be insensitive to price changes, to be self-financing, and to produce a zero real return so there are no risk-free profits. This definition identifies the risk-free rate as the return of an infinitely diver…

Method determines latent dimensionality in international trade flows.

In the present paper, we find a system of non-linear ODEs that gives rotationally invariant solutions to the Kapustin-Witten equations in 4-dimensional Euclidean space. We explicitly solve these ODEs in some special cases and find decaying rational solutions, which provide solutions to the Kapustin-Witten equations. Th…

Improved LLM pre-training performance through better weight and variance control.

Let X and Y be finite-type CW-complexes (X connected, Y simply connected), such that the rational cohomology ring of Y is a k-rescaling of the rational cohomology ring of X. Assume H^*(X,Q) is a Koszul algebra. Then, the homotopy Lie algebra pi_*(Omega Y) tensor Q equals, up to k-rescaling, the graded rational Lie alge…

Let $(M,g)$ be an $n-$dimensional Riemannian manifold and $T_{1}^{1}(M)$ be its $(1,1)-$tensor bundle equipped with the rescaled Sasaki type metric $$ which rescale the horizontal part by a nonzero differentiable function $f$. In the present paper, we discuss curvature properties of the Levi-Civita connectio…

We show that we can release the rigidity of the skew Howe duality process for ${\mathfrak sl}_n$ knot invariants by rescaling the quantum Weyl group action, and recover skein modules for web-tangles. This skew Howe duality phenomenon can be extended to the affine ${\mathfrak sl}_m$ case, corresponding to looking at tan…

We show that the action functional of the nonlinear sigma model with gravitino considered in a previous article [18] is invariant under rescaled conformal transformations, super Weyl transformations and diffeomorphisms. We give a careful geometric explanation how a variation of the metric leads to the corresponding var…