arXiv research

A new method normalizes activations to match batch normalization without batch dependence.

Online Normalization is a new technique for normalizing the hidden activations of a neural network. Like Batch Normalization, it normalizes the sample dimension. While Online Normalization does not use batches, it is as accurate as Batch Normalization. We resolve a theoretical limitation of Batch Normalization by intro…

This paper analyzes how normalization layers improve neural network training.

Normalization techniques have only recently begun to be exploited in supervised learning tasks. Batch normalization exploits mini-batch statistics to normalize the activations. This was shown to speed up training and result in better models. However its success has been very limited when dealing with recurrent neural n…

This paper presents a normalization mechanism called Instance-Level Meta Normalization (ILM~Norm) to address a learning-to-normalize problem. ILM~Norm learns to predict the normalization parameters via both the feature feed-forward and the gradient back-propagation paths. ILM~Norm provides a meta normalization mechanis…

The paper analyzes various normalization methods in deep learning.

Distance, normals, and double normals for real plane curves with singularities

Normalizes pseudo-Einstein contact forms for easier analysis.

This paper reviews normalization techniques for DNNs.

We consider the Laplace normal vector field of relatively normalized ruled surfaces with non-vanishing Gaussian curvature in the three-dimensional Euclidean space $\mathbb{R}^{3}$. We determine all ruled surfaces and all relative normalizations for which the Laplace normal image degenerates into a point or into a curve…

Monotonicity of normalized implied-volatility coordinates under no-arbitrage

Training state-of-the-art, deep neural networks is computationally expensive. One way to reduce the training time is to normalize the activities of the neurons. A recently introduced technique called batch normalization uses the distribution of the summed input to a neuron over a mini-batch of training cases to compute…

The paper defines normal forms for rational 3-tangles and shows a sequence of moves to transform one form to another.

This paper investigates the impact of normalization on deep neural networks for click-through rate prediction.

We propose a simple but effective multi-source domain generalization technique based on deep neural networks by incorporating optimized normalization layers that are specific to individual domains. Our approach employs multiple normalization methods while learning separate affine parameters per domain. For each domain,…

The study finds abundant normal generators for mapping class groups.

This paper deals with skew ruled surfaces in the Euclidean space $\mathbb{E}^{3}$ which are equipped with polar normalizations, that is, relative normalizations such that the relative normal at each point of the ruled surface lies on the corresponding polar plane. We determine the invariants of a such normalized ruled …

New method trains normalizing flows using entropy-regularized transport.

The paper characterizes surfaces in 4D space forms with flat normal connection.

Image normalization is a critical step in medical imaging. This step is often done on a per-dataset basis, preventing current segmentation algorithms from the full potential of exploiting jointly normalized information across multiple datasets. To solve this problem, we propose an adversarial normalization approach for…

We define a 2-normal surface to be one which intersects every 3-simplex of a triangulated 3-manifold in normal triangles and quadrilaterals, with one or two exceptions. The possible exceptions are a pair of octagons, a pair of unknotted tubes, an octagon and a tube, or a 12-gon. In this paper we use the theory of criti…

Many statistical models are given in the form of non-normalized densities with an intractable normalization constant. Since maximum likelihood estimation is computationally intensive for these models, several estimation methods have been developed which do not require explicit computation of the normalization constant,…

Following Matveev, a k-normal surface in a triangulated 3-manifold is a generalization of both normal and (octagonal) almost normal surfaces. Using spines, complexity, and Turaev-Viro invariants of 3-manifolds, we prove the following results: 1) a minimal triangulation of a closed irreducible or a bounded hyperbolic 3-…

Layer normalization improves federated learning with skewed labels.

In this paper, we study normal homogeneous Finsler spaces. We first define the notion of a normal homogeneous Finsler space, using the method of isometric submersion of Finsler metrics. Then we study the geometric properties. In particular, we establish a technique to reduce the classification of normal homogeneous Fin…

Paper classifies rational 3-tangles using normal forms and minimal coordinates.

Normalization methods are a central building block in the deep learning toolbox. They accelerate and stabilize training, while decreasing the dependence on manually tuned learning rate schedules. When learning from multi-modal distributions, the effectiveness of batch normalization (BN), arguably the most prominent nor…

This paper considers asymptotically hyperbolic manifolds with a finite boundary intersecting the usual infinite boundary -- cornered asymptotically hyperbolic manifolds -- and proves a theorem of Cartan-Hadamard type near infinity for the normal exponential map on the finite boundary. As a main application, a normal fo…

PL-MCMC samples from normalizing flows' conditional distributions.

A virtual link diagram is called normal if the associated abstract link diagram is checkerboard colorable, and a virtual link is normal if it has a normal diagram as a representative.In this paper, we introduce a method of converting a virtual link diagram to a normal virtual link diagram by use of the double covering …

Calculation of the log-normalizer is a major computational obstacle in applications of log-linear models with large output spaces. The problem of fast normalizer computation has therefore attracted significant attention in the theoretical and applied machine learning literature. In this paper, we analyze a recently pro…

We establish existence and regularity results for normal Coulomb frames in the normal bundle of two-dimensional surfaces of disc-type embedded in Euclidean spaces of higher dimensions.

We study the normal holonomy group, i.e. the holonomy group of the normal connection, of a CR-submanifold of a complex space form. We complete the local classification of normal holonomies for complex submanifolds. We show that the normal holonomy group of a coisotropic submanifold acts as the holonomy representation o…

We say A is a quasi-normal subgroup of the group G if the commensurator of A in G is all of G. We develop geometric versions of commensurators in finitely generated groups. In particular, g is an element of the commensurator of A in G iff the Hausdorff distance between A and gA is finite. We show that a quasi-normal su…

We show that a topologically minimal disk in a tetrahedron with index $n$ is either a normal triangle, a normal quadrilateral, or a normal helicoid with boundary length 4(n+1). This mirrors geometric results of Colding and Minicozzi.

Paper proves non-existence of certain hypersurfaces in complex quadric.

In this article, we first describe a normal form of real-analytic, Levi-nondegenerate submanifolds of $C^N$ of codimension d $\ge$ 1 under the action of formal biholomorphisms, that is, of perturbations of Levi-nondegenerate hyperquadrics. We give a sufficient condition on the formal normal form that ensures that the n…

Formal normal form created for real-smooth hypersurfaces.

Proposes a unified normalization method for multi-domain medical images.

We introduce in this paper normal twistor equations for differential forms and study their solutions, the so-called normal conformal Killing forms. The twistor equations arise naturally from the canonical normal Cartan connection of conformal geometry. Reductions of its holonomy are related to solutions of the normal t…

A new proof shows almost every normal to a smooth convex body intersects at least 6 normals from different points.

One of the challenges in the study of generative adversarial networks is the instability of its training. In this paper, we propose a novel weight normalization technique called spectral normalization to stabilize the training of the discriminator. Our new normalization technique is computationally light and easy to in…

Minimal normal curvature immersions in the unit ball studied.

For orthonormal normal sections of two-dimensional immersions in R^4 we define torsion coefficients and a functional for the total torsion. We discuss normal sections which are critical for this functional. In particular, a global estimate for the torsion coefficients of a critical normal section in terms of the curvat…

Class of Newtonian dynamical systems admitting normal blow-up of points in Riemannian manifolds is considered. Geometric interpretation for weak normality condition, which arose earlier in the theory of dynamical systems admitting the normal shift of hypersurfaces, is found.

This paper revisits the Bayesian CMA-ES and provides updates for normal Wishart. It emphasizes the difference between a normal and normal inverse Wishart prior. After some computation, we prove that the only difference relies surprisingly in the expected covariance. We prove that the expected covariance should be lower…

A virtual link diagram is called normal if the associated abstract link diagram is checkerboard colorable, and a virtual link is normal if it has a normal diagram as a representative. Normal virtual links have some properties similar to classical links.In this paper, we introduce a method of converting a virtual link d…

The paper connects least area surfaces to quasi-normal surfaces in 3-manifolds.