A new method normalizes activations to match batch normalization without batch dependence.
problem Performance degradation with batch-independent normalization techniques.
method Proxy-Normalizing Activations
result Proxy-Normalization technique emulates batch normalization's behavior and performance.
Online Normalization normalizes neural network activations without batching for better accuracy.
problem Theoretical limitations of Batch Normalization and its inapplicability to certain network types.
method Introduces an unbiased gradient computation technique for normalized activations without using batches.
result Equivalent accuracy to Batch Normalization without batch usage.
This paper analyzes how normalization layers improve neural network training.
problem Improving generalization performance and training speed of neural networks.
method Global convergence analysis of two-layer neural networks with ReLU activations and Weight Normalization.
result Introduction of normalization layers changes the optimization landscape, enabling faster convergence.
ILM-Norm normalizes instances individually for better performance.
problem Learning to normalize parameters for improved model performance.
method ILM-Norm learns normalization parameters via feature feed-forward and gradient back-propagation.
result ILM-Norm consistently improves model performance across different architectures and tasks.
Normalization techniques have only recently begun to be exploited in supervised learning tasks. Batch normalization exploits mini-batch statistics to normalize the activations. This was shown to speed up training and result in better models. However its success has been very limited when dealing with recurrent neural n…
The paper analyzes various normalization methods in deep learning.
problem Lack of mathematical tools to analyze normalization methods.
method Proposed a lemma to define tools, analyzed BN, LN, WN, GN.
result Normalization methods can be unified on a sphere, improving training stability and weight norm.
Distance, normals, and double normals for real plane curves with singularities
problem Relation between normals and double normals and critical points of the squared distance function for real algebraic curves with singularities
method Investigate the topological discriminant of the distance function
result The topological discriminant consists of the evolute and distinguished normal lines at algebraic singular points
Normalizes pseudo-Einstein contact forms for easier analysis.
problem Understanding pseudo-Einstein contact forms.
method Constructing intrinsic CR normal coordinates using parabolic normal coordinates.
result Normal form for pseudo-Einstein contact forms.
This paper reviews normalization techniques for DNNs.
problem Improving training speed and generalization of DNNs.
method Taxonomy of normalization methods, decomposition of activation methods.
result Insight for designing new normalization techniques.
Optimized normalization layers improve domain generalization.
problem Improving model generalization across different domains.
method Learning separate normalization parameters per domain using multiple normalization methods (batch and instance).
result State-of-the-art accuracy on domain generalization benchmarks.
We consider the Laplace normal vector field of relatively normalized ruled surfaces with non-vanishing Gaussian curvature in the three-dimensional Euclidean space R3. We determine all ruled surfaces and all relative normalizations for which the Laplace normal image degenerates into a point or into a curve…
Monotonicity of normalized implied-volatility coordinates under no-arbitrage
problem Monotonicity of normalized implied-volatility coordinates under no-arbitrage
method Elementary discrete no-arbitrage proof
result Monotonicity principle extended to Bachelier implied volatility
Proposes adversarial normalization for multi-domain image segmentation.
problem Current image normalization is per-dataset, limiting multi-domain segmentation.
method Adversarial training to learn common normalizing functions across multiple datasets.
result Optimal normalizer improves segmentation accuracy and realism.
Training state-of-the-art, deep neural networks is computationally expensive. One way to reduce the training time is to normalize the activities of the neurons. A recently introduced technique called batch normalization uses the distribution of the summed input to a neuron over a mini-batch of training cases to compute…
Proposes a flexible normalization method to handle multi-modal data.
problem Reduced effectiveness of batch normalization in multi-modal distributions.
method Extends normalization to multiple means and variances, detecting data modes on-the-fly.
result Outperforms batch normalization and other methods in various experiments.
The paper defines normal forms for rational 3-tangles and shows a sequence of moves to transform one form to another.
problem Understanding and manipulating rational 3-tangles.
method Definition of normal forms and sequence of normal jump moves.
result There is a sequence of normal jump moves leading to equivalent normal forms of rational 3-tangles.
Four improvements to Batch Normalization improve deep learning performance.
problem Improving Batch Normalization for better deep learning performance.
method Proposed improvements include reasoning about current examples, Ghost Batch Normalization, weight decay regularization, and a new normalization algorithm for small batch sizes.
result Performance gains across all batch sizes with no additional computation during training.
Developed criteria for selecting non-normalized models using NCE and score matching.
problem No information criteria for non-normalized models estimated by NCE or score matching.
method Developed information criteria based on discrepancy measures for non-normalized models estimated by NCE or score matching.
result The proposed criteria enable selection of the appropriate non-normalized model in a data-driven manner.
This paper investigates the impact of normalization on deep neural networks for click-through rate prediction.
problem The effect of normalization on deep neural network models for CTR estimation.
method Systematic study of various normalization approaches applied to feature embedding and MLP part of DNN models.
result Correct normalization significantly enhances model performance, as demonstrated by extensive experiments on real-world datasets.
The study finds abundant normal generators for mapping class groups.
problem Understanding normal generation in mapping class groups.
method Analyzing restrictions on invariant subsurfaces and Teichmüller spaces.
result Reducible mapping classes can normally generate mapping class groups based on their asymptotic translation lengths.
This paper deals with skew ruled surfaces in the Euclidean space E3 which are equipped with polar normalizations, that is, relative normalizations such that the relative normal at each point of the ruled surface lies on the corresponding polar plane. We determine the invariants of a such normalized ruled …
New method trains normalizing flows using entropy-regularized transport.
problem Training continuous normalizing flows efficiently.
method Formulates flows as gradients of scalar potentials, training only these potentials.
result Trains normalizing flows without explicit flow computation during training.
The paper characterizes surfaces in 4D space forms with flat normal connection.
problem Characterizing surfaces in 4D space forms with specific geometric properties.
method Analyzing linearly dependent conditions and using properties of sectional curvature.
result Characterizations of space-like and time-like surfaces with flat normal connection.
We define a 2-normal surface to be one which intersects every 3-simplex of a triangulated 3-manifold in normal triangles and quadrilaterals, with one or two exceptions. The possible exceptions are a pair of octagons, a pair of unknotted tubes, an octagon and a tube, or a 12-gon. In this paper we use the theory of criti…
The paper updates Bayesian CMA-ES with normal Wishart and proves lower expected covariance.
problem Improving the Bayesian CMA-ES algorithm with normal Wishart prior.
method Revisits Bayesian CMA-ES, proves lower expected covariance in normal Wishart, and presents a generalized model.
result Proves that the expected covariance is lower in the normal Wishart prior model due to convexity of the inverse.
Layer normalization improves federated learning with skewed labels.
problem Label skewness in federated learning datasets.
method Identified feature normalization as key mechanism; applied to latent features before classifier.
result Normalization accelerates global training and improves convergence under extreme label shift.
Following Matveev, a k-normal surface in a triangulated 3-manifold is a generalization of both normal and (octagonal) almost normal surfaces. Using spines, complexity, and Turaev-Viro invariants of 3-manifolds, we prove the following results: 1) a minimal triangulation of a closed irreducible or a bounded hyperbolic 3-…
In this paper, we study normal homogeneous Finsler spaces. We first define the notion of a normal homogeneous Finsler space, using the method of isometric submersion of Finsler metrics. Then we study the geometric properties. In particular, we establish a technique to reduce the classification of normal homogeneous Fin…
Paper classifies rational 3-tangles using normal forms and minimal coordinates.
problem Classifying rational 3-tangles up to isotopy.
method Defined normal form and normal coordinate, investigated minimal coordinates, constructed contractible simplicial complex.
result Simplicial complex of normal forms is contractible, leading to classification of rational 3-tangles.
Channel normalization prevents vanishing gradients in convolutional neural networks.
problem Vanishing gradients in convolutional neural networks during optimization.
method Channel normalization, which centers and normalizes each channel individually.
result Channel normalization avoids vanishing gradients, enabling efficient optimization.
This paper considers asymptotically hyperbolic manifolds with a finite boundary intersecting the usual infinite boundary -- cornered asymptotically hyperbolic manifolds -- and proves a theorem of Cartan-Hadamard type near infinity for the normal exponential map on the finite boundary. As a main application, a normal fo…
This study investigates global normalization in neural models, showing its effectiveness in search-aware training.
problem Theoretical equivalence of global and local normalization in high-capacity models, practical advantage unclear.
method Continuous relaxation of beam search for training globally normalized recurrent sequence models.
result Globally normalized models are more effective than locally normalized ones in inexact search.
PL-MCMC samples from normalizing flows' conditional distributions.
problem Sampling from complex conditional distributions learned by normalizing flows.
method Metropolis-Hastings implementation of PL-MCMC.
result PL-MCMC asymptotically samples from exact conditional distributions.
New normalization technique balances positive and negative weights for faster convergence.
problem Balancing positive and negative weights for faster convergence.
method Transformation of layer weights instead of outputs, balancing positive and negative contributions.
result Balanced normalization leads to faster convergence on standard benchmarks.
A virtual link diagram is called normal if the associated abstract link diagram is checkerboard colorable, and a virtual link is normal if it has a normal diagram as a representative.In this paper, we introduce a method of converting a virtual link diagram to a normal virtual link diagram by use of the double covering …
Calculation of the log-normalizer is a major computational obstacle in applications of log-linear models with large output spaces. The problem of fast normalizer computation has therefore attracted significant attention in the theoretical and applied machine learning literature. In this paper, we analyze a recently pro…
We establish existence and regularity results for normal Coulomb frames in the normal bundle of two-dimensional surfaces of disc-type embedded in Euclidean spaces of higher dimensions.
We study the normal holonomy group, i.e. the holonomy group of the normal connection, of a CR-submanifold of a complex space form. We complete the local classification of normal holonomies for complex submanifolds. We show that the normal holonomy group of a coisotropic submanifold acts as the holonomy representation o…
We say A is a quasi-normal subgroup of the group G if the commensurator of A in G is all of G. We develop geometric versions of commensurators in finitely generated groups. In particular, g is an element of the commensurator of A in G iff the Hausdorff distance between A and gA is finite. We show that a quasi-normal su…
Paper proves non-existence of certain hypersurfaces in complex quadric.
problem Non-existence of Hopf real hypersurfaces with parallel normal Jacobi operator.
method Introducing C-parallel and Reeb parallel normal Jacobi operators, proving non-existence theorems. result Non-existence of Hopf real hypersurfaces with C-parallel normal Jacobi operator. We show that a topologically minimal disk in a tetrahedron with index n is either a normal triangle, a normal quadrilateral, or a normal helicoid with boundary length 4(n+1). This mirrors geometric results of Colding and Minicozzi.
In this article, we first describe a normal form of real-analytic, Levi-nondegenerate submanifolds of CN of codimension d ≥ 1 under the action of formal biholomorphisms, that is, of perturbations of Levi-nondegenerate hyperquadrics. We give a sufficient condition on the formal normal form that ensures that the n…
Proposes a unified normalization method for multi-domain medical images.
problem Inadequate joint information across multiple datasets hinders image segmentation performance.
method Adversarial and task-driven normalization approach to learn a common normalizing function across multiple datasets.
result Jointly normalized images improve segmentation accuracy by up to 57.5%.
TaskNorm improves meta-learning performance by rethinking batch normalization.
problem Challenges in batch normalization for meta-learning with deep networks.
method Developed TaskNorm, a novel approach to batch normalization for meta-learning.
result TaskNorm consistently improves meta-learning performance across various datasets and meta-learning approaches.
Adaptive feature normalization improves model robustness to extraneous variables.
problem Degrading model performance due to extraneous variables in deep learning.
method Adaptive feature normalization using instance normalization instead of batch normalization.
result Adaptive normalization leads to significant performance gains across different datasets and architectures.
Formal normal form created for real-smooth hypersurfaces.
problem Real-smooth hypersurfaces in complex spaces.
method Iterative normalization procedure.
result Formal normal form constructed for a large class of hypersurfaces.
We introduce in this paper normal twistor equations for differential forms and study their solutions, the so-called normal conformal Killing forms. The twistor equations arise naturally from the canonical normal Cartan connection of conformal geometry. Reductions of its holonomy are related to solutions of the normal t…
A new proof shows almost every normal to a smooth convex body intersects at least 6 normals from different points.
problem The conjecture about normals to convex bodies in high dimensions.
method Short proof of Y. Martinez-Maure's result for n≥3. result Almost every normal through a boundary point intersects at least 6 normals from different points.