arXiv research

Improved sampling for Bayesian neural networks reduces vanishing acceptance rates and increases predictive accuracy.

Maxout networks study gradients and propose initialization strategies.

New L2 regularization improves softmax MAB performance.

Sigmoid-type networks avoid vanishing gradients with regularization and rescaling.

In this paper we prove the infinitesimal uniqueness theorem for the Newton potential of non simply connected bodies using the singularity theory approach. We consider the Newtonian potentials of the domains in ${\bf R}^n$ boundaries of which are the vanishing cycles on the level hypersurface of a holomorphic function w…

The paper is on the vanishing topology of singular Milnor fibres of holomorphic families of arbitrary square, symmetric and skew-symmetric matrices with sufficiently many parameters. We define vanishing cycles on such fibres, prove an extended form of the Damon-Pike $μ=τ$ conjecture about the families of a special type…

New findings on shrinking Ricci solitons with vanishing Bach-like tensors.

Paper proves vanishing terms in a second Poisson bracket for a specific system.

Vanishing long-term gradients are a major issue in training standard recurrent neural networks (RNNs), which can be alleviated by long short-term memory (LSTM) models with memory cells. However, the extra parameters associated with the memory cells mean an LSTM layer has four times as many parameters as an RNN with the…

It is well known that the problem of vanishing/exploding gradients is a challenge when training deep networks. In this paper, we describe another phenomenon, called vanishing nodes, that also increases the difficulty of training deep neural networks. As the depth of a neural network increases, the network's hidden node…

New algorithm explains neural network extrema for better model generalization.

On a Riemannian manifold we define a one-parameter family of Laplacians acting on sections of any bundle associated to the principal frame bundle via a representation, and show how various examples fit into this framework.

Using geometric quantization procedure, the quantization of algebra of observables for physical system with Ricci-flat phase space is obtained. In the classical case the appointed physical system is reduced to harmonic oscillator when the one real parameter is vanished.

Introduces new curvature concept for Kähler manifolds.

Variational autoencoders (VAEs) with an auto-regressive decoder have been applied for many natural language processing (NLP) tasks. The VAE objective consists of two terms, (i) reconstruction and (ii) KL regularization, balanced by a weighting hyper-parameter β. One notorious training difficulty is that the KL term ten…

A new formulation of the Anomaly flow in the case of vanishing slope parameter is given, where the dependence on the global section of the canonical bundle appears only in the initial data. This allows a natural unification of the Anomaly flow with the Kähler-Ricci flow.

Stagewise boosting improves gradient boosting for distributional regression.

We provide an explicit description of all rigid hypersurfaces that are equivalent to a Heisenberg sphere. These hypersurfaces are determined by 4 real parameters. The defining equations of the rigid spheres can also be viewed as the complete solution of a non-linear PDE that expresses the vanishing Cartan curvature con…

We derive explicit formulas for time decay, for the European call and put options at expiry, and use them to calculate analytical approximations to the price of the American put and early exercise boundary near expiry. We show that for many families of non-Gaussian processes used in empirical studies of financial marke…

Estimates change point in high-dimensional dynamic graphical models.

Gradient amplification boosts deep learning model performance without increasing training time.

New paradigm for Neural ODEs stabilizes training and improves model performance.

This paper generalize [7](math.GT/0601291): We construct new links invariants from g, a type I basic classical Lie superalgebra. The construction uses the existence of an unexpected replacement of the vanishing quantum dimension of typical module. Using this, we get a multivariable link invariant associated to any one …

The gradient-based optimization method for deep machine learning models suffers from gradient vanishing and exploding problems, particularly when the computational graph becomes deep. In this work, we propose the tangent-space gradient optimization (TSGO) for the probabilistic models to keep the gradients from vanishin…

The paper proves cohomology vanishing for a specific type of minimal submanifolds in a weighted Euclidean ball.

Ghost mechanism explains abrupt learning in RNNs, revealing constraints on optimization landscapes.

Express Wavenet is an improved optical diffractive neural network. At each layer, it uses wavelet-like pattern to modulate the phase of optical waves. For input image with n2 pixels, express wavenet reduce parameter number from O(n2) to O(n). Only need one percent of the parameters, and the accuracy is still very high.…

We simplify SVI volatility smile constraints for three sub-SVIs without numerical methods.

The paper constructs singularities for Lagrangian flow in Gibbons-Hawking spaces with vanishing mean curvature.

Using Weitzenböck techniques on any compact Riemannian spin manifold we derive inequalities that involve a real parameter and join the eigenvalues of the Dirac operator with curvature terms. The discussion of these inequalities yields vanishing theorems for the kernel of the Dirac operator $D$ and lower bounds for the …

Estimates change point in high dimensional time series models.

Study evaluates capacity and trainability of parametrized quantum circuits.

Spectral normalization stabilizes GANs by controlling gradient explosion and vanishing.

Analyzes Saito vanishing theorem using $L^{2}$ methods.

BPI models 2D patterns on multiple planes and 3D scene from a single image.

We consider smooth 1-parameter families of plane curves tangent to a semicubic parabola, when the curvature radius of their curves at the tangency point vanishes at the cusp point. We find the $\A$-normal form of these families, their envelopes and local patterns near the cusp. We obtain a new codimension 2 singularity…

The main object of Bayesian statistical inference is the determination of posterior distributions. Sometimes these laws are given for quantities devoid of empirical value. This serious drawback vanishes when one confines oneself to considering a finite horizon framework. However, assuming infinite exchangeability gives…

Nontrivial connectivity has allowed the training of very deep networks by addressing the problem of vanishing gradients and offering a more efficient method of reusing parameters. In this paper we make a comparison between residual networks, densely-connected networks and highway networks on an image classification tas…

This research improves binary classification by balancing overfitting and generalization with a novel Bayesian approach.

Overparametrized neural networks retain significant epistemic uncertainty even with sufficient data.

When the parameters are independently and identically distributed (initialized) neural networks exhibit undesirable properties that emerge as the number of layers increases, e.g. a vanishing dependency on the input and a concentration on restrictive families of functions including constant functions. We consider parame…

Recently the first author studied the bifurcation of critical points of families of functionals on a Hilbert space, which are parametrised by a compact and orientable manifold having a non-vanishing first integral cohomology group. We improve this result in two directions: topologically and analytically. From the analy…

The paper uses non-abelian Hodge theory to generalize Kodaira vanishing theorems.

Study on a knot invariant's vanishing order.

Capillarity functionals are parameter invariant functionals defined on classes of two-dimensional parametric surfaces in R3 as the sum of the area integral and a non homogeneous term of suitable form. Here we consider the case of a class of non homogenous terms vanishing at infinity for which the corresponding capillar…

Proves a vanishing property for symplectic manifold cohomology.

We consider a magnetic Schrödinger operator $H^h$, depending on a semiclassical parameter $h>0$, on a compact Riemannian manifold. We assume that there is no electric field. We suppose that the minimal value $b_0$ of the intensity of the magnetic field $b$ is strictly positive. We give a survey of the results on asympt…

We introduce a framework for studying the effects of self-interaction on the construction of point particle initial data in General Relativity. Within this framework we rigorously prove the vanishing mass claim made by Arnowitt, Deser and Misner regarding point sources. We identify a geometric structure and a scaling p…