arXiv research

This study connects Jacobian regularization to adversarial robustness and improves generalization.

Design of reliable systems must guarantee stability against input perturbations. In machine learning, such guarantee entails preventing overfitting and ensuring robustness of models against corruption of input data. In order to maximize stability, we analyze and develop a computationally efficient implementation of Jac…

Recovering hidden influence networks from cascade data using Jacobian-based machine learning.

New algorithms estimate Jacobian matrices for large-scale machine learning.

The Jacobian Conjecture is proven for all Jacobian maps.

GrokAlign aligns Jacobians to accelerate grokking in deep networks.

Study shows connections between Jacobian torsors and Fermat curves.

Normalizing flows optimize Jacobian determinant for unique likelihood objective.

We propose a fast, simple and robust algorithm for computing shortest paths and distances on Riemannian manifolds learned from data. This amounts to solving a system of ordinary differential equations (ODEs) subject to boundary conditions. Here standard solvers perform poorly because they require well-behaved Jacobians…

A new method for faster bandwidth selection in Gaussian kernel ridge regression.

This work proves the asymptotic freeness of layerwise Jacobians in MLPs with Haar orthogonal matrices.

Efficiently regularizes deep learning models using Jacobian nuclear norm.

Abstract: Unknown status of Jacobian Conjecture, proof has a gap.

A well-conditioned Jacobian spectrum has a vital role in preventing exploding or vanishing gradients and speeding up learning of deep neural networks. Free probability theory helps us to understand and handle the Jacobian spectrum. We rigorously show almost sure asymptotic freeness of layer-wise Jacobians of deep neura…

The Jacobian conjecture is simplified using polynomial mappings.

New approach ties loss curvature to model performance in deep learning.

Paper tackles Hessian/Jacobian-free stochastic bilevel optimization with ${O}(ε^{-1.5})$ complexity.

A new method speeds up training of deep models by avoiding Jacobian determinant computation.

Recent work (Pennington et al, 2017) suggests that controlling the entire distribution of Jacobian singular values is an important design consideration in deep learning. Motivated by this, we study the distribution of singular values of the Jacobian of the generator in Generative Adversarial Networks (GANs). We find th…

The paper discusses fractional Sobolev immersions of flat domains into 3D space.

We derive an analytic formula for the dual Jacobian matrix of a generalised hyperbolic tetrahedron. Two cases are considered: a mildly truncated and a prism truncated tetrahedron. The Jacobian for the latter arises as an analytic continuation of the former, that falls in line with a similar behaviour of the correspondi…

The paper extends infinite-width analysis to neural network Jacobians, revealing convergence to Gaussian processes and linear ODEs.

This work introduces the concept of tangent space regularization for neural-network models of dynamical systems. The tangent space to the dynamics function of many physical systems of interest in control applications exhibits useful properties, e.g., smoothness, motivating regularization of the model Jacobian along sys…

We provide a characterization for complex analytic curves among two-dimensional minimal graphs in $\mathbb{R}^{4}$ via the Jacobian

We extend the well-known result that any $f \in W^{1,n}(Ω,\mathbb{R}^n)$, $Ω\subset \mathbb{R}^n$ with strictly positive Jacobian is actually continuous: it is also true for fractional Sobolev spaces $W^{s,\frac{n}{s}}(Ω)$ for any $s \geq \frac{n}{n+1}$, where the sign condition on the Jacobian is understood in a distr…

New method reduces deep learning training costs by approximating vector-jacobian products.

We study the energy distribution of harmonic 1-forms on a compact hyperbolic Riemann surface $S$ where a short closed geodesic is pinched. If the geodesic separates the surface into two parts, then the Jacobian torus of $S$ develops into a torus that splits. If the geodesic is nonseparating then the Jacobian torus of $…

The Jacobian of Douady-Earle extension equals 1 only for isometries.

Recent years have witnessed the rapid development of block coordinate update (BCU) methods, which are particularly suitable for problems involving large-sized data and/or variables. In optimization, BCU first appears as the coordinate descent method that works well for smooth problems or those with separable nonsmooth …

The aim here is to continue the investigation in \cite{AB} of Jacobians of a Klein surface and also to correct an error in \cite{AB}.

To a compact Riemann surface of genus g can be assigned a principally polarized abelian variety (PPAV) of dimension g, the Jacobian of the Riemann surface. The Schottky problem is to discern the Jacobians among the PPAVs. Buser and Sarnak showed, that the square of the first successive minimum, the squared norm of the …

We show that the Goldman flows preserve the holomorphic structure on the moduli space of homomorphisms of the fundamental group of a Riemann surface into U(1), in other words the Jacobian.

Generalization in nonlinear least squares can be studied via algorithmic stability and effective dimension.

Generative adversarial networks (GANs) are notoriously difficult to train and the reasons underlying their (non-)convergence behaviors are still not completely understood. By first considering a simple yet representative GAN example, we mathematically analyze its local convergence behavior in a non-asymptotic way. Furt…

Geometrically represents path integral reduction Jacobian for interacting systems.

This work relaxes energy constraints in self-attention layers for a more general analysis.

We compute some value of the harmonic volume for the Fermat sextic. Using this computation, we prove that some special algebraic cycle in the Jacobian variety of the Fermat sextic is not algebraically equivalent to zero.

JacNet learns Jacobians to enforce structure on derivatives for invertibility and Lipschitz functions.

The paper studies global invertibility of maps on Finsler manifolds.

The Jacobian matrix (or the gradient for single-output networks) is directly related to many important properties of neural networks, such as the function landscape, stationary points, (local) Lipschitz constants and robustness to adversarial attacks. In this paper, we propose a recursive algorithm, RecurJac, to comput…

We show that there are separated nets in the Euclidean plane which are not biLipschitz equivalent to the integer lattice. The argument is based on the construction of a continuous function which is not the Jacobian of a biLipschitz map.

New method recovers causal graphs from data scores in non-linear models.

We study families of Galois covers of curves of positive genus. It is known that under a numerical condition these families yield Shimura subvarieties generically contained in the Jacobian locus. We prove that there are only 6 families satisfying this condition, all of them in genus 2,3 or 4. We also show that these fa…

To any compact Riemann surface of genus g one may assign a principally polarized abelian variety of dimension g, the Jacobian of the Riemann surface. The Jacobian is a complex torus, and a Gram matrix of the lattice of a Jacobian is called a period Gram matrix. This paper provides upper and lower bounds for all the ent…

The paper develops methods to reduce deployment risk under dynamic covariate shifts.

Paper tackles catastrophic forgetting in sequential learning.

Differentiable methods fail due to spectral issues in Jacobians.

The paper proves a distribution claim for neural network Jacobians.