arXiv research

ISAAC Newton uses input-based curvature for efficient training.

New approach ties loss curvature to model performance in deep learning.

New defence against data-poisoning attacks in neural networks.

Sandpile Economics explains how economies can be prone to large crises from small shocks.

The paper uses Cartan moving frames to analyze data manifolds and neural network outputs.

Ricci-Filtration enhances retrieval-augmented generation rerankers for query-answer tasks by using discrete Ricci flow on graphs.

A new method for computing image curvature efficiently and accurately.

Statistical neurodynamics studies macroscopic behaviors of randomly connected neural networks. We consider a deep layered feedforward network where input signals are processed layer by layer. The manifold of input signals is embedded in a higher dimensional manifold of the next layer as a curved submanifold, provided t…

Ridgeless ReLU networks interpolate datasets and extrapolate based on curvature signs.

The Finsleroid-Finsler space is constructed over an underlying Riemannian space by the help of a scalar $g(x)$ and an input 1-form $b$ of unit length. Explicit form of the entailed tensors, as well as the respective spray coefficients, is evaluated. The involutive case means the framework in which the characteristic sc…

SOC-ICNN expands neural network representational capacity by using conic optimization.

Enhances deep learning robustness to noise without sacrificing clean data accuracy.

We study the geometry of deep (neural) networks (DNs) with piecewise affine and convex nonlinearities. The layers of such DNs have been shown to be {\em max-affine spline operators} (MASOs) that partition their input space and apply a region-dependent affine mapping to their input to produce their output. We demonstrat…

Paper investigates hardness of learning neural networks under manifold hypothesis.

The goal of this paper is to analyze the geometric properties of deep neural network classifiers in the input space. We specifically study the topology of classification regions created by deep networks, as well as their associated decision boundary. Through a systematic empirical investigation, we show that state-of-t…

GOIMDA selects inputs to maximize expected influence on a goal functional, reducing data acquisition needs.

Graph rewiring method alleviates over-squashing in GNNs.

State-of-the-art classifiers have been shown to be largely vulnerable to adversarial perturbations. One of the most effective strategies to improve robustness is adversarial training. In this paper, we investigate the effect of adversarial training on the geometry of the classification landscape and decision boundaries…

Neural networks learn discrete tasks on continuous data via emergent geometry.

Sharp Veronese rigidity theorem for submanifolds of unit ball.

A neural network deployed in the wild may be asked to make predictions for inputs that were drawn from a different distribution than that of the training data. A plethora of work has demonstrated that it is easy to find or synthesize inputs for which a neural network is highly confident yet wrong. Generative models are…

Over-parameterized neural networks generalize well in practice without any explicit regularization. Although it has not been proven yet, empirical evidence suggests that implicit regularization plays a crucial role in deep learning and prevents the network from overfitting. In this work, we introduce the gradient gap d…

The paper studies the asymptotic expansion of Gaussian integral operators on Riemannian submanifolds.

Paper provides efficient robustness certificates for neural networks.

Deep generative models provide a systematic way to learn nonlinear data distributions, through a set of latent variables and a nonlinear "generator" function that maps latent points into the input space. The nonlinearity of the generator imply that the latent space gives a distorted view of the input space. Under mild …

The problem of multiple surface clustering is a challenging task, particularly when the surfaces intersect. Available methods such as Isomap fail to capture the true shape of the surface nearby the intersection and result in incorrect clustering. The Isomap algorithm uses the shortest path between points. The main draw…

Constructs optimal symplectic connections for Kaehler metrics on holomorphic submersions.

Graph convolutional neural networks (GCNs) embed nodes in a graph into Euclidean space, which has been shown to incur a large distortion when embedding real-world graphs with scale-free or hierarchical structure. Hyperbolic geometry offers an exciting alternative, as it enables embeddings with much smaller distortion. …

The Finsleroid--Finsler space becomes regular when the norm $||b||=c$ of the input 1-form $b$ is taken to be an arbitrary positive scalar $c(x) < 1$. By performing required direct evaluations, the respective spray coefficients have been obtained in a simple and transparent form. The adequate continuation into the regul…

A novel ABC method for high-dimensional inverse problems using generative modeling and subset simulation.

New results show flat minima in neural networks suffer from high dimensionality.

Develops a dynamical method to prove the sharp Berezin-Li-Yau inequality.

A production function is a mathematical formalization in economics which denotes the relations between the output generated by a firm, an industry or an economy and the inputs that have been used in obtaining it. In this paper, we study the product production functions of 2 variables in terms of the geometry of their a…

Financial markets are notoriously complex environments, presenting vast amounts of noisy, yet potentially informative data. We consider the problem of forecasting financial time series from a wide range of information sources using online Gaussian Processes with Automatic Relevance Determination (ARD) kernels. We measu…

Proposes a new method to initialize neural networks by estimating global curvature of weights.

Several important algorithms for machine learning and data analysis use pairwise distances as input. On Riemannian manifolds these distances may be prohibitively costly to compute, in particular for large datasets. To tackle this problem, we propose a distance approximation which requires only a linear number of geodes…

Mirror flow in shallow neural networks shows similar implicit bias to gradient flow, with key differences in curvature penalties.

Mapping complex input data into suitable lower dimensional manifolds is a common procedure in machine learning. This step is beneficial mainly for two reasons: (1) it reduces the data dimensionality and (2) it provides a new data representation possibly characterised by convenient geometric properties. Euclidean spaces…

Rewiring networks using discrete geometry improves GNN training accuracy and reduces runtime.

We continue our investigations into Toda's algorithm [14,3]; a Weierstrass-type representation of Gauss curvature $K=-1$ surfaces in $\mathbb{R}^3$. We show that $C^0$ input potentials correspond in an appealing way to a special new class of surfaces, with $K=-1$, which we call $C^{1M}$. These are surfaces which may no…

Analyzes the Hessian of ReLU networks, proving skewed eigenvalue distribution.

A new method improves adversarial robustness and interpretability with reduced training time.

The Yamabe invariant is an invariant of a closed smooth manifold defined using conformal geometry and the scalar curvature. Recently, Petean showed that the Yamabe invariant is non-negative for all closed simply connected manifolds of dimension $\ge 5$. We extend this to show that Yamabe invariant is non-negative for a…

Linear bound on Betti numbers of negatively curved orbifolds.

The regularity theory for pluriclosed flow hinges on obtaining $C^α$ regularity for the metric assuming uniform equivalence to a background metric. This estimate was established in \cite{StreetsPCFBI} by an adaptation of ideas from Evans-Krylov, the key input being a sharp differential inequality satisfied by the assoc…

In the absence of explicit regularization, Kernel "Ridgeless" Regression with nonlinear kernels has the potential to fit the training data perfectly. It has been observed empirically, however, that such interpolated solutions can still generalize well on test data. We isolate a phenomenon of implicit regularization for…

ETC improves Transformer models for long and structured inputs.

New bounds explain deterministic non-smooth deep nets without large Lipschitz constants.