arXiv research

Bayesian neural networks learn efficiently at infinite width, matching polynomial-width performance.

Study shows bound on Uryson width for specific 3D manifolds.

Study shows polynomial-width neural networks can closely approximate infinite-width networks in polynomial time.

New framework for understanding infinite-width neural networks.

The oloid is the convex hull of two circles with equal radius in perpendicular planes so that the center of each circle lies on the other circle. We calculate the mean width of the oloid in two ways, first via the integral of mean curvature, and then directly. Using this result, the surface area and the volume of the p…

Three configurations of two perpendicular disks in R^3 are examined, the first in which the disks share centers and the other two in which the disks touch at precisely one point. Volume, surface area and mean width calculations dominate the discussion. Integrated mean curvature also appears as an indirect way to comput…

This paper focuses on curves and surfaces of constant width, with some additional results about general ovals. We emphasize the use of Fourier series to derive properties, some of which are known. Amongst other results, we show that the perimeter of an oval is $π$ times its average width, and provide a bound for the ra…

Researchers create annular translators for mean curvature flow.

We prove the precise scaling, at finite depth and width, for the mean and variance of the neural tangent kernel (NTK) in a randomly initialized ReLU network. The standard deviation is exponential in the ratio of network depth to width. Thus, even in the limit of infinite overparameterization, the NTK is not determinist…

Study on fluctuations in neural network kernels and predictions, focusing on finite width effects.

The study examines spectral dynamics in deep neural networks, predicting how outliers evolve during training.

Elman-type RNNs converge to globally optimal solutions in the mean-field regime.

Study of manifolds with specific curvature properties using capillary surfaces.

We construct a compact, convex ancient solution of mean curvature flow in $\mathbb R^{n+1}$ with $O(1)\times O(n)$ symmetry that lies in a slab of width $π$. We provide detailed asymptotics for this solution and show that, up to rigid motions, it is the only compact, convex, $O(n)$-invariant ancient solution that lies …

Study shows critical width for rigidity of equatorial zones on spheres.

In this paper we adopted state-of-the-art machine learning algorithms, namely: random forest (RF) and least squares boosting, to model crash data and identify the optimum model to study the impact of narrow lanes on the safety of arterial roads. Using a ten-year crash dataset in four cities in Nebraska, two machine lea…

Surface area and mean width of a cylinder (the convex hull of two parallel disks) in R^3 are computed. It is more difficult to obtain analogous results for a cone (the convex hull of a disk D and a point p). Oblique formulas for mean width, as well as those for mean curvature, are new. Let L denote the unique diameter …

New framework connects two neural network theories, improving finite-width approximations.

Develops a new theory for neural systems stability and width effects.

Residual networks with depthwise hyperparameter scaling transfer optimal hyperparameters across width and depth.

We give a bound on the extinction time for a compact, strictly convex hypersurface in R^{n+1} evolving by a geometric flow where the velocity is given in terms of the curvature. This result generalizes a theorem of Colding and Minicozzi for mean curvature flow solutions to a wider class of flows studied by Ben Andrews.…

A new algorithm for learning shallow neural networks with infinite width.

Geodesic balls with non-negative Ricci curvature have a sharp lower bound on their first Dirichlet eigenvalue.

The betting CI outperforms classical methods in constructing confidence intervals for bounded means.

Lower bounds for surface area and volume of convex hypersurfaces.

The paper confirms a conjecture about convex bodies and their properties.

Study on random linear programs and their connection to mean widths of random polyhedrons.

This work studies fluctuation in multilayer neural networks using mean field theory.

Study on neural network dynamics in high dimensions with quadratic activation.

We develop a mathematically rigorous framework for multilayer neural networks in the mean field regime. As the network's widths increase, the network's learning trajectory is shown to be well captured by a meaningful and dynamically nonlinear limit (the \textit{mean field} limit), which is characterized by a system of …

Despite their prevalence in neural networks we still lack a thorough theoretical characterization of ReLU layers. This paper aims to further our understanding of ReLU layers by studying how the activation function ReLU interacts with the linear component of the layer and what role this interaction plays in the success …

Given a Riemannian metric on a homotopy $n$-sphere, sweep it out by a continuous one-parameter family of closed curves starting and ending at point curves. Pull the sweepout tight by, in a continuous way, pulling each curve as tight as possible yet preserving the sweepout. We show: Each curve in the tightened sweepout …

New optimizers control network width scaling, improving stability and transfer across different model sizes.

The study reveals a transition in neural network performance from infinite-width to variance-limited behavior as dataset size increases.

The study analyzes deep linear networks from random initialization, capturing dynamics and hyperparameter effects.

In order to investigate the origin of large price fluctuations, we analyze stock price changes of ten frequently traded NASDAQ stocks in the year 2002. Though the influence of the trading frequency on the aggregate return in a certain time interval is important, it cannot alone explain the heavy tailed distribution of …

Large learning rates work surprisingly well in standard parameterization, contrary to theory.

Study on curvature bounds for specific hypersurfaces in Anti-de Sitter space.

This paper addresses the so-called conformal capacities in $\mathbb R^n$, $n\ge 3$, through comparing three existing definitions (due to Betsakos, Colesanti-Cuoghi, Anderson-Vamananmurthy-Fuglede respectively) and studying their associated iso-capacitary inequalities with connection to half-diameter, mean-width, mean-c…

Proves properties of neural network basins of attraction and their expressiveness.

Neural network-based post-processing improves ensemble forecast sharpness

Adaptive kernels from neural networks improve model performance.

Derives a family of hyperparameter scaling strategies for neural networks.

RAmmStein optimizes liquidity management in AMMs by learning to rebalance efficiently.

Paper characterizes gradient descent dynamics for neural networks with finite width.

Global convergence proved for three-layer neural networks in mean field regime.

Bayesian linear networks reveal optimal depth and width trade-offs.

It has long been known that a single-layer fully-connected neural network with an i.i.d. prior over its parameters is equivalent to a Gaussian process (GP), in the limit of infinite network width. This correspondence enables exact Bayesian inference for infinite width neural networks on regression tasks by means of eva…