arXiv research

A key element of understanding the efficacy of overparameterized neural networks is characterizing how they represent functions as the number of weights in the network approaches infinity. In this paper, we characterize the norm required to realize a function $f:\mathbb{R}^d\rightarrow\mathbb{R}$ as a single hidden-lay…

New method certifies neural network function space norms from point evaluations.

Learning rates for least-squares regression are typically expressed in terms of $L_2$-norms. In this paper we extend these rates to norms stronger than the $L_2$-norm without requiring the regression function to be contained in the hypothesis space. In the special case of Sobolev reproducing kernel Hilbert spaces used …

Complexity measures for neural nets with general activations using path-based norms.

Targeting at sparse learning, we construct Banach spaces B of functions on an input space X with the properties that (1) B possesses an l1 norm in the sense that it is isometrically isomorphic to the Banach space of integrable functions on X with respect to the counting measure; (2) point evaluations are continuous lin…

We consider the question of what functions can be captured by ReLU networks with an unbounded number of units (infinite width), but where the overall network Euclidean norm (sum of squares of all weights in the system, except for an unregularized bias term for each unit) is bounded; or equivalently what is the minimal …

Characterizes inductive bias in multi-channel linear CNNs with bounded weight norm.

Unified theory of deep neural networks with diverse activations.

This paper studies neural networks with bounded norms to avoid the curse of dimensionality.

We consider a class of operator-induced norms, acting as finite-dimensional surrogates to the L2 norm, and study their approximation properties over Hilbert subspaces of L2 . The class includes, as a special case, the usual empirical norm encountered, for example, in the context of nonparametric regression in reproduci…

We consider immersions admitting uniform graph representations over the affine tangent space over a ball of fixed radius r>0. We show that for sufficiently small C^0-norm of the graph functions, each graph function is smooth with small C^1-norm.

We study contractivity properties of gradient flows for functions on normed spaces or, more generally, on Finsler manifolds. Contractivity of the flows turns out to be equivalent to a new notion of convexity for the functions. This is different from the usual convexity along geodesics in non-Riemannian Finsler manifold…

We do further investigation in a certain cosine function defined for smooth Minkowski spaces. We prove that such function is symmetric if and only if the referred space is Euclidean, and also that it can be given in terms of the Gateaux derivative of the norm. As an application we use it to study the ratio between the …

Study extends neural network approximation to time-varying PDEs using Fourier-Lebesgue spaces.

New Banach spaces for ReLU networks enable better function approximation and gradient dynamics analysis.

The paper uses Banach spaces to analyze neural networks.

New method approximates complex kernel norms with random features, making learning tractable.

Characterizes isometries between non-reversible Finsler manifolds.

One of the key issues in the analysis of machine learning models is to identify the appropriate function space and norm for the model. This is the set of functions endowed with a quantity which can control the approximation and estimation errors by a particular machine learning model. In this paper, we address this iss…

The paper characterizes functions of shallow ReLU NN denoisers under minimal norm constraints.

Optimal rates for vector-valued regression on various norms.

A compact Riemannian manifold may be immersed into Euclidean space by using high frequency Laplace eigenfunctions. We study the geometry of the manifold viewed as a metric space endowed with the distance function from the ambient Euclidean space. As an application we give a new proof of a result of Burq-Lebeau and othe…

We introduce here a natural functional associated to any $b \in QH_* (M, ω)$: \emph{spectral length functional}, on the space of "generalized paths" in $\text {Ham}(M, ω)$, closely related to both the Hofer length functional and spectral invariants and establish some of its properties. This functional is smooth on its…

Let $R$ be a compact surface and let $Γ$ be a Jordan curve which separates $R$ into two connected components $Σ_1$ and $Σ_2$. A harmonic function $h_1$ on $Σ_1$ of bounded Dirichlet norm has boundary values $H$ in a certain conformally invariant non-tangential sense on $Γ$. We show that if $Γ$ is a quasicircle, then th…

The L 1-Sobolev inequality states that the L n/(n--1)-norm of a compactly supported function on Euclidean n-space is controlled by the L 1-norm of its gradient. The generalization to differential forms (due to Lanzani & Stein and Bourgain & Brezis) is recent, and states that a the L n/(n--1)-norm of a compactly support…

For supervised and unsupervised learning, positive definite kernels allow to use large and potentially infinite dimensional feature spaces with a computational cost that only depends on the number of observations. This is usually done through the penalization of predictor functions by Euclidean or Hilbertian norms. In …

This paper studies optimal approximation factors in misspecified off-policy RL, identifying key factors under various settings.

Study on minimal hypersurfaces in a special normed space.

We propose a systematic construction of native Banach spaces for general spline-admissible operators ${\rm L}$. In short, the native space for ${\rm L}$ and the (dual) norm $\|\cdot\|_{\mathcal{X}'}$ is the largest space of functions $f: \mathbb{R}^d \to \mathbb{R}$ such that $\|{\rm L} f\|_{\mathcal{X}'}<\infty$, subj…

The paper improves confidence regions for band-limited functions using tighter norm bounds and majority voting.

Revisits shallow neural networks using Lipschitz norms and measures.

We investigate stability and local minimizing properties of the Riemannian functional defined by the L^p norm of the curvature tensor on the space of Riemannian metrics on a closed manifold. Riemannian metrics with constant curvature and products of such metrics are critical points of this functional. We prove that the…

Motivated by some applications in signal processing and machine learning, we consider two convex optimization problems where, given a cone $K$, a norm $\|\cdot\|$ and a smooth convex function $f$, we want either 1) to minimize the norm over the intersection of the cone and a level set of $f$, or 2) to minimize over the…

The paper accelerates ISTA and FISTA algorithms for composite optimization problems.

Deep networks with path norm regularization can approximate analytic functions.

The study explores special surfaces in a normed space.

New neural architectures with multivariate nonlinearities are optimal in function space.

Study on minimal surfaces in a 3D space with 2m-norm.

New framework for private convex optimization in arbitrary norms.

New analysis shows a gap between Gaussian RKHS and neural networks on unbounded domains.

Kähler information manifolds for signal filters in weighted Hardy spaces are explored.

Unified formula for higher traces of linear maps on finite-dimensional normed spaces.

Classifies surfaces with special curvature properties.

The real homology of a compact Riemannian manifold $M$ is naturally endowed with the stable norm. The stable norm on $H_1(M,\mathbb{R})$ arises from the Riemannian length functional by homogenization. It is difficult and interesting to decide which norms on the finite-dimensional vector space $H_1(M,\mathbb{R})$ are st…

Infinite diameter proved for contractible loops space.

We show that monochromatic Finsler metrics, i.e., Finsler metrics such that each two tangent spaces are isomorphic as normed spaces, are generalized Berwald metrics, i.e., there exists an affine connection, possibly with torsion, that preserves the Finsler function

New framework explains deep neural networks using variational spline theory.

Develops a new asymptotic efficiency theory for non-Euclidean parameter spaces.