arXiv research

Uniform convergence of interpolators proven for Gaussian data.

General $(α, β)$ norms are an important class of Minkowski norms which contains the original $(α, β)$ norms. In this note, by studying the behavior of the Darboux curves of the indicatrix, we give a characterization of 3-dimensional general $(α, β)$ norms. By studying the isoperimetric properties of the indicatrix, as …

Paper introduces new risk norms based on ES with flexible distortion functions.

We introduce twisted Alexander norms of a compact connected orientable 3-manifold with first Betti number bigger than one generalizing norms of McMullen and Turaev. We show that twisted Alexander norms give lower bounds on the Thurston norm of a 3-manifold. Using these we completely determine the Thurston norm of many …

A toolkit for path-norms enhances neural network generalization bounds.

We introduce a new family of matrix norms, the "local max" norms, generalizing existing methods such as the max norm, the trace norm (nuclear norm), and the weighted or smoothed weighted trace norms, which have been extensively used in the literature as regularizers for matrix reconstruction problems. We show that this…

The paper defines minimal norm tensors for curvature and divergence tensors, explaining Weyl and Cotten tensors.

The study explores special surfaces in a normed space.

The paper explores why a specific type of predictor works well in noisy data.

Unified algorithm for any $p$-norm experimental design problems.

We study a regularizer which is defined as a parameterized infimum of quadratics, and which we call the box-norm. We show that the k-support norm, a regularizer proposed by [Argyriou et al, 2012] for sparse vector prediction problems, belongs to this family, and the box-norm can be generated as a perturbation of the fo…

The $k$-support norm is a regularizer which has been successfully applied to sparse vector prediction problems. We show that it belongs to a general class of norms which can be formulated as a parameterized infimum over quadratics. We further extend the $k$-support norm to matrices, and we observe that it is a special …

Kernel interpolation is inconsistent for norms with smoothness above a constant.

Complexity measures for neural nets with general activations using path-based norms.

Recently, path norm was proposed as a new capacity measure for neural networks with Rectified Linear Unit (ReLU) activation function, which takes the rescaling-invariant property of ReLU into account. It has been shown that the generalization error bound in terms of the path norm explains the empirical generalization b…

Near-interpolating models grow norms quickly, affecting generalization.

SAM improves generalization in overparameterized models, but its behavior in tensorized models is less understood.

Improved LDA with capped l_{2,1}-norm reduces outlier sensitivity.

Exact spectral norm regularization improves neural network generalization.

The spectral $k$-support norm enjoys good estimation properties in low rank matrix learning problems, empirically outperforming the trace norm. Its unit ball is the convex hull of rank $k$ matrices with unit Frobenius norm. In this paper we generalize the norm to the spectral $(k,p)$-support norm, whose additional para…

For sutured 3-manifolds M, there is a sutured Thurston norm due to Scharlemann. We show how depth one foliations of M and corresponding fibrations and the usual Thurston norm on the double of M are useful tools for computing this norm. In many examples, the faces of the unit ball of the sutured norm are related to cone…

New insights into network generalization show learning rate affects both norm and sharpness.

CNN layers with large norms are still robust to adversarial attacks.

The study analyzes robustness of estimators in linear models with adversarial errors.

We provide recovery guarantees for compressible signals that have been corrupted with noise and extend the framework introduced in \cite{bafna2018thwarting} to defend neural networks against $\ell_0$-norm, $\ell_2$-norm, and $\ell_{\infty}$-norm attacks. Our results are general as they can be applied to most unitary tr…

We propose a data aggregation-based algorithm with monotonic convergence to a global optimum for a generalized version of the L1-norm error fitting model with an assumption of the fitting function. The proposed algorithm generalizes the recent algorithm in the literature, aggregate and iterative disaggregate (AID), whi…

New bound relaxes uniform gradient norm assumptions for PAC-Bayesian bounds.

In this note, we derive concentration inequalities for random vectors with subGaussian norm (a generalization of both subGaussian random vectors and norm bounded random vectors), which are tight up to logarithmic factors.

Deep neural networks with adversarial training achieve sup-norm convergence for nonparametric regression.

The paper proves inequalities linking geometric norms and Thurston norms in hyperbolic 3-manifolds.

This paper tackles robustness of ensemble stumps and trees under general ℓ_p norm perturbations.

Loose bounds found for least-norm interpolant in over-parameterized settings.

New framework for private convex optimization in arbitrary norms.

We propose a Generalized Dantzig Selector (GDS) for linear models, in which any norm encoding the parameter structure can be leveraged for estimation. We investigate both computational and statistical aspects of the GDS. Based on conjugate proximal operator, a flexible inexact ADMM framework is designed for solving GDS…

The trace norm is widely used in multi-task learning as it can discover low-rank structures among tasks in terms of model parameters. Nowadays, with the emerging of big datasets and the popularity of deep learning techniques, tensor trace norms have been used for deep multi-task models. However, existing tensor trace n…

We investigate the capacity, convexity and characterization of a general family of norm-constrained feed-forward networks.

The L 1-Sobolev inequality states that the L n/(n--1)-norm of a compactly supported function on Euclidean n-space is controlled by the L 1-norm of its gradient. The generalization to differential forms (due to Lanzani & Stein and Bourgain & Brezis) is recent, and states that a the L n/(n--1)-norm of a compactly support…

New insights into optimization and generalization for linear models.

The Thurston norm of a 3-manifold measures the complexity of surfaces representing two-dimensional homology classes. We study the possible unit balls of Thurston norms of 3-manifolds $M$ with $b_1(M) = 2$, and whose fundamental groups admit presentations with two generators and one relator. We show that even among this…

In a previous paper, the second author defined integer-valued functions delta_n on the first cohomology of a 3-manifold, generalizing McMullen's Alexander norm. It was shown that these functions give lower bounds on the Thurston norm. In this paper, we reformulate these invariants in terms of Reidemeister torsion over …

We study the multiscale simplicial flat norm (MSFN) problem, which computes flat norm at various scales of sets defined as oriented subcomplexes of finite simplicial complexes in arbitrary dimensions. We show that the multiscale simplicial flat norm is NP-complete when homology is defined over integers. We cast the mul…

Study bounds on harmonic forms in hyperbolic 3-manifolds using Thurston norm and minimal surfaces.

We consider in this paper the problem of noisy 1-bit matrix completion under a general non-uniform sampling distribution using the max-norm as a convex relaxation for the rank. A max-norm constrained maximum likelihood estimate is introduced and studied. The rate of convergence for the estimate is obtained. Information…

We study the generalization properties of minimum-norm solutions for three over-parametrized machine learning models including the random feature model, the two-layer neural network model and the residual network model. We proved that for all three models, the generalization error for the minimum-norm solution is compa…

Recently, $l_{2,1}$ matrix norm has been widely applied to many areas such as computer vision, pattern recognition, biological study and etc. As an extension of $l_1$ vector norm, the mixed $l_{2,1}$ matrix norm is often used to find jointly sparse solutions. Moreover, an efficient iterative algorithm has been designed…

General lower bounds on neural network approximation in L^p norm.

Improved sample complexity for ReLU networks with norm constraints.

The Schatten quasi-norm was introduced to bridge the gap between the trace norm and rank function. However, existing algorithms are too slow or even impractical for large-scale problems. Motivated by the equivalence relation between the trace norm and its bilinear spectral penalty, we define two tractable Schatten norm…