arXiv research

In this note, we derive concentration inequalities for random vectors with subGaussian norm (a generalization of both subGaussian random vectors and norm bounded random vectors), which are tight up to logarithmic factors.

The paper studies how norms of random vectors are preserved by random projections.

Support Vector Machine (SVM) is an efficient classification approach, which finds a hyperplane to separate data from different classes. This hyperplane is determined by support vectors. In existing SVM formulations, the objective function uses L2 norm or L1 norm on slack variables. The number of support vectors is a me…

The paper introduces a new method for tail bounds of random vectors and matrices.

This work describes compactifications of metric spaces and vector spaces using asymmetric norms.

In compressed sensing, in order to recover a sparse or nearly sparse vector from possibly noisy measurements, the most popular approach is $\ell_1$-norm minimization. Upper bounds for the $\ell_2$- norm of the error between the true and estimated vectors are given in [1] and reviewed in [2], while bounds for the $\ell_…

This study reveals the critical role of scale vectors in large language models, improving optimization and expressivity.

The study characterizes quasi Yamabe solitons with potential vector fields.

Feature hashing and other random projection schemes are commonly used to reduce the dimensionality of feature vectors. The goal is to efficiently project a high-dimensional feature vector living in $\mathbb{R}^n$ into a much lower-dimensional space $\mathbb{R}^m$, while approximately preserving Euclidean norm. These sc…

The paper examines convergence of distances in Lipschitz structures on manifolds.

FedNNNN improves FL by adjusting model update vector norms.

Timelike geometry of spherical simplices is shown to be isometric to vector spaces.

The paper tackles multi-armed bandits with vector losses, focusing on minimizing the $\ell^\infty$-norm of relative losses.

We propose $\ell_1$ norm regularized quadratic surface support vector machine models for binary classification in supervised learning. We establish their desired theoretical properties, including the existence and uniqueness of the optimal solution, reduction to the standard SVMs over (almost) linearly separable data s…

Recently, $l_{2,1}$ matrix norm has been widely applied to many areas such as computer vision, pattern recognition, biological study and etc. As an extension of $l_1$ vector norm, the mixed $l_{2,1}$ matrix norm is often used to find jointly sparse solutions. Moreover, an efficient iterative algorithm has been designed…

Optimal rates for vector-valued regression on various norms.

The study connects norms and filtrations on section rings of projective manifolds.

We prove a spanning result for vector-valued Poincaré series on a bounded symmetric domain. We associate a sequence of holomorphic automorphic forms to a submanifold of the domain. When the domain is the unit ball in ${\Bbb{C}}^n$, we provide estimates for the norms of these automorphic forms and we find asymptotics of…

Paper presents a novel method to assess boundedness and stability of nonlinear systems with variable delays.

RVFL networks can efficiently approximate Lipschitz functions in L∞ norm.

Max-convolution is an important problem closely resembling standard convolution; as such, max-convolution occurs frequently across many fields. Here we extend the method with fastest known worst-case runtime, which can be applied to nonnegative vectors by numerically approximating the Chebyshev norm $\| \cdot \|_\infty…

Optimal DP mechanisms for vector queries are found to be staircase distributions.

We combine Gromov's amenable localization technique with the Poincaré duality to study the traversally generic vector flows on smooth compact manifolds $X$ with boundary. Such flows generate well-understood stratifications of $X$ by the trajectories that are tangent to the boundary in a particular canonical fashion. Sp…

The $k$-support norm is a regularizer which has been successfully applied to sparse vector prediction problems. We show that it belongs to a general class of norms which can be formulated as a parameterized infimum over quadratics. We further extend the $k$-support norm to matrices, and we observe that it is a special …

Single ReLU neuron's gradient dynamics reveal support vectors as key to generalization.

New ONMF model minimizes KL divergence for better sparse data modeling.

This paper studies the problem of estimating the covariance of a collection of vectors using only highly compressed measurements of each vector. An estimator based on back-projections of these compressive samples is proposed and analyzed. A distribution-free analysis shows that by observing just a single linear measure…

Derives integral formula for Hodge and Teichmüller norms.

We theoretically and experimentally investigate tensor-based regression and classification. Our focus is regularization with various tensor norms, including the overlapped trace norm, the latent trace norm, and the scaled latent trace norm. We first give dual optimization methods using the alternating direction method …

Improved online PCA algorithm learns from evolving norm of parameter vector.

We study a regularizer which is defined as a parameterized infimum of quadratics, and which we call the box-norm. We show that the k-support norm, a regularizer proposed by [Argyriou et al, 2012] for sparse vector prediction problems, belongs to this family, and the box-norm can be generated as a perturbation of the fo…

Using the $\ell_1$-norm to regularize the estimation of the parameter vector of a linear model leads to an unstable estimator when covariates are highly correlated. In this paper, we introduce a new penalty function which takes into account the correlation of the design matrix to stabilize the estimation. This norm, ca…

The stochastic linear bandit problem proceeds in rounds where at each round the algorithm selects a vector from a decision set after which it receives a noisy linear loss parameterized by an unknown vector. The goal in such a problem is to minimize the (pseudo) regret which is the difference between the total expected …

Fast algorithm for rescaling vectors with clipping, improving training efficiency.

We derive a bound on the $L^{\infty}$-norm of the covariant derivative of Laplace eigensections on general Riemannian vector bundles depending on the diameter, the dimension, the Ricci curvature of the underlying manifold, and the curvature of the Riemannian vector bundle. Our result implies that eigensections with sma…

Article provides polytopes as dual unit balls of Thurston norms on 3-manifolds.

We consider the problem of distributed mean estimation (DME), in which $n$ machines are each given a local $d$-dimensional vector $x_v \in \mathbb{R}^d$, and must cooperate to estimate the mean of their inputs $μ= \frac 1n\sum_{v = 1}^n x_v$, while minimizing total communication cost. DME is a fundamental construct in …

Consider the recovery of an unknown signal ${x}$ from quantized linear measurements. In the one-bit compressive sensing setting, one typically assumes that ${x}$ is sparse, and that the measurements are of the form $\operatorname{sign}(\langle {a}_i, {x} \rangle) \in \{\pm1\}$. Since such measurements give no informati…

Proximal operators are of particular interest in optimization problems dealing with non-smooth objectives because in many practical cases they lead to optimization algorithms whose updates can be computed in closed form or very efficiently. A well-known example is the proximal operator of the vector $\ell_1$ norm, whic…

Classical scalar-response regression methods treat covariates as a vector and estimate a corresponding vector of regression coefficients. In medical applications, however, regressors are often in a form of multi-dimensional arrays. For example, one may be interested in using MRI imaging to identify which brain regions …

Improved greedy 2-coordinate updates for optimization problems with constraints.

Gradient flow on softmax attention minimizes nuclear norm of weight matrices.

The paper generalizes product inequalities for random vectors and their applications.

Support vector machines (SVMs) are an important tool in modern data analysis. Traditionally, support vector machines have been fitted via quadratic programming, either using purpose-built or off-the-shelf algorithms. We present an alternative approach to SVM fitting via the majorization--minimization (MM) paradigm. Alg…

Study analyzes perturbations in singular subspaces under random noise.

It is shown that the Hilbert metric on the interior of a convex polytope is bilipschitz to a normed vector space of the same dimension.

We propose in this contribution a method for l one regularization in prototype based relevance learning vector quantization (LVQ) for sparse relevance profiles. Sparse relevance profiles in hyperspectral data analysis fade down those spectral bands which are not necessary for classification. In particular, we consider …

In this paper, we propose an unifying view of several recently proposed structured sparsity-inducing norms. We consider the situation of a model simultaneously (a) penalized by a set- function de ned on the support of the unknown parameter vector which represents prior knowledge on supports, and (b) regularized in Lp-n…