arXiv research

Data often comes in the form of an array or matrix. Matrix factorization techniques attempt to recover missing or corrupted entries by assuming that the matrix can be written as the product of two low-rank matrices. In other words, matrix factorization approximates the entries of the matrix by a simple, fixed function-…

A new method uses Gram matrix for efficient multivariate functional principal components.

Study of Gaussian distributions using entropic Gromov-Wasserstein and inner product Gromov-Wasserstein.

Inner product-based convolution has been the founding stone of convolutional neural networks (CNNs), enabling end-to-end learning of visual representation. By generalizing inner product with a bilinear matrix, we propose the neural similarity which serves as a learnable parametric similarity measure for CNNs. Neural si…

We propose a robust elastic net (REN) model for high-dimensional sparse regression and give its performance guarantees (both the statistical error bound and the optimization bound). A simple idea of trimming the inner product is applied to the elastic net model. Specifically, we robustify the covariance matrix by trimm…

Paper proposes a new method to optimize feature coordinates for better image classification.

Theoretical studies have proven that the Hilbert space has remarkable performance in many fields of applications. Frames in tensor product of Hilbert spaces were introduced to generalize the inner product to high-order tensors. However, these techniques require tensor decomposition which could lead to the loss of infor…

Study reveals an equivalence principle for the spectrum of random inner-product kernel matrices in polynomial scaling.

This article investigates the eigenspectrum of the inner product-type kernel matrix $\sqrt{p} \mathbf{K}=\{f( \mathbf{x}_i^{\sf T} \mathbf{x}_j/\sqrt{p})\}_{i,j=1}^n$ under a binary mixture model in the high dimensional regime where the number of data $n$ and their dimension $p$ are both large and comparable. Based on…

In this paper we develop several algebraic structures on the simplicial cochains of a triangulated manifold that are analogues of objects in differential geometry. We study a cochain product and prove several statements about its convergence to the wedge product on differential forms. Also, for cochains with an inner p…

This work further develops the properties of fractional differential forms. In particular, finite dimensional subspaces of fractional form spaces are considered. An inner product, Hodge dual, and covariant derivative are defined. Coordinate transformation rules for integral order forms are also computed. Matrix order f…

Autoencoders learn data representations (codes) in such a way that the input is reproduced at the output of the network. However, it is not always clear what kind of properties of the input data need to be captured by the codes. Kernel machines have experienced great success by operating via inner-products in a theoret…

This paper shows universality in spectrum behavior for random inner-product kernel matrices in polynomial regime.

How many samples are sufficient to guarantee that the eigenvectors and eigenvalues of the sample covariance matrix are close to those of the actual covariance matrix? For a wide family of distributions, including distributions with finite second moment and distributions supported in a centered Euclidean ball, we prove …

Study higher rank inner products and their tilings to describe tori degenerations.

We analyze kernel matrices in polynomial high-dimensional settings and explain double descent in KRR.

Layer normalization with activations prevents Gram matrix rank collapse at initialization.

Paper builds neural networks on matrix manifolds using gyrovector spaces.

Independent Component Analysis (ICA) is a popular model for blind signal separation. The ICA model assumes that a number of independent source signals are linearly mixed to form the observed signals. We propose a new algorithm, PEGI (for pseudo-Euclidean Gradient Iteration), for provable model recovery for ICA with Gau…

Researchers prove inner product recovery is impossible in latent space models.

Kernel method is a very powerful tool in machine learning. The trick of kernel has been effectively and extensively applied in many areas of machine learning, such as support vector machine (SVM) and kernel principal component analysis (kernel PCA). Kernel trick is to define a kernel function which relies on the inner-…

We propose a quantization based approach for fast approximate Maximum Inner Product Search (MIPS). Each database vector is quantized in multiple subspaces via a set of codebooks, learned directly by minimizing the inner product quantization error. Then, the inner product of a query to a database vector is approximated …

The parametrization theorem is derived in a flat nD pseudo-complex affine space. The pseudo-complex hyperbolic space accomodates n-number of uncompactified time-like extra dimensions with sugnature (s,r), where s and r are the numbers of minus and plus signs associated with the diagonalized metric matrix. The main resu…

Random projections (RP) are a popular tool for reducing dimensionality while preserving local geometry. In many applications the data set to be projected is given to us in advance, yet the current RP techniques do not make use of information about the data. In this paper, we provide a computationally light way to extra…

We point out that the Homfly polynomial (that is to say, Ocneanu's trace functional) contains two polynomial-valued inner products on the Hecke algebra representation of Artin's braid group. These bear a close connection to the Morton-Franks-Williams inequality. In these structures, the sets of positive, respectively n…

Study on kernel regression risk in high dimensions using Pinsker bound.

Kernel methods are successful approaches for different machine learning problems. This success is mainly rooted in using feature maps and kernel matrices. Some methods rely on the eigenvalues/eigenvectors of the kernel matrix, while for other methods the spectral information can be used to estimate the excess risk. An …

The paper develops concentration inequalities for structured random data, extending beyond independent terms.

This research simplifies Riemannian LBFGS for SPD manifolds.

Graphical notation simplifies tensor operations and decompositions.

Unified framework for hyperbolic embeddings from mixed data types.

This paper proposes a representational model for grid cells. In this model, the 2D self-position of the agent is represented by a high-dimensional vector, and the 2D self-motion or displacement of the agent is represented by a matrix that transforms the vector. Each component of the vector is a unit or a cell. The mode…

Convex learning for diverse invariances in semi-inner-product space.

We present the first provably sublinear time algorithm for approximate \emph{Maximum Inner Product Search} (MIPS). Our proposal is also the first hashing algorithm for searching with (un-normalized) inner product as the underlying similarity measure. Finding hashing schemes for MIPS was considered hard. We formally sho…

Kernel-based learning algorithms are widely used in machine learning for problems that make use of the similarity between object pairs. Such algorithms first embed all data points into an alternative space, where the inner product between object pairs specifies their distance in the embedding space. Applying kernel met…

Given a sparse rating matrix and an auxiliary matrix of users or items, how can we accurately predict missing ratings considering different data contexts of entities? Many previous studies proved that utilizing the additional information with rating data is helpful to improve the performance. However, existing methods …

Method selects interpretable circular coordinates from data.

We study the problem of learning similarity functions over very large corpora using neural network embedding models. These models are typically trained using SGD with sampling of random observed and unobserved pairs, with a number of samples that grows quadratically with the corpus size, making it expensive to scale to…

Being E a vector space with inner product and S the sphere of E, will be given a demonstration that every application of the sphere S itself it such that preserve inner product is the restriction of a linear isometry in E.

We propose a novel Riemannian manifold preconditioning approach for the tensor completion problem with rank constraint. A novel Riemannian metric or inner product is proposed that exploits the least-squares structure of the cost function and takes into account the structured symmetry that exists in Tucker decomposition…

This study approximates neural network features for modeling relations and attention mechanisms.

The Bergman kernels of holomorphic vector bundles are studied to extend the Fubini-Study map.

Classifies metrics on specific Lie groups.

We study estimation of (semi-)inner products between two nonparametric probability distributions, given IID samples from each distribution. These products include relatively well-studied classical $\mathcal{L}^2$ and Sobolev inner products, as well as those induced by translation-invariant reproducing kernels, for whic…

We show that every unimodular Lie algebra, of dimension at most 4, equipped with an inner product, possesses an orthonormal basis comprised of geodesic elements. On the other hand, we give an example of a solvable unimodular Lie algebra of dimension 5 that has no orthonormal geodesic basis, for any inner product.

We describe a construction of fibrewise inner products on the cotangent bundle of the smooth free loop space of a Riemannian manifold. Using this inner product, we construct an operator over the loop space of a string manifold which is directly analogous to the Dirac operator of a spin manifold.

Signature Isolation Forest removes constraints from FIF by using rough path theory's signature transform.

Vector embedding is a foundational building block of many deep learning models, especially in natural language processing. In this paper, we present a theoretical framework for understanding the effect of dimensionality on vector embeddings. We observe that the distributional hypothesis, a governing principle of statis…