arXiv research

Equivalent tests for SGD batch size selection found.

Paper proposes a new method to optimize feature coordinates for better image classification.

In this paper, we propose a stochastic optimization method that adaptively controls the sample size used in the computation of gradient approximations. Unlike other variance reduction techniques that either require additional storage or the regular computation of full gradients, the proposed method reduces variance by …

Study higher rank inner products and their tilings to describe tori degenerations.

Researchers prove inner product recovery is impossible in latent space models.

Inverted file and asymmetric distance computation (IVFADC) have been successfully applied to approximate nearest neighbor search and subsequently maximum inner product search. In such a framework, vector quantization is used for coarse partitioning while product quantization is used for quantizing residuals. In the ori…

Study of Gaussian distributions using entropic Gromov-Wasserstein and inner product Gromov-Wasserstein.

We propose a quantization based approach for fast approximate Maximum Inner Product Search (MIPS). Each database vector is quantized in multiple subspaces via a set of codebooks, learned directly by minimizing the inner product quantization error. Then, the inner product of a query to a database vector is approximated …

We study one extremal problem on the product of power of generalized inner radii of non-overlapping domains in $\mathbb{C}^{n}$.

We point out that the Homfly polynomial (that is to say, Ocneanu's trace functional) contains two polynomial-valued inner products on the Hecke algebra representation of Artin's braid group. These bear a close connection to the Morton-Franks-Williams inequality. In these structures, the sets of positive, respectively n…

Study on kernel regression risk in high dimensions using Pinsker bound.

Convex learning for diverse invariances in semi-inner-product space.

We present the first provably sublinear time algorithm for approximate \emph{Maximum Inner Product Search} (MIPS). Our proposal is also the first hashing algorithm for searching with (un-normalized) inner product as the underlying similarity measure. Finding hashing schemes for MIPS was considered hard. We formally sho…

We consider stochastic zero-order optimization problems, which arise in settings from simulation optimization to reinforcement learning. We propose an adaptive sampling quasi-Newton method where we estimate the gradients of a stochastic function using finite differences within a common random number framework. We emplo…

We study estimation of (semi-)inner products between two nonparametric probability distributions, given IID samples from each distribution. These products include relatively well-studied classical $\mathcal{L}^2$ and Sobolev inner products, as well as those induced by translation-invariant reproducing kernels, for whic…

Being E a vector space with inner product and S the sphere of E, will be given a demonstration that every application of the sphere S itself it such that preserve inner product is the restriction of a linear isometry in E.

This study approximates neural network features for modeling relations and attention mechanisms.

The Bergman kernels of holomorphic vector bundles are studied to extend the Fubini-Study map.

Classifies metrics on specific Lie groups.

We propose a robust elastic net (REN) model for high-dimensional sparse regression and give its performance guarantees (both the statistical error bound and the optimization bound). A simple idea of trimming the inner product is applied to the elastic net model. Specifically, we robustify the covariance matrix by trimm…

We show that every unimodular Lie algebra, of dimension at most 4, equipped with an inner product, possesses an orthonormal basis comprised of geodesic elements. On the other hand, we give an example of a solvable unimodular Lie algebra of dimension 5 that has no orthonormal geodesic basis, for any inner product.

We describe a construction of fibrewise inner products on the cotangent bundle of the smooth free loop space of a Riemannian manifold. Using this inner product, we construct an operator over the loop space of a string manifold which is directly analogous to the Dirac operator of a spin manifold.

Signature Isolation Forest removes constraints from FIF by using rough path theory's signature transform.

We propose $\textit{weighted inner product similarity}$ (WIPS) for neural network-based graph embedding. In addition to the parameters of neural networks, we optimize the weights of the inner product by allowing positive and negative values. Despite its simplicity, WIPS can approximate arbitrary general similarities in…

We give a condition for a function to produce a Möbius invariant weighted inner product on the tangent space of the space of knots, and show that some kind of Möbius invariant knot energies can produce Möbius invariant and parametrization invariant weighted inner products. They would give a natural way to study the evo…

The paper classifies hypersurfaces in Riemannian manifolds with constant inner product and torse-forming axes.

We formulate a quantization commutes with reduction principle in the setting where the Lie group $G$, the symplectic manifold it acts on, and the orbit space of the action may all be noncompact. It is assumed that the action is proper, and the zero set of a deformation vector field, associated to the momentum map and a…

In this paper we develop several algebraic structures on the simplicial cochains of a triangulated manifold that are analogues of objects in differential geometry. We study a cochain product and prove several statements about its convergence to the wedge product on differential forms. Also, for cochains with an inner p…

H-type Lie algebras were introduced by Kaplan as a class of real Lie algebras generalizing the familiar Heisenberg Lie algebra $\mathfrak{h}^3$. The H-type property depends on a choice of inner product on the Lie algebra $\mathfrak{g}$. Among the H-type Lie algebras are the complex Heisenberg Lie algebras $\mathfrak{h}…

Minwise hashing (Minhash) is a widely popular indexing scheme in practice. Minhash is designed for estimating set resemblance and is known to be suboptimal in many applications where the desired measure is set overlap (i.e., inner product between binary vectors) or set containment. Minhash has inherent bias towards sma…

In this article we define the twisted product of groups as the generalization of the semidirect product of groups. We will find the necessary and sufficient condition in order that the twisted product of groups to be a group. In particular, for two copies of the same group, the twisted product of group by itself throug…

The paper challenges the belief that more inner iterations at test time improve performance in implicit deep learning.

GPU-accelerates multiuser detection for 5G URLLC systems.

We propose shifted inner-product similarity (SIPS), which is a novel yet very simple extension of the ordinary inner-product similarity (IPS) for neural-network based graph embedding (GE). In contrast to IPS, that is limited to approximating positive-definite (PD) similarities, SIPS goes beyond the limitation by introd…

A metric Lie algebra g is a Lie algebra equipped with an inner product. A subalgebra h of a metric Lie algebra g is said to be totally geodesic if the Lie subgroup corresponding to h is a totally geodesic submanifold relative to the left-invariant Riemannian metric defined by the inner product, on the simply connected …

The paper explores linear representations in language models using counterfactuals.

In Riemann geometry, the relations among two transversal submanifolds and global manifold are discussed. By replacing the normal vector of a submanifold with the tangent vector of another submanifold, the metric tensors, Christoffel symbols and curvature tensors of the three manifolds are linked together. When the inne…

In this paper, we determine the automorphism group of the $p$-cones ($p\neq 2$) in dimension greater than two. In particular, we show that the automorphism group of those $p$-cones are the positive scalar multiples of the generalized permutation matrices that fix the main axis of the cone. Next, we take a look at a pro…

We prove that the tangent bundle of an inner symmetric space $M$ of compact type is weakly complex if and only if $M$ is a Riemannian product $M_1\times...\times M_k$, each $M_i$ being an even-dimensional round sphere or Hermitian symmetric.

This paper addresses the nearest neighbor search problem under inner product similarity and introduces a compact code-based approach. The idea is to approximate a vector using the composition of several elements selected from a source dictionary and to represent this vector by a short code composed of the indices of th…

Recently, new defense techniques have been developed to tolerate Byzantine failures for distributed machine learning. The Byzantine model captures workers that behave arbitrarily, including malicious and compromised workers. In this paper, we break two prevailing Byzantine-tolerant techniques. Specifically we show robu…

In Euclidean geometry, all metric notions (arc length for curves, the first fundamental form for surfaces, etc.) are derived from the Euclidean inner product on tangent vectors, and this inner product is preserved by the full symmetry group of Euclidean space (translations, rotations, and reflections). In equiaffine ge…

Gradient matching method improves domain generalization across various datasets.

Inner product-based convolution has been the founding stone of convolutional neural networks (CNNs), enabling end-to-end learning of visual representation. By generalizing inner product with a bilinear matrix, we propose the neural similarity which serves as a learnable parametric similarity measure for CNNs. Neural si…

Optimizes shapes of curves using Möbius energy gradients.

The paper studies geometric properties of Grassman manifolds within Euclidean spaces.

Geometrically, tensors of fixed rank form a minimal submanifold.

Heegaard Floer homology connects to polynomial representations of Hecke algebras.