A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New method classifies manifold-valued data using Riemannian geometry.
Improved vector quantization using Gaussian mixtures for better codebook utilization.
A new framework enhances generative modeling by learning local flows over complex manifolds.
In this paper we propose a simple yet powerful method for learning representations in supervised learning scenarios where each original input datapoint is described by a set of vectors and their associated outputs may be given by soft labels indicating, for example, class probabilities. We represent an input datapoint …
Based on the notion of information bottleneck (IB), we formulate a quantization problem called "IB quantization". We show that IB quantization is equivalent to learning based on the IB principle. Under this equivalence, the standard neural network models can be viewed as scalar (single sample) IB quantizers. It is know…
PRISM-VQ combines financial priors with vector quantization for better stock prediction.
New bound on partition function proves Kähler-Einstein stability.
Study shows simple vector quantization measures correlate with deep learning generalization.
Develops mixed quantization for graph vector bundles.
A new method for multiclass calibration using vector quantization.
We propose a quantization based approach for fast approximate Maximum Inner Product Search (MIPS). Each database vector is quantized in multiple subspaces via a set of codebooks, learned directly by minimizing the inner product quantization error. Then, the inner product of a query to a database vector is approximated …
Vectors of data are at the heart of machine learning and data mining. Recently, vector quantization methods have shown great promise in reducing both the time and space costs of operating on vectors. We introduce a vector quantization algorithm that can compress vectors over 12x faster than existing techniques while al…
Quantizes symplectic fibrations to analyze vector bundles and metrics.
In this work we present a review of the state of the art of Learning Vector Quantization (LVQ) classifiers. A taxonomy is proposed which integrates the most relevant LVQ approaches to date. The main concepts associated with modern LVQ approaches are defined. A comparison is made among eleven LVQ classifiers using one r…
This paper provides theoretical foundations for using quantized actions in behavior cloning.
UVeQFed tackles FL model compression over limited channels.
We present Rotated Adaptive Tetra-iterated Quantizer (RATQ), a fixed-length quantizer for gradients in first order stochastic optimization. RATQ is easy to implement and involves only a Hadamard transform computation and adaptive uniform quantization with appropriately chosen dynamic ranges. For noisy gradients with al…
A new method learns discrete representations for images and videos, improving upon previous models.
A simple and computationally efficient scheme for tree-structured vector quantization is presented. Unlike previous methods, its quantization error depends only on the intrinsic dimension of the data distribution, rather than the apparent dimension of the space in which the data happen to lie.
Quantizes Kähler manifolds using sheaves and differential operators.
Inverted file and asymmetric distance computation (IVFADC) have been successfully applied to approximate nearest neighbor search and subsequently maximum inner product search. In such a framework, vector quantization is used for coarse partitioning while product quantization is used for quantizing residuals. In the ori…
We consider the problem of learning a neural network classifier. Under the information bottleneck (IB) principle, we associate with this classification problem a representation learning problem, which we call "IB learning". We show that IB learning is, in fact, equivalent to a special class of the quantization problem.…
An appropriate choice of the activation function (like ReLU, sigmoid or swish) plays an important role in the performance of (deep) multilayer perceptrons (MLP) for classification and regression learning. Prototype-based classification learning methods like (generalized) learning vector quantization (GLVQ) are powerful…
Continuous representation of words is a standard component in deep learning-based NLP models. However, representing a large vocabulary requires significant memory, which can cause problems, particularly on resource-constrained platforms. Therefore, in this paper we propose an isotropic iterative quantization (IIQ) appr…
Note on subgaussian bounds for sign-quantized linear maps.
Autoencoders and their variations provide unsupervised models for learning low-dimensional representations for downstream tasks. Without proper regularization, autoencoder models are susceptible to the overfitting problem and the so-called posterior collapse phenomenon. In this paper, we introduce a quantization-based …
Quantization based techniques are the current state-of-the-art for scaling maximum inner product search to massive databases. Traditional approaches to quantization aim to minimize the reconstruction error of the database points. Based on the observation that for a given query, the database points that have the largest…
We propose in this contribution a method for l one regularization in prototype based relevance learning vector quantization (LVQ) for sparse relevance profiles. Sparse relevance profiles in hyperspectral data analysis fade down those spectral bands which are not necessary for classification. In particular, we consider …
A new method for robust product Markovian quantization overcomes numerical instabilities.
The paper examines obstacles to extending deformation quantization of vector bundles.
A new model trains prior and encoder/decoder networks simultaneously for efficient generation.
We consider the post-training quantization problem, which discretizes the weights of pre-trained deep neural networks without re-training the model. We propose multipoint quantization, a quantization method that approximates a full-precision weight vector using a linear combination of multiple vectors of low-bit number…
It is of fundamental importance to find algorithms obtaining optimal performance for learning of statistical models in distributed and communication limited systems. Aiming at characterizing the optimal strategies, we consider learning of Gaussian Processes (GPs) in distributed systems as a pivotal example. We first ad…
Paper proves existence of a universal codebook for low-precision quantization.
A geometric quantization of a Kähler manifold, viewed as a symplectic manifold, depends on the complex structure compatible with the symplectic form. The quantizations form a vector bundle over the space of such complex structures. Having a canonical quantization would amount to finding a natural (projectively) flat co…
Similarity search retrieves the nearest neighbors of a query vector from a dataset of high-dimensional vectors. As the size of the dataset grows, the cost of performing the distance computations needed to implement a query can become prohibitive. A method often used to reduce this computational cost is quantization of …
VQ-GNN scales GNNs to large graphs using vector quantization.
The nearest neighbor method together with the dynamic time warping (DTW) distance is one of the most popular approaches in time series classification. This method suffers from high storage and computation requirements for large training sets. As a solution to both drawbacks, this article extends learning vector quantiz…
The paper quantizes concatenated noisy vectors to a common cluster center, improving performance over naive methods.
In this work, we present a family of vector quantization schemes \emph{vqSGD} (Vector-Quantized Stochastic Gradient Descent) that provide an asymptotic reduction in the communication cost with convergence guarantees in first-order distributed optimization. In the process we derive the following fundamental information …
The paper classifies quantizable functions and explores symmetry in quantization methods.
Discretizing multi-dimensional data distributions is a fundamental step of modern indexing methods. State-of-the-art techniques learn parameters of quantizers on training data for optimal performance, thus adapting quantizers to the data. In this work, we propose to reverse this paradigm and adapt the data to the quant…
Let be a holomorphic Hermitian vector bundle over a polarized manifold. We provide a canonical quantization of the Laplacian operator acting on sections of the bundle of Hermitian endomorphisms of . If is simple we obtain an approximation of the eigenvalues and eigenspaces of the Laplacian.
By computing certain cohomology of Vect(M) of smooth vector fields we prove that on 1-dimensional manifolds M there is no quantization map intertwining the action of non-projective embeddings of the Lie algebra sl(2) into the Lie algebra Vect(M). Contrariwise, for projective embeddings sl(2)-equivariant quantization ex…
We study two quantization schemes for compact symplectic manifolds with almost complex structures. The first of these is the Spin-c quantization. We prove the analog of Kodaira vanishing for the Spin-c Dirac operator, which shows that the index space of this operator provides an honest (not virtual) vector space semicl…
Data parallelism can boost the training speed of convolutional neural networks (CNN), but could suffer from significant communication costs caused by gradient aggregation. To alleviate this problem, several scalar quantization techniques have been developed to compress the gradients. But these techniques could perform …
Transfer learning improves sparse, interpretable probabilistic classification.
Extends geometric quantization to singular spaces.