A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Introduces sheaf quantization, a topological approach to geometric quantization.
HMQ improves quantization for edge devices with mixed precision.
Unified finetuning of all quantization degrees of freedom achieves state-of-the-art 4-bit quantization.
The paper defines and analyzes coherent and squeezed states on manifolds and their quantization.
LSQ+ improves quantization of neural nets with Swish activations, achieving state-of-the-art results.
Quantization based techniques are the current state-of-the-art for scaling maximum inner product search to massive databases. Traditional approaches to quantization aim to minimize the reconstruction error of the database points. Based on the observation that for a given query, the database points that have the largest…
A unified approach to geometric, symbol and deformation quantizations on a generalized flag manifold endowed with an invariant pseudo-Kaehler structure is proposed. The Hilbert space of states is realized via the Bott-Borel-Weil theorem in the sheaf cohomology of the geometric quantization line bundle. The correspondin…
LIQSS method improves accuracy and efficiency for power system simulations.
In the paper we investigate a method of quantization based on the concept of positive definite kernel on a principal -bundle with compact structural group G. For G=U(1) our approach leads to Kostant-Souriau geometric quantization as well as to coherent state method of quantization. So, the theory proposed here can b…
Computational aspects of the optimal consumption and investment with the partially observed stochastic volatility of the asset prices are considered. The new quantization approach to filtering - density quantization - is introduced which reduces the original infinite dimensional state space of the problem to the finite…
To make deep neural networks feasible in resource-constrained environments (such as mobile devices), it is beneficial to quantize models by using low-precision weights. One common technique for quantizing neural networks is the straight-through gradient method, which enables back-propagation through the quantization ma…
DBQ quantizes lightweight networks efficiently for resource-constrained devices.
State-of-the-art neural machine translation methods employ massive amounts of parameters. Drastically reducing computational costs of such methods without affecting performance has been up to this point unsuccessful. To this end, we propose FullyQT: an all-inclusive quantization strategy for the Transformer. To the bes…
Network quantization is an effective solution to compress deep neural networks for practical usage. Existing network quantization methods cannot sufficiently exploit the depth information to generate low-bit compressed network. In this paper, we propose two novel network quantization approaches, single-level network qu…
Discretizing multi-dimensional data distributions is a fundamental step of modern indexing methods. State-of-the-art techniques learn parameters of quantizers on training data for optimal performance, thus adapting quantizers to the data. In this work, we propose to reverse this paradigm and adapt the data to the quant…
In this paper, we propose a new perspective for quantizing a signal and more specifically the channel state information (CSI). The proposed point of view is fully relevant for a receiver which has to send a quantized version of the channel state to the transmitter. Roughly, the key idea is that the receiver sends the r…
We extend quantization-aware training to extreme model compression.
The article defines and compares two types of quantizations on compact manifolds.
With the development of deep neural networks, the size of network models becomes larger and larger. Model compression has become an urgent need for deploying these network models to mobile or embedded devices. Model quantization is a representative model compression technique. Although a lot of quantization methods hav…
Adler had shown in 1979 that the Toda system can be given a coad- joint orbit description. We quantize the Toda system by viewing it as a single orbit of a multiplicative group of lower triangular matrices of determinant one with pos- itive diagonal entries. We get a unitary representation of the group with square inte…
New quantization methods improve accuracy of Random Fourier Features.
QuantEase optimizes LLMs with CD-based quantization, achieving state-of-the-art performance.
The deep layers of modern neural networks extract a rather rich set of features as an input propagates through the network. This paper sets out to harvest these rich intermediate representations for quantization with minimal accuracy loss while significantly reducing the memory footprint and compute intensity of the DN…
The study bounds entanglement entropy for coherent states on Kähler manifolds.
Recent machine learning methods use increasingly large deep neural networks to achieve state of the art results in various tasks. The gains in performance come at the cost of a substantial increase in computation and storage requirements. This makes real-time implementations on limited resources hardware a challenging …
Although deep neural networks are highly effective, their high computational and memory costs severely challenge their applications on portable devices. As a consequence, low-bit quantization, which converts a full-precision neural network into a low-bitwidth integer version, has been an active and promising research t…
Continuous representations have been widely adopted in recommender systems where a large number of entities are represented using embedding vectors. As the cardinality of the entities increases, the embedding components can easily contain millions of parameters and become the bottleneck in both storage and inference du…
Single model corrects JPEG artifacts for various compression settings.
Consider a physical system for which a mathematically rigorous geometric quantization procedure exists. Now subject the system to a finite set of irreducible first class (bosonic) constraints. It is shown that there is a mathematically rigorous BRST quantization of the constrained system whose cohomology at ghost numbe…
One of the most significant bottleneck in training large scale machine learning models on parameter server (PS) is the communication overhead, because it needs to frequently exchange the model gradients between the workers and servers during the training iterations. Gradient quantization has been proposed as an effecti…
We propose Additive Powers-of-Two~(APoT) quantization, an efficient non-uniform quantization scheme for the bell-shaped and long-tailed distribution of weights and activations in neural networks. By constraining all quantization levels as the sum of Powers-of-Two terms, APoT quantization enjoys high computational effic…
The Dirac equation for massive free electrically neutral spin 1/2 particles in a gravitation field is considered. The secondary quantization procedure is applied to it and the Hilbert space of multiparticle quantum states is constructed.
DQA efficiently quantizes deep neural network activations for resource-constrained devices.
AskewSGD optimizes quantized neural networks with interval-constrained optimization.
For phase-space manifolds which are compact Kaehler manifolds relations between the Berezin-Toeplitz quantization and the quantization with the help of Berezin's coherent states and symbols are studied. First the results on the Berezin-Toeplitz quantization of arbitrary compact Kaehler manifolds due to Bordemann, Meinr…
Adversarial examples have emerged as a significant threat to machine learning algorithms, especially to the convolutional neural networks (CNNs). In this paper, we propose two quantization-based defense mechanisms, Constant Quantization (CQ) and Trainable Quantization (TQ), to increase the robustness of CNNs against ad…
New method shows unitarity in quantization for toric manifolds.
Due to its efficiency and ease to implement, stochastic gradient descent (SGD) has been widely used in machine learning. In particular, SGD is one of the most popular optimization methods for distributed learning. Recently, quantized SGD (QSGD), which adopts quantization to reduce the communication cost in SGD-based di…
This paper addresses a challenging problem - how to reduce energy consumption without incurring performance drop when deploying deep neural networks (DNNs) at the inference stage. In order to alleviate the computation and storage burdens, we propose a novel dataflow-based joint quantization approach with the hypothesis…
The consequences for Berezin's quantization on symmetric spaces of the identity of the set of coherent vectors orthogonal to a fixed one with the cut locus are stated precisely. It is shown that functions expressing the coherent states, the covariant symbols of operators, the diastasis function, the characteristic and …
Quantization needs evaluation of all of states of a quantized object rather than its stationary states with respect to its energy. In this paper, we have investigated moduli $\CMeP$ of a quantized elastica, a quantized loop with an energy functional associated with the Schwarz derivative, on a Riemann sphere $\PP$. The…
We introduce a data-free quantization method for deep neural networks that does not require fine-tuning or hyperparameter selection. It achieves near-original model performance on common computer vision architectures and tasks. 8-bit fixed-point quantization is essential for efficient inference on modern deep learning …
In this lecture results on the Berezin-Toeplitz quantization of arbitrary compact quantizable Kaehler manifolds are presented. These results are obtained in joint work with M. Bordemann and E. Meinrenken. The existence of the Berezin-Toeplitz deformation quantization is also covered. Recent results obtained in joint wo…
Study SO(3)-knot states for torus complements, linking to simplicial volume.
In the framework of geometric quantization we extend the Bohr-Sommerfeld rules to a full quantization theory which resembles Heisenberg's matrix theory. This extension is possible because Bohr-Sommerfeld rules not only provide an orthogonal basis in the space of quantum states, but also give a lattice structure to this…
This paper compresses neural networks by permuting and quantizing weights.
In this paper, we introduce the problem of decision-oriented communications, that is, the goal of the source is to send the right amount of information in order for the intended destination to execute a task. More specifically, we restrict our attention to how the source should quantize information so that the destinat…
It is shown that the heat operator in the Hall coherent state transform for a compact Lie group is related with a Hermitian connection associated to a natural one-parameter family of complex structures on . The unitary parallel transport of this connection establishes the equivalence of (geometric) quantizati…