A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Quantized Adam reduces communication cost in deep learning training.
Adaptive quantization improves SGD accuracy in data-parallel settings.
Neural network models are resource hungry. It is difficult to deploy such deep networks on devices with limited resources, like smart wearables, cellphones, drones, and autonomous vehicles. Low bit quantization such as binary and ternary quantization is a common approach to alleviate this resource requirements. Ternary…
We present Rotated Adaptive Tetra-iterated Quantizer (RATQ), a fixed-length quantizer for gradients in first order stochastic optimization. RATQ is easy to implement and involves only a Hadamard transform computation and adaptive uniform quantization with appropriately chosen dynamic ranges. For noisy gradients with al…
AdaQuantFL reduces communication in federated learning by adaptively quantizing model updates.
The study analyzes convergence of adaptive optimizers under low-precision training.
AdaRound improves post-training quantization of neural networks.
Discretizing multi-dimensional data distributions is a fundamental step of modern indexing methods. State-of-the-art techniques learn parameters of quantizers on training data for optimal performance, thus adapting quantizers to the data. In this work, we propose to reverse this paradigm and adapt the data to the quant…
We formulate the notion of minimax estimation under storage or communication constraints, and prove an extension to Pinsker's theorem for nonparametric estimation over Sobolev ellipsoids. Placing limits on the number of bits used to encode any estimator, we give tight lower and upper bounds on the excess risk due to qu…
We present a novel method for neural network quantization that emulates a non-uniform -quantile quantizer, which adapts to the distribution of the quantized parameters. Our approach provides a novel alternative to the existing uniform quantization techniques for neural networks. We suggest to compare the results as …
Adaptive Quantization Modules enable online continual compression of non-i.i.d data streams.
New protocols show 1-bit mean estimation can be order-optimal without interaction.
The existence of a natural and projectively invariant quantization in the sense of P. Lecomte [Progr. Theoret. Phys. Suppl. (2001), no. 144, 125-132] was proved by M. Bordemann [math.DG/0208171], using the framework of Thomas-Whitehead connections. We extend the problem to the context of supermanifolds and adapt M. Bor…
A new method for robust product Markovian quantization overcomes numerical instabilities.
In recent years Deep Neural Networks (DNNs) have been rapidly developed in various applications, together with increasingly complex architectures. The performance gain of these DNNs generally comes with high computational costs and large memory consumption, which may not be affordable for mobile platforms. Deep model q…
In this work, we propose to quantize all parts of standard classification networks and replace the activation-weight--multiply step with a simple table-based lookup. This approach results in networks that are free of floating-point operations and free of multiplications, suitable for direct FPGA and ASIC implementation…
Proposes QEP to mitigate quantization error propagation in layer-wise post-training quantization.
The present paper develops a novel aggregated gradient approach for distributed machine learning that adaptively compresses the gradient communication. The key idea is to first quantize the computed gradients, and then skip less informative quantized gradient communications by reusing outdated gradients. Quantizing and…
DFS dynamically decides bitwidths for layers to balance accuracy and efficiency.
The paper offers error bounds for quantized dynamical models.
A quantization over a manifold can be seen as a way to construct a differential operator with prescribed principal symbol. The quantization map is moreover required to be a linear bijection. It is known that there is in general no natural quantization procedure. However, considering manifolds endowed with additional st…
We adapt the framework of geometric quantization to the polysymplectic setting. Considering prequantization as the extension of symmetries from an underlying polysymplectic manifold to the space of sections of a Hermitian vector bundle, a natural definition of prequantum vector bundle is obtained which incorporates in …
Post-training quantization method using multiple low-precision points achieves higher precision for critical weights.
We consider the problem of deep neural net compression by quantization: given a large, reference net, we want to quantize its real-valued weights using a codebook with entries so that the training loss of the quantized net is minimal. The codebook can be optimally learned jointly with the net, or fixed, as for bina…
In the first part of this article we provide a geometrically oriented approach to the theory of orbispaces which originally had been introduced by Chen. We explain the notion of a vector orbibundle and characterize the good sections of a reduced vector orbibundle as the smooth stratified sections. In the second part of…
A new algorithm for compressing latent representations in deep models.
Distributed sensors compress and send features to a fusion center for linear regression.
In this article, we propose a communication-efficient decentralized machine learning (ML) algorithm, coined quantized group ADMM (Q-GADMM). To reduce the number of communication links, every worker in Q-GADMM communicates only with two neighbors, while updating its model via the group alternating direction method of mu…
The concept of conformally equivariant quantizations was introduced by Duval, Lecomte and Ovsienko in \cite{DLO} for manifolds endowed with flat conformal structures. They obtained results of existence and uniqueness (up to normalization) of such a quantization procedure. A natural generalization of this concept is to …
We investigate the concept of equivariant quantization over the superspace R^{p+q|2r}, with respect to the orthosymplectic algebra osp(p+1,q+1|2r). Our methods and results vary upon the superdimension p+q-2r. When the superdimension is nonzero, we manage to obtain a result which is similar to the classical theorem of D…
One of the most significant bottleneck in training large scale machine learning models on parameter server (PS) is the communication overhead, because it needs to frequently exchange the model gradients between the workers and servers during the training iterations. Gradient quantization has been proposed as an effecti…
In this paper, we propose a new perspective for quantizing a signal and more specifically the channel state information (CSI). The proposed point of view is fully relevant for a receiver which has to send a quantized version of the channel state to the transmitter. Roughly, the key idea is that the receiver sends the r…
VQ-DRAW compresses images and generates realistic samples.
Deep Neural Networks (DNNs) typically require massive amount of computation resource in inference tasks for computer vision applications. Quantization can significantly reduce DNN computation and storage by decreasing the bitwidth of network encodings. Recent research affirms that carefully selecting the quantization l…
WaveQ uses sinusoidal regularization to optimize deep quantization for neural networks, improving both efficiency and accuracy.
A new method improves quantile regression for high-dimensional data.
Improved vector quantization using Gaussian mixtures for better codebook utilization.
Winograd convolutions are used to improve quantized neural networks.
Paper proves existence of a universal codebook for low-precision quantization.
Quantized deep neural networks (QDNNs) are attractive due to their much lower memory storage and faster inference speed than their regular full precision counterparts. To maintain the same performance level especially at low bit-widths, QDNNs must be retrained. Their training involves piecewise constant activation func…
We develop a Vector Quantized Spectral Clustering (VQSC) algorithm that is a combination of Spectral Clustering (SC) and Vector Quantization (VQ) sampling for grouping Soybean genomes. The inspiration here is to use SC for its accuracy and VQ to make the algorithm computationally cheap (the complexity of SC is cubic in…
Survey on quantization methods on Kähler manifolds.
GLAD improves latent graph generation by quantizing discrete latent space.
We show that using the family of adapted Kähler polarizations of the phase space of a compact, simply connected, Riemannian symmetric space of rank-1, the obtained field of quantum Hilbert spaces produced by geometric quantization including the half-form correction is flat if is the 3-dimensional sphere …
AIS corrects rollout-training mismatch in quantized RL, improving speed and stability.
The paper classifies quantizable functions and explores symmetry in quantization methods.
This paper introduces a differentiable, scalable quantization method for neural networks.
Geometric quantization often produces not one Hilbert space to represent the quantum states of a classical system but a whole family of Hilbert spaces, and the question arises if the spaces are canonically isomorphic. [ADW] and [Hi] suggest to view as fibers of a Hilbert bundle , introduce a connec…