ProxQuant improves quantized neural networks using proximal operators.
problem Making neural networks work on devices with limited resources.
method Formulates quantized network training as a regularized learning problem and optimizes it via the prox-gradient method.
result ProxQuant outperforms state-of-the-art results on binary quantization and is on par with state-of-the-art on multi-bit quantization.
ADD embeds a 48-bit message into images, achieving high accuracy and speed.
problem Embedding high-fidelity messages into images to detect authenticity and source.
method Two-stage process: linear combination and addition of watermark to image, followed by decoding.
result ADD achieves 100% decoding accuracy for 48-bit watermarking, with minimal performance drop under various distortions.
Recurrent neural networks have achieved excellent performance in many applications. However, on portable devices with limited resources, the models are often too large to deploy. For applications on the server with large scale concurrent requests, the latency during inference can also be very critical for costly comput…
Improved 2-bit covariance estimator with reduced operator norm error and no tuning needed.
problem Improving 2-bit covariance estimation with reduced operator norm error and no tuning needed.
method Proposed a new 2-bit covariance matrix estimator using triangular dithering scales.
result Improved operator norm error rate that depends on effective rank of covariance matrix, closing theoretical gap.
Survey on quantization methods on Kähler manifolds.
problem None explicitly stated; focuses on methods.
method Deformation quantization, geometric quantization, Berezin-Toeplitz quantization, BV quantization.
result New relationships among quantization methods on Kähler manifolds.
Paper models entropy-based impact of soft errors on neural network inference.
problem Estimating impact of radiation-induced faults on neural network inference.
method Entropy-based statistical models for SEU and MBU across layers.
result Accurate models to evaluate error-resiliency of neural network topologies.
The paper classifies quantizable functions and explores symmetry in quantization methods.
problem Classifying quantizable functions and understanding symmetry in quantization methods.
method Deformation quantization and geometric quantization methods are compared and classified.
result Formal quantizable functions are of a specific form and relate to Hamiltonian Killing vector fields.
This paper introduces a differentiable, scalable quantization method for neural networks.
problem Previous quantization methods lacked differentiability and scalability.
method The approach is differentiable and scalable, using bit-shifting and logarithmic quantization.
result The method achieves comparable accuracy to state-of-the-art approaches with less training time and lower inference cost.
StatQAT optimizes quantization for deep networks, reducing computational cost and memory usage.
problem Optimal quantization parameters selection for deep neural networks with diverse data distributions.
method Statistical error analysis framework for uniform and floating-point quantization, iterative and analytic quantizers designed for arbitrary and Gaussian-like distributions.
result Improved accuracy and stability in training low-precision neural networks.
Smart Quantization adapts binary and ternary quantization for neural networks.
problem Resource constraints in deploying neural networks on devices with limited resources.
method Adaptive combination of binary and ternary quantization with a regularization function.
result Adapts quantization depth during training to maintain high model accuracy.
This study optimizes quantized neural networks by considering model architecture and quantization types.
problem Optimizing quantized neural networks for low-power, high-throughput applications.
method Holistic approach including training methods and quantization-friendly architecture design.
result Deeper models are more sensitive to activation quantization, while wider models improve resilience to both weight and activation quantization.
RATQ is a new quantizer for optimizing noisy gradients in machine learning.
problem Optimizing noisy gradients in stochastic optimization.
method RATQ uses Hadamard transform and adaptive uniform quantization, and achieves near-optimal performance.
result RATQ nearly achieves information theoretic lower bounds for optimization accuracy.
Extends ONNX for quantized neural networks with new formats and operators.
problem Handling arbitrary-precision quantization in neural networks.
method Introduces new formats and operators in ONNX to represent quantized neural networks.
result Enabled representation of uniform quantization in neural networks.
New method for quantizing symplectic manifolds with Lagrangian bundles.
problem Quantization of symplectic manifolds with Lagrangian bundles.
method A new construction of strict deformation quantization.
result Established a correspondence between differential operators and principal symbols.
HMQ improves quantization for edge devices with mixed precision.
problem Efficient quantization for edge devices with uniform, power-of-two thresholds.
method Introduces HMQ, a mixed precision quantization block that repurposes Gumbel-Softmax for searching over quantization schemes.
result Achieves competitive and state-of-the-art results on ImageNet despite restrictions.
Introduces sheaf quantization, a topological approach to geometric quantization.
problem Topological realization of WKB-states in geometric quantization.
method Enhancement of constructible sheaves, Betti counterpart of Fukaya--Floer theory.
result Introduction to sheaf quantization as a topological realization of WKB-states.
Network quantization is an effective solution to compress deep neural networks for practical usage. Existing network quantization methods cannot sufficiently exploit the depth information to generate low-bit compressed network. In this paper, we propose two novel network quantization approaches, single-level network qu…
The article defines and compares two types of quantizations on compact manifolds.
problem Quantization on arbitrary compact smooth manifolds.
method Embedding into CP^n and inducing quantizations from there.
result Generalizations of earlier quantization methods.
Quantized Adam reduces communication cost in deep learning training.
problem Reducing communication cost in distributed deep learning training.
method Gradient and weight quantization with error feedback in Adam.
result Proposed methods converge to first-order stationary points.
Proposes a robust neural network quantization method.
problem Training model's dependency on specific quantization methods.
method Intrinsic robustness to various quantization processes.
result Single model capable of operating at various bit-widths and policies.
The paper studies quantization on symplectic manifolds with real polarizations, comparing different quantization methods.
problem Quantization on compact symplectic manifolds with real polarizations.
method Geometric quantization, Toeplitz operators, Fourier transforms, asymptotic expansion of traces.
result Deformation quantization is realized through asymptotic traces of Toeplitz operators.
Quantizes neural networks using frame theory for improved accuracy.
problem Improving neural network efficiency and accuracy through quantization.
method Sigma-Delta (ΣΔ) quantization with finite unit-norm tight frames. result Error bound between original and quantized neural networks derived.
Unified finetuning of all quantization degrees of freedom achieves state-of-the-art 4-bit quantization.
problem Achieving high accuracy in quantized neural networks while maintaining speed and resource constraints.
method Quantization-aware finetuning (QFT) that jointly optimizes all quantization degrees of freedom.
result 4-bit weight quantization results on-par with state-of-the-art (SoTA) within PTQ constraints.
Abstract proposes a new categorical approach to quantization of Poisson algebras.
problem Quantization of Poisson algebras.
method Defining quantization categories as subcategories of R-module categories with classical limits.
result Categories of strict deformation quantization, prequantization, and matrix regularization are equivalent, while Poisson enveloping algebra is not.
Geometric quantization of a Poisson manifold need not imply quantization of its symplectic leaves. We provide the leafwise geometric quantization of a Poisson manifold, seen as a foliated one, whose quantum algebra restricted to each leaf is quantized.
Meta learning optimizes neural network quantization for efficient inference.
problem Uniform bitwidth quantization is sub-optimal for neural network compression.
method Meta learning to automatically generate hybrid quantization policies.
result Meta learning outperforms uniform quantization and RL approaches.
Study quantization effects on high-dimensional linear regression learning.
problem Understanding quantization's impact on learning high-dimensional linear regression models.
method Analyzes stochastic gradient descent for high-dimensional linear regression under various quantization targets.
result Establishes precise bounds on excess risk for different quantization schemes.
Simplified neural network quantization speeds up process by 8 epochs.
problem Efficient quantization of neural networks for mobile devices.
method Trained scale factors for quantization thresholds.
result Significant reduction in training time and model size.
We present an overview of techniques for quantizing convolutional neural networks for inference with integer weights and activations. Per-channel quantization of weights and per-layer quantization of activations to 8-bits of precision post-training produces classification accuracies within 2% of floating point networks…
FrostNet improves INT8 quantization efficiency in mobile networks.
problem The importance of network architecture for optimal INT8 quantization.
method Quantization-aware training (QAT) with StatAssist and GradBoost, hardware-aware NAS.
result FrostNets achieve higher recognition accuracy with comparable latency when quantized.
A new method quantizes LSTM gate parameters without performance loss.
problem Quantization loss in LSTM gate parameters without performance degradation.
method Lossy quantization of gate parameters during training, weight parameters adjust to offset quantization loss.
result F1 score decreased by only 0.7% on Named Entity Recognition dataset.
Optimal gradient quantization reduces communication costs in distributed deep learning.
problem High communication costs in distributed training of deep neural networks.
method Deduced optimal gradient quantization conditions for binary and multi-level quantization, developed novel schemes for dynamic quantization levels.
result Demonstrated superior performance of proposed quantization schemes on CIFAR and ImageNet datasets.
Quantizes symplectic manifolds with bounded geometry using Berezin-Toeplitz method.
problem Quantization of symplectic manifolds with bounded geometry.
method Berezin-Toeplitz quantization theory.
result Correct semiclassical limit achieved.
WNQ uses weight normalization to reduce quantization error in deep neural networks.
problem High quantization error in deep neural networks due to long-tail distribution of weights.
method Weight normalization to avoid long-tail distribution and reduce quantization error.
result WNQ achieves state-of-the-art performance on CIFAR-100 and ImageNet.
A new method defends neural networks from adversarial attacks.
problem Adversarial attacks on quantized neural networks.
method Defensive Quantization (DQ) method to control Lipschitz constant.
result DQ method defends neural networks from adversarial attacks and improves accuracy.
New method quantizes neural networks for mobile devices.
problem High computational and memory costs of deep neural networks.
method Formulates quantization as a differentiable function.
result Quantization networks outperform state-of-the-art methods.
Quantization-aware training can recover accuracy lost by post-training quantization.
problem Post-training quantization (PTQ) can fail sharply at aggressive bitwidths.
method A unified geometric framework that explains PTQ failure and QAT recovery.
result QAT has a useful bias that steers iterates back into the basin.
This study analyzes quantization in deep learning models using statistical physics methods.
problem The computational resource requirements for large-scale data analysis models.
method Typical case analysis from statistical physics, specifically the replica method.
result Optimal quantization width minimizes error and delays overfitting.
Paper develops a statistical framework for quantized training of deep neural networks.
problem Lack of theoretical understanding of gradient quantization in FQT.
method Presented a statistical framework for analyzing FQT algorithms, viewing quantized gradient as a stochastic estimator of QAT gradient.
result Developed two novel gradient quantizers with smaller variance than existing per-tensor quantizer.
A new method for robust product Markovian quantization overcomes numerical instabilities.
problem Numerical instabilities in the PMQ algorithm limit its adoption, especially for stochastic volatility models.
method Reformulated PMQ as standard vector quantization, applying accelerated Lloyd's algorithm for robustness.
result The method overcomes numerical instabilities and extends applicability to stochastic volatility models.
In [3], the authors showed the existence and the uniqueness of a sl(m+1,\R)-equivariant quantization in the non-critical situations. The curved generalization of the sl(m+1,\R)-equivariant quantization is the natural and projectively equivariant quantization. In [1] and [7], the existence of such a quantization was pro…
A unified approach to geometric, symbol and deformation quantizations on a generalized flag manifold endowed with an invariant pseudo-Kaehler structure is proposed. The Hilbert space of states is realized via the Bott-Borel-Weil theorem in the sheaf cohomology of the geometric quantization line bundle. The correspondin…
A new method improves maximum inner product search by locally decomposing residual vectors.
problem Maximum inner product search efficiency and accuracy.
method Local Orthogonal Decomposition (LOD) combined with multiscale quantization.
result LOD consistently achieves higher recall than previous methods under the same bitrates.
Memory-augmented neural networks (MANNs) refer to a class of neural network models equipped with external memory (such as neural Turing machines and memory networks). These neural networks outperform conventional recurrent neural networks (RNNs) in terms of learning long-term dependency, allowing them to solve intrigui…
Quantizes contact structures using dynamical methods.
problem Quantizing contact structures in a flat connection.
method Constructs a dynamical quantization using a flat connection on a Hilbert tractor bundle.
result Determines a contact tractor connection whose parallel sections determine a distinguished choice of Reeb dynamics.
Paper proposes a new binary quantization method for faster DNN inference.
problem Accelerating deep neural network inference on resource-limited devices.
method Quantized Compressed Sensing (QCS) for binary quantization.
result The proposed method preserves benefits of standard methods while reducing quantization error.
LSQ+ improves quantization of neural nets with Swish activations, achieving state-of-the-art results.
problem Quantization of neural nets with Swish activations, especially negative activations, leads to significant performance loss.
method Introduces learnable scale and offset parameters for asymmetric quantization, and uses MSE-based initialization for quantization parameters.
result Significantly outperforms LSQ for low-bit quantization of neural nets with Swish activations, achieving up to 5.6% gain with W2A2 quantization of EfficientNet-B0.
New vector quantization method reduces relevance of parallel components in database points.
problem Scaling maximum inner product search to massive databases.
method Developed anisotropic vector quantization loss functions.
result Achieves state-of-the-art results on public benchmarks.