This paper introduces a quantization-based regularizer for autoencoders to improve latent representations.
problem Autoencoders can overfit and collapse, leading to poor latent representations.
method The authors combine VQ-VAE and denoising methods to introduce a bottleneck Bayesian estimator that soft quantizes latent codes.
result The method results in better latent representations for supervised and clustering tasks.
Geometrically constructs representations for quantization on Kähler manifolds.
problem Quantization of Kähler manifolds using Berezin-Toeplitz method.
method Using peak sections to localize Hilbert spaces around points in the large volume limit.
result Geometric construction of representations for Berezin-Toeplitz quantization.
NEMO framework quantizes DNNs for efficient deployment.
problem Efficient deployment of quantized DNNs.
method Formal framework for quantizing DNN layers, focusing on IntegerDeployable representation.
result Quantized DNNs can be deployed using only integers.
Quantized-TinyLLaVA reduces communication costs in split learning for multimodal models.
problem High communication costs in split learning for multimodal models.
method Integrates a compression module that quantizes intermediate features into discrete representations before transmission.
result Achieves an approximate 87.5% reduction in communication overhead with 2-bit quantization.
Recent research implies that training and inference of deep neural networks (DNN) can be computed with low precision numerical representations of the training/test data, weights and gradients without a general loss in accuracy. The benefit of such compact representations is twofold: they allow a significant reduction o…
New method for quantizing symplectic manifolds with Lagrangian bundles.
problem Quantization of symplectic manifolds with Lagrangian bundles.
method A new construction of strict deformation quantization.
result Established a correspondence between differential operators and principal symbols.
Geometric quantization shows compatibility of symmetries on coadjoint orbits and Kähler-Einstein manifolds.
problem Compatibility of symmetries in geometric quantization.
method Deformation and geometric quantization on Kähler manifolds, Hamiltonian actions.
result Strict compatibility of symmetries on coadjoint orbits and Kähler-Einstein manifolds.
We construct Hermitian representations of Lie algebroids and associated unitary representations of Lie groupoids by a geometric quantization procedure. For this purpose we introduce a new notion of Hamiltonian Lie algebroid actions. The first step of our procedure consists of the construction of a prequantization line …
Divide and conquer quantizes neural networks, improving accuracy.
problem Quantizing neural networks to reduce memory and compute.
method Divide a pretrained network into sections, train each section independently, then stitch them.
result Improves quantized training accuracy by 21.6% on average.
Researchers extend geometric quantization to complex Abelian Lie supergroups.
problem Quantization of super Kähler structures on complex Abelian Lie supergroups.
method Extended geometric quantization scheme to super Kähler setting, constructed unitary representation.
result Irreducible subrepresentations of the constructed representation are determined by the moment map.
We prove that there are no nontrivial finite-dimensional Lie representations of certain Poisson algebras of polynomials on a compact symplectic manifold. This result is used to establish the existence of a universal obstruction to quantizing a compact symplectic manifold, regardless of the dimensionality of the represe…
Construct geometric interpretation of Heston model using group quantization.
problem Geometric interpretation of Heston model
method Lifted local Lie groupoid formulation
result Geometric interpretation of Heston pricing operator and Riccati equations
This paper proposes a new weight representation scheme for efficient model compression and performance enhancement.
problem Challenges in achieving performance enhancement on devices due to irregular sparse matrix representations.
method Fine-grained and unstructured pruning method combined with structured weight encryption.
result Achieved high compression ratios and performance on various deep learning models.
Extends ONNX for quantized neural networks with new formats and operators.
problem Handling arbitrary-precision quantization in neural networks.
method Introduces new formats and operators in ONNX to represent quantized neural networks.
result Enabled representation of uniform quantization in neural networks.
A G-equivariant spin^c structure on a manifold gives rise to a virtual representation of the group G, called the spin^c quantization of the manifold. We present a cutting construction for S^1-equivariant spin^c manifolds, and show that the quantization of the original manifold is isomorphic to the direct sum of the qua…
Paper introduces a technique to simplify RNN policies for better understanding and analysis.
problem Difficulty in explaining and analyzing RNN policies due to continuous-valued memory vectors and observation features.
method Quantized Bottleneck Insertion technique to learn finite representations of RNN vectors and features.
result Finite representations of RNN policies can be as small as 3 discrete memory states and 10 observations, improving interpretability.
A new algorithm for compressing latent representations in deep models.
problem Compressing continuous latent representations in deep models.
method Separates model design and training from quantization; uses adaptive quantization based on posterior uncertainty.
result Image compression with the proposed algorithm outperforms JPEG over a wide range of bit rates.
Deep task-based quantization improves MIMO signal processing.
problem Improving performance of MIMO signal processing with scalar ADCs.
method Data-driven task-oriented quantization using deep learning.
result Deep task-based quantization can approach optimal performance limits.
A new framework for neural network classification using vector quantization.
problem Learning a neural network classifier under the IB principle.
method Aggregated Learning framework, combining vector quantization and variational techniques.
result The effectiveness of Aggregated Learning verified through experiments.
The relations between the infinite dimensional geometry of qR-conformal symmetries at qR→∞, Berezin quantization of the Lobachevskii plane and Karasev-Maslov asymptotic quantization are explicated. Some aspects of the ``approximate'' representation theory are discussed.
This work reduces model size by 86.11% for recommender systems using 4-bit quantization.
problem Large memory consumption in embedding vectors for recommender systems.
method Post-training 4-bit quantization on embedding tables, including row-wise uniform quantization and codebook-based quantization.
result Consistently reduces accuracy degradation while significantly reducing model size.
Quantizes moduli space of 3D gravity metrics.
problem Quantize moduli space of 3D gravity metrics.
method Develops geometrically natural classes of observables and uses cluster X-varieties. result Obtains projective unitary representations of mapping class group.
Verma Howe duality connects tensor products of Verma modules to LKB representations.
problem Understanding the relationship between tensor products of Verma modules and LKB representations.
method Established a quantized version of Verma Howe duality and used it to prove the simplicity of LKB representations.
result LKB representations arise from the quantized Verma Howe duality and are shown to be simple modules.
Mackey showed that for a compact Lie group K, the pair (K,C0(K)) has a unique non-trivial irreducible covariant pair of representations. We study the relevance of this result to the unitary equivalence of quantizations for an infinite-dimensional family of K×K invariant polarizations on T∗K. The …
Efficiently learns quantizable embeddings for fast search.
problem Learning binary hamming code representations for search efficiency.
method Directly learns a quantizable embedding representation and sparse binary hash code end-to-end.
result Achieves state-of-the-art search accuracy and significant speedup.
A geometric quantization of a Kähler manifold, viewed as a symplectic manifold, depends on the complex structure compatible with the symplectic form. The quantizations form a vector bundle over the space of such complex structures. Having a canonical quantization would amount to finding a natural (projectively) flat co…
We construct an explicit scheme to associate to any potential symbol an operator acting between sections of natural bundles (associated to irreducible representations) for a so-called AHS-structure. Outside of a finite set of critical (or resonant) weights, this procedure gives rise to a quantization, which is intrinsi…
This paper is devoted to the pricing of Barrier options by optimal quadratic quantization method. From a known useful representation of the premium of barrier options one deduces an algorithm similar to one used to estimate nonlinear filter using quadratic optimal functional quantization. Some numerical tests are fulfi…
A new model SEQ clusters and classifies encoded features for better interpretability.
problem Lack of interpretability in classical supervised classification tasks.
method Proposes a novel supervised learning model named Supervised-Encoding Quantizer (SEQ) that applies a quantizer to cluster and classify encoded features.
result The quantizer provides an interpretable graph where each cluster represents a class with a particular style.
Paper proposes IIQ for compressing embedding vectors.
problem Memory issues in representing large vocabularies.
method Isotropic iterative quantization (IIQ) for binary compression.
result More than 30x compression ratio with comparable performance.
We quantize the interaction of gravity with Yang-Mills and spinor fields, hence offering a quantum theory incorporating all four fundamental forces of nature. Using canonical quantization we obtain solutions of the Wheeler-DeWitt equation in a vector bundle and the method of second quantization leads to a symplectic ve…
In this article we develop tools to compute the Geometric Quantization of a symplectic manifold with respect to a regular Lagrangian foliation via sheaf cohomology and obtain important new applications in the case of real polarizations. The starting point is the definition of representation spaces due to Kostant. Besid…
We apply the geometric quantization method with real polarizations to the quantization of a symplectic torus. By quantizing with half-densities we canonically associate to the symplectic torus a projective Hilbert space and prove that the projective factor is expressible in terms of the Maslov-Kashiwara index. As in th…
We prove that there is no faithful finite-dimensional representation by skew-hermitian matrices of a ``basic algebra of observables'' B on a noncompact symplectic manifold M. Consequently there exists no finite-dimensional quantization of any Lie subalgebra of the Poisson algebra C^\infty(M) containing B.
Improved vector quantization using Gaussian mixtures for better codebook utilization.
problem Training instability and information loss in discrete vector quantization.
method Generalized vector quantization with Gaussian mixture model and aggregated categorical posterior evidence lower bound.
result GM-VQ improves codebook utilization and reduces information loss without heuristics.
Develops Hamiltonian quantization for complex Chern-Simons theory at even level k.
problem Quantum holonomies and representation theory in complex Chern-Simons theory.
method Combinatorial quantization and operator algebra construction.
result Physical Hilbert space identified and Fenchel-Nielsen representation demonstrated.
Study asymptotics of unitary matrix elements in quantum mechanics.
problem Asymptotic behavior of unitary matrix elements in quantum mechanics.
method Uses Berezin-Toeplitz quantization and symplectic geometry.
result Recover asymptotics of Wigner's d-matrix elements for spin representations.
Quantization of universal Teichmüller space provides projective representations of the Ptolemy-Thompson group, which is isomorphic to the Thompson group T. This yields certain central extensions of T by Z, called dilogarithmic central extensions. We compute a presentation of the dilogarithmic central ext…
Quantized neural networks reduce model size and energy consumption.
problem Memory and energy constraints in mobile devices.
method Using integer or binary representations to store weights instead of 32-bit floats.
result Quantization can reduce model size and energy consumption without significantly compromising performance.
Distributed quantization improves classification accuracy with less data.
problem Efficiently classify features from distributed nodes with limited communication.
method Designs tailored quantization schemes for classification, proving NP-hardness and proposing polynomial-time algorithms.
result Tailored quantizers can reduce bit communication by more than a factor of two for the same accuracy.
A new method learns discrete representations for images and videos, improving upon previous models.
problem Learning discrete representations for images and videos to improve performance.
method Depthwise application of Vector Quantized Variational Autoencoders (VQVAE) to feature axis.
result 33% improvement in performance compared to previous discrete models.
VQ-GNN scales GNNs to large graphs using vector quantization.
problem Scaling GNNs to large graphs with stable performance and speed.
method VQ-GNN uses vector quantization to preserve all messages passed to a mini-batch of nodes, avoiding the 'neighbor explosion' problem.
result VQ-GNN achieves competitive performance on large-graph node classification and link prediction benchmarks.
New method corrects quantization errors in LLMs using low-rank matrices.
problem Correcting quantization errors in large language models.
method Introducing low-rank weight matrices to correct quantized activations in LLMs.
result Reduces accuracy gap with original model by more than 50% using low-rank matrices.
Deep learning compresses and quantizes log-likelihood ratios for fading channels.
problem Efficiently compress and quantize log-likelihood ratios for fading channels.
method Trains a deep autoencoder network to map log-likelihood ratios to a latent space and reconstruct them.
result Achieves a compression factor of nearly three times with minimal performance loss.
Geometric quantization for symplectic maps via Toeplitz operators.
problem Quantization of symplectic maps and Witten's conjecture.
method Berezin-Toeplitz operators and holomorphic sections over Kähler manifolds.
result Established a semi-classical trace formula for quantum representations of mapping class groups.
A new quantization strategy reduces Transformer model size and inference time.
problem Heavy computation load and memory overhead in Transformer models for mobile devices.
method Mixed precision quantization with varying bits per word in embedding blocks.
result 11.8x smaller model size and 3.5x speed up for on-device NMT.
Adler had shown in 1979 that the Toda system can be given a coad- joint orbit description. We quantize the Toda system by viewing it as a single orbit of a multiplicative group of lower triangular matrices of determinant one with pos- itive diagonal entries. We get a unitary representation of the group with square inte…
Decentralized detection avoids sharing data, controls false discoveries.
problem Global false discovery rate control in decentralized novelty detection.
method Quantized surrogate models for low-precision sharing, preserving exchangeability.
result Quantized composite scores maintain competitive statistical power with reduced communication.