This paper introduces a differentiable, scalable quantization method for neural networks.
problem Previous quantization methods lacked differentiability and scalability.
method The approach is differentiable and scalable, using bit-shifting and logarithmic quantization.
result The method achieves comparable accuracy to state-of-the-art approaches with less training time and lower inference cost.
Differentiable model compression adds noise to parameters during training.
problem Model compression for efficient deployment.
method Adding pseudo quantization noise to model parameters during training.
result Compresses model by more than 8x on ImageNet with 0.3% accuracy loss.
DJPQ optimizes neural network pruning and quantization for hardware efficiency.
problem Efficiently compress neural networks for hardware inference.
method Joint gradient-based optimization of pruning and quantization into a differentiable loss function.
result Significant reduction in Bit-Operations (BOPs) with minimal accuracy loss.
Quantizes Kähler manifolds using sheaves and differential operators.
problem Quantizing Kähler manifolds with sheaves and differential operators.
method Constructing a category enriched over sheaves of modules, defining quantizable morphisms, and showing equivalence to differential operator categories.
result Equivalence of quantized categories under certain conditions.
Paper shows how to integrate quantization into neural compression models.
problem Integrating quantization into neural compression models.
method Integrates uniform noise channel at test time using universal quantization.
result Eliminates mismatch between training and test phases while maintaining differentiability.
Differentially quantized gradient methods improve convergence in noisy communication channels.
problem Optimizing distributed learning with limited communication bandwidth and noise.
method Introduces Differential Quantization (DQ) to compensate for quantization errors in gradient descent.
result DQ-GD achieves the same contraction factor as unquantized GD at high bitrates, proving asymptotic optimality.
Quantizes functions on Kähler manifolds without formal deformation.
problem Deforming smooth functions on Kähler manifolds to non-formal quantization.
method Using Fedosov connections and prequantum line bundles.
result Quantizable functions form a sheaf of twisted differential operators.
DBQ quantizes lightweight networks efficiently for resource-constrained devices.
problem High computational and storage complexity of deep neural networks on resource-constrained devices.
method A differentiable non-uniform quantizer that can be mapped onto efficient ternary-based dot product engines.
result Achieves state-of-the-art results with minimal training overhead and best accuracy-complexity trade-off.
The paper quantizes Kähler manifolds using differential operators.
problem Quantizing classical observables on Kähler manifolds as differential operators.
method Constructing higher-order differential operators using Fedosov-type constructions and proving asymptotic equivalence to Berezin-Toeplitz operators.
result Holomorphic differential operators are precisely those that arise as Berezin-Toeplitz operators for quantizable functions.
Study non-commutative function algebras using contact geometry.
problem Quantize non-commutative function algebras in several variables.
method Contact geometry and rational differential operators.
result Generalizes known constructions of classical equivariants.
New method for flux quantization on phase space stacks.
problem Defining and constructing flux-quantized phase space stacks.
method Observation of flux densities and characterization of Cauchy data.
result Flux-quantized phase space stacks have classifying spaces with rational Whitehead L-infinity algebra.
New method for quantizing symplectic manifolds with Lagrangian bundles.
problem Quantization of symplectic manifolds with Lagrangian bundles.
method A new construction of strict deformation quantization.
result Established a correspondence between differential operators and principal symbols.
We review and elaborate on some aspects of the quantization of certain classes of higher abelian gauge theories using techniques of generalized differential cohomology. Particular emphasis is placed on the examples of generalized Maxwell theory and Cheeger-Simons cohomology, and of Ramond-Ramond fields in Type II super…
New method quantizes neural networks for mobile devices.
problem High computational and memory costs of deep neural networks.
method Formulates quantization as a differentiable function.
result Quantization networks outperform state-of-the-art methods.
Study differential operators over maps and their applications in supermanifolds.
problem Understanding differential operators over smooth maps and their applications.
method Recall and study differential operators, formal ℏ-differential operators, pullbacks by thick morphisms, and quantization of symplectic micromorphisms. result Developed constructions and examples of differential operators over maps.
We present a theoretical and experimental investigation of the quantization problem for artificial neural networks. We provide a mathematical definition of quantized neural networks and analyze their approximation capabilities, showing in particular that any Lipschitz-continuous map defined on a hypercube can be unifor…
New insights into quantized neural networks reveal learning dynamics and generalization errors.
problem Understanding the impact of quantization hyperparameters on learning dynamics in high-dimensional models.
method Theoretical analysis and fixed-point analysis of STE dynamics in quantized models.
result STE training in quantized models converges to a plateau followed by a sharp drop in generalization error, influenced by quantization range.
Noncommutative geometry connects higher order connections to quantization.
problem Quantization in noncommutative geometry.
method Introducing natural linear differential operators and Spencer operators.
result Higher order connections are equivalent to quantization.
We study the existence of natural and projectively equivariant quantizations for differential operators acting between order 1 vector bundles over a smooth manifold M. To that aim, we make use of the Thomas-Whitehead approach of projective structures and construct a Casimir operator depending on a projective Cartan con…
Neural network quantization has become an important research area due to its great impact on deployment of large models on resource constrained devices. In order to train networks that can be effectively discretized without loss of performance, we introduce a differentiable quantization procedure. Differentiability can…
Conformally equivariant quantization is a peculiar map between symbols of real weight δ and differential operators acting on tensor densities, whose real weights are designed by λ and λ+δ. The existence and uniqueness of such a map has been proved by Duval, Lecomte and Ovsienko for a generic weight δ. Later, Si…
We investigate (pseudo)differential forms in the framework of supergeometry. Definitions, basic properties and Cartan calculus (DeRham differential, Lie derivative, inner product, Hodge operator) are presented; the symplectic supermechanics (even and odd) is formulated; and the question of quantization is discussed. In…
DPQ offers significant compression at minimal cost for embedding layers.
problem Memory and storage constraints in embedding layers.
method Differentiable Product Quantization (DPQ) framework with two instantiations.
result Significant compression ratios (14-238imes) with negligible performance cost. We give an explicit local formula for any formal deformation quantization, with separation of variables, on a Kähler manifold. The formula is given in terms of differential operators, parametrized by acyclic combinatorial graphs.
In this paper, Hamiltonian monodromy is studied from the point of view of geometric quantization abd theta functions, and various differential geometric aspects thereof are dealt with, all related to holonomies of suitable flat connections.
This thesis introduces the notion of "relative gerbes" for smooth maps of manifolds, and discusses their differential geometry. The equivalence classes of relative gerbes are classified by the relative integral cohomology in degree three. Furthermore, by using the concept of relative gerbes, the pre-quantization of Lie…
This talk reports on results on the deformation quantization (star products) and on approximative operator representations for quantizable compact K"ahler manifolds obtained via Berezin-Toeplitz operators. After choosing a holomorphic quantum line bundle the Berezin-Toeplitz operator associated to a differentiable func…
The spaces of higher-order differential operators (in Dimension 1|2), which are modules over the stringy Lie superalgebra K(2), are isomorphic to the corresponding spaces of symbols as orthosymplectic modules in non resonant cases. Such an osp (2|2)-equivariant quantization, which has been given in second-order differe…
A new method for robust product Markovian quantization overcomes numerical instabilities.
problem Numerical instabilities in the PMQ algorithm limit its adoption, especially for stochastic volatility models.
method Reformulated PMQ as standard vector quantization, applying accelerated Lloyd's algorithm for robustness.
result The method overcomes numerical instabilities and extends applicability to stochastic volatility models.
We construct Hermitian representations of Lie algebroids and associated unitary representations of Lie groupoids by a geometric quantization procedure. For this purpose we introduce a new notion of Hamiltonian Lie algebroid actions. The first step of our procedure consists of the construction of a prequantization line …
We extend projectively equivariant quantization and symbol calculus to symbols of pseudo-differential operators. An explicit expression in terms of hypergeometric functions with noncommutative arguments is given. Some examples are worked out, one of them yielding a quantum length element on S3.
Lectures on symplectic and Poisson geometry, quantization, and quantum field theory.
problem Exploring symplectic and Poisson structures and their applications in quantum field theory.
method Introduction to differential geometry, symplectic geometry, Poisson geometry, and deformation quantization.
result Detailed understanding of symplectic and Poisson structures and their quantization.
Extends quantization theory to mixed polarizations using transverse differential operators.
problem Quantization in mixed polarization.
method Developed a theory of transverse differential operators associated to non-singular polarizations.
result Obtained a geometric interpretation of deformation quantization and sheaf of subalgebras acting on polarized sections.
We give an explicit formula for the projectively invariant quantization map between the space of symbols of degree three and the space of third-order linear differential operators, both viewed as modules over the group of diffeomorphisms and the Lie algebra of vector fields on a manifold.
Differential Cohomotopy theory predicts brane interactions via chord diagrams.
problem Quantization of brane charges and moduli spaces.
method Differential refinement of Cohomotopy theory, configuration spaces, chord diagrams.
result Higher observables on brane moduli spaces are given by weight systems on chord diagrams.
This article is a survey of recent work of the authors developing a new approach to quantization based on the equivariance with respect to some Lie group of symmetries. Examples are provided by conformal and projective differential geometry: given a smooth manifold M endowed with a flat conformal/projective structure, …
I have chosen, in this presentation of Deformation Quantization, to focus on 3 points: the uniqueness --up to equivalence-- of a universal star product (universal in the sense of Kontsevich) on the dual of a Lie algebra, the cohomology classes introduced by Deligne for equivalence classes of differential star products …
Introduces a new operator generating higher Koszul brackets on differential forms.
problem Developing a new operator for higher Koszul brackets on differential forms.
method Introducing a formal ℏ-differential operator Δ generating higher Koszul brackets on differential forms. result Established properties of the introduced BV type operator and its inclusion in a one-parameter family.
Analyzes quantization of flux observables in gauge theories.
problem Lifting Poisson-brackets of flux observables to higher moduli stacks.
method Systematic analysis of canonical quantization and flux quantization laws.
result Topological quantum observables form homology Pontrjagin algebra of loop space.
For a real symmetric domain GR/KR, with complexification GC/KC, we introduce the concept of "star-restriction" (a real analogue of the "star-products" for quantization of Kähler manifolds) and give a geometric construction of the GR-invariant differential ope…
In the first part of the paper we describe the complex geometry of the universal Teichmüller space T, which may be realized as an open subset in the complex Banach space of holomorphic quadratic differentials in the unit disc. The quotient S of the diffeomorphism group of the circle modulo Möbius …
Efficient deep neural network (DNN) inference on mobile or embedded devices typically involves quantization of the network parameters and activations. In particular, mixed precision networks achieve better performance than networks with homogeneous bitwidth for the same size constraint. Since choosing the optimal bitwi…
A quantization over a manifold can be seen as a way to construct a differential operator with prescribed principal symbol. The quantization map is moreover required to be a linear bijection. It is known that there is in general no natural quantization procedure. However, considering manifolds endowed with additional st…
Paper optimizes KWS models using NAS and quantization for limited resources.
problem Developing efficient keyword spotting models in resource-constrained environments.
method Neural Architecture Search (NAS) for model structure optimization and quantization of weights and activations.
result Achieved high accuracy (95.55%) with minimal parameters and operations using NAS and quantization.
Quantization of (-1)-shifted derived Poisson manifolds via BV-infinity operators.
problem Quantizing (−1)-shifted derived Poisson manifolds. method Using BV-infinity operators on the space of Berezinian half-densities, proving quantization via lifting of Maurer-Cartan elements.
result Quantization of (−1)-shifted derived Poisson manifolds is equivalent to the vanishing of the second Poisson cohomology group. Discretizing multi-dimensional data distributions is a fundamental step of modern indexing methods. State-of-the-art techniques learn parameters of quantizers on training data for optimal performance, thus adapting quantizers to the data. In this work, we propose to reverse this paradigm and adapt the data to the quant…
CoDeQ simplifies joint model compression by integrating pruning and quantization.
problem Joint pruning and quantization methods are complex and require additional procedures.
method CoDeQ uses a dead-zone quantizer to directly induce sparsity and learn quantization parameters.
result CoDeQ achieves high sparsity and low-precision accuracy with minimal bit operations.
Paper proposes BQNs for efficient Bayesian quantized neural networks.
problem Learning with well-calibrated uncertainty in neural networks.
method Bayesian quantized networks (BQNs) with efficient algorithms for learning and prediction without sampling.
result BQNs achieve lower predictive errors and better-calibrated uncertainties than E-QNN with less than 20% negative log-likelihood.