Media is generally stored digitally and is therefore discrete. Many successful deep distribution models in deep learning learn a density, i.e., the distribution of a continuous random variable. Naïve optimization on discrete data leads to arbitrarily high likelihoods, and instead, it has become standard practice to add…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,694 papers · 148 categories
Trend · papers per month
23 results for “Dequantization”
MSD removes dequantization bottleneck in LLM inference by approximating high-precision activations.
problem Dequantization bottleneck in LLM inference on modern AI accelerators.
method MSD decomposes high-precision activations into multiple low-precision components for direct multiplication with quantized weights.
result MSD avoids INT8-to-BF16 weight conversion, reducing dequantization cycles and HBM traffic.
On the relation between trainability and dequantization of variational quantum learning modelsquant-ph
This work explores the relation between trainability and dequantization in variational QML models.
problem Understanding the interplay between trainability and dequantization in variational QML models.
method Provide precise definitions of trainability and dequantization, study their relation, and introduce recipes for building PQC-based QML models.
result Identify conditions under which trainability and non-dequantization are not mutually exclusive.
Method estimates densities on manifolds using dequantization.
problem Estimating densities on non-Euclidean manifolds.
method Inspired by dequantization, coordinate transformation, and normalizing flows.
result Successfully models densities on spheres, tori, and orthogonal groups.
Flow models have recently made great progress at modeling ordinal discrete data such as images and audio. Due to the continuous nature of flow models, dequantization is typically applied when using them for such discrete data, resulting in lower bound estimates of the likelihood. In this paper, we introduce subset flow…
Flow-based generative models are powerful exact likelihood models with efficient sampling and inference. Despite their computational efficiency, flow-based models generally have much worse density modeling performance compared to state-of-the-art autoregressive models. In this paper, we investigate and improve upon thr…
Quantum circuits reveal pathways to dequantization in machine learning models.
problem Navigating the complex landscape of quantum machine learning models and algorithms.
method Introducing a framework connecting quantum circuit structure to function representability.
result Fundamental properties of quantum circuits determine classical simulability of models.
In this work, we propose a deep neural network architecture motivated by primal-dual splitting methods from convex optimization. We show theoretically that there exists a close relation between the derived architecture and residual networks, and further investigate this connection in numerical experiments. Moreover, we…
We consider the reconstruction problem in compressed sensing in which the observations are recorded in a finite number of bits. They may thus contain quantization errors (from being rounded to the nearest representable value) and saturation errors (from being outside the range of representable values). Our formulation …
Paper introduces new methods for modeling categorical data.
problem Training generative models on categorical data like text and segmentation.
method Argmax Flows and Multinomial Diffusion models.
result Models outperform existing methods in log-likelihood.
Sparse Max-Affine Regressionstat.ML
Paper proposes Sp-GD for sparse max-affine regression with theoretical guarantees.
problem Sparse max-affine regression model selection and estimation.
method Sparse Gradient Descent (Sp-GD) initialization using sparse PCA and covering search.
result Sp-GD provides ε-accurate estimates with optimal number of observations.
Novel flows generate molecules without post-processing.
problem Generating new molecules efficiently and without post-processing issues.
method Continuous normalizing E(3)-equivariant flows based on node ODEs coupled as a graph PDE.
result Generated samples achieve state-of-the-art performance on QM9 and ZINC250K benchmarks.
Extends ONNX for quantized neural networks with new formats and operators.
problem Handling arbitrary-precision quantization in neural networks.
method Introduces new formats and operators in ONNX to represent quantized neural networks.
result Enabled representation of uniform quantization in neural networks.
Unified framework for robust, stable, and efficient density ratio estimation.
problem Density-chasm and support-chasm problems in density ratio estimation.
method Dequantified diffusion-Schrödinger bridge (D3RE) framework with DDBI and DSBI.
result Offers uniform approximation and bounded time scores in theory and empirical performance.
A new method learns quantization boundaries in continuous space using tessellation.
problem Mapping between discrete and continuous distributions is difficult.
method Constructs normalizing flows on convex polytopes with exact likelihood evaluations.
result Improves likelihood evaluation and quantization learning across various data modalities.
Let be a knot, its complement, and the circle group identified with . To any oriented long knot diagram of , we associate a quadratic polynomial in variables bijectively associated with the bridges of the diagram such that, when the variables pr…
FleXOR trains fractional quantization for neural networks, improving accuracy and size.
problem Quantization limits to integer bits restricts compression and accuracy.
method Encryption algorithm with XOR gates for fractional bits during inference.
result FleXOR achieves high accuracy with fractional sub-1-bit weights.
SurVAE Flows combine VAEs and flows using surjective transformations.
problem Combining the strengths of VAEs and flows to model complex densities.
method Modular framework of composable deterministic and stochastic transformations.
result Exact likelihood computation and lower bound on likelihood.
Improved EXACT strategy reduces GNN memory consumption and runtime.
problem Efficiently training large-scale GNNs with reduced memory usage.
method Block-wise quantization of intermediate activation maps with improved variance minimization.
result Further reduction in memory consumption (>15%) and runtime speedup (5%) with similar performance trade-offs.
Develops a new method for learning discrete distributions without embedding them in a continuous space.
problem Challenges in learning discrete distributions using current methodologies.
method Introduces a MAD invertible map and a mixed variational flow (MAD Mix) for discrete distributions.
result MAD Mix produces more reliable approximations than continuous-embedding flows.
Paper introduces Categorical Normalizing Flows for better handling of categorical data.
problem Limited application of normalizing flows on categorical data due to lack of intrinsic order.
method Categorical Normalizing Flows use continuous transformations to model latent relations in categorical data, optimizing both continuous representation and model likelihood.
result GraphCNF, a permutation-invariant generative model, outperforms state-of-the-art on molecule generation.
New classical algorithm outperforms quantum in neural network subnetwork selection.
problem Selecting sparse subnetworks from large neural networks efficiently.
method Quantum-inspired classical algorithm using ridgelet transform sampling.
result Runs in polynomial time, outperforming naive classical methods.
Personalized Public Policy Analysis in Social Sciences using Causal-Graphical Normalizing Flowscs.LG
Develops c-GNF for personalized social science policy analysis.
problem Challenges in estimating causal effects and counterfactual inference in social sciences.
method causal-Graphical Normalizing Flow (c-GNF) method.
result c-GNF performs well in estimating causal effects and counterfactual inference.