A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Paper proposes a CNN-based method for estimating intra frame bits and quality.
Improved neural quantization reduces accuracy loss to less than 1% with 4-bit weights.
QTAML models quantum tunneling errors for AI robustness.
We formulate the notion of minimax estimation under storage or communication constraints, and prove an extension to Pinsker's theorem for nonparametric estimation over Sobolev ellipsoids. Placing limits on the number of bits used to encode any estimator, we give tight lower and upper bounds on the excess risk due to qu…
New techniques improve 16-bit training accuracy without 32-bit units.
Bayesian Bits unifies quantization and pruning through gradient optimization.
Low-bit training framework reduces energy consumption in CNNs.
Paper improves DNN accelerator robustness against bit errors with energy savings.
The bits-back argument suggests that latent variable models can be turned into lossless compression schemes. Translating the bits-back argument into efficient and practical lossless compression schemes for general latent variable models, however, is still an open problem. Bits-Back with Asymmetric Numeral Systems (BB-A…
In this paper we explore techniques for generating new music using a Variational Autoencoder (VAE) neural network that was trained on a corpus of specific style. Instead of randomly sampling the latent states of the network to produce free improvisation, we generate new music by querying the network with musical input …
The state-of-the-art hardware platforms for training Deep Neural Networks (DNNs) are moving from traditional single precision (32-bit) computations towards 16 bits of precision -- in large part due to the high energy efficiency and smaller bit storage associated with using reduced-precision representations. However, un…
Paper proves existence of a universal codebook for low-precision quantization.
Majority bit estimation in noisy random recursive DAGs.
Emerging resistive random-access memory (ReRAM) has recently been intensively investigated to accelerate the processing of deep neural networks (DNNs). Due to the in-situ computation capability, analog ReRAM crossbars yield significant throughput improvement and energy reduction compared to traditional digital methods.…
Improves matrix multiplication throughput for asymmetric bit-width operands.
New protocols show 1-bit mean estimation can be order-optimal without interaction.
Topological data analysis classifies encrypted bits with success.
We consider the problem of estimating the mean of a symmetric log-concave distribution under the constraint that only a single bit per sample from this distribution is available to the estimator. We study the mean squared error as a function of the sample size (and hence the number of bits). We consider three settings:…
Paper offers robust recovery for 1-bit sensing with partial Gaussian circulant matrices.
One-bit feedback suffices for a bandit problem's optimal strategy.
New algorithm tackles batched stochastic linear bandits with 1-bit communication constraints.
Implementing large-scale deep neural networks with high computational complexity on low-cost IoT devices may inevitably be constrained by limited computation resource, making the devices hard to respond in real-time. This disjunction makes the state-of-art deep learning algorithms, i.e. CNN (Convolutional Neural Networ…
ADD embeds a 48-bit message into images, achieving high accuracy and speed.
This letter proposes a dictionary learning algorithm for blind one bit compressed sensing. In the blind one bit compressed sensing framework, the original signal to be reconstructed from one bit linear random measurements is sparse in an unknown domain. In this context, the multiplication of measurement matrix $\Ab$ an…
New method tightens federated probe-logit distillation rates under varying bandwidths.
Generalizes bits back coding for time-series models with latent Markov structures.
FleXOR trains fractional quantization for neural networks, improving accuracy and size.
Reduced precision computation for deep neural networks is one of the key areas addressing the widening compute gap driven by an exponential growth in model size. In recent years, deep learning training has largely migrated to 16-bit precision, with significant gains in performance and energy efficiency. However, attemp…
Paper studies distributed learning with limited communication bits, achieving optimal error exponents.
WrapNet optimizes inference for low-resolution neural networks by using 8-bit additions.
The method of random projections has become a standard tool for machine learning, data mining, and search with massive data at Web scale. The effective use of random projections requires efficient coding schemes for quantizing (real-valued) projected data into integers. In this paper, we focus on a simple 2-bit coding …
Kernel Quantization improves CNN compression without sacrificing performance.
Minwise hashing is the standard technique in the context of search and databases for efficiently estimating set (e.g., high-dimensional 0/1 vector) similarities. Recently, b-bit minwise hashing was proposed which significantly improves upon the original minwise hashing in practice by storing only the lowest b bits of e…
Paper proposes a 1-bit quantization scheme for high-dimensional statistical estimation.
Bit threads provide an alternative description of holographic entanglement, replacing the Ryu-Takayanagi minimal surface with bulk curves connecting pairs of boundary points. We use bit threads to prove the monogamy of mutual information (MMI) property of holographic entanglement entropies. This is accomplished using t…
Study quantile reward identification with 1-bit feedback constraints.
This letter proposes a sparse diffusion steepest-descent algorithm for one bit compressed sensing in wireless sensor networks. The approach exploits the diffusion strategy from distributed learning in the one bit compressed sensing framework. To estimate a common sparse vector cooperatively from only the sign of measur…
One-bit quantization improves inference speed for Random Features models.
The article applies Occam's Razor to non-parametric model building, minimizing the number of bits for data encoding.
We study an extreme scenario in multi-label learning where each training instance is endowed with a single one-bit label out of multiple labels. We formulate this problem as a non-trivial special case of one-bit rank-one matrix sensing and develop an efficient non-convex algorithm based on alternating power iteration. …
A new method for 1-bit matrix completion that is faster and more accurate.
This paper introduces a differentiable, scalable quantization method for neural networks.
Low-precision DNNs have been extensively explored in order to reduce the size of DNN models for edge devices. Recently, the posit numerical format has shown promise for DNN data representation and compute with ultra-low precision in [5..8]-bits. However, previous studies were limited to studying posit for DNN inference…
A new quantization strategy reduces Transformer model size and inference time.
Recently, the posit numerical format has shown promise for DNN data representation and compute with ultra-low precision ([5..8]-bit). However, majority of studies focus only on DNN inference. In this work, we propose DNN training using posits and compare with the floating point training. We evaluate on both MNIST and F…
Machine learning, and more specifically deep learning, have shown remarkable performance in sensing, communications, and inference. In this paper, we consider the application of the deep unfolding technique in the problem of signal reconstruction from its one-bit noisy measurements. Namely, we propose a model-based mac…
New constraints on space and adaptivity in bandits force more batches and memory use.
In this paper, we propose to (seamlessly) integrate b-bit minwise hashing with linear SVM to substantially improve the training (and testing) efficiency using much smaller memory, with essentially no loss of accuracy. Theoretically, we prove that the resemblance matrix, the minwise hashing matrix, and the b-bit minwise…