For fast and energy-efficient deployment of trained deep neural networks on resource-constrained embedded hardware, each learned weight parameter should ideally be represented and stored using a single bit. Error-rates usually increase when this requirement is imposed. Here, we report large improvements in error rates …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,982 papers · 148 categories
Trend · papers per month
48 results for “Bit Error Rates”
Improves bit error tolerance in RRAM-based BNNs without overfitting.
problem Bit errors in RRAM-based BNNs reduce accuracy and overfit to training error rates.
method Proposes straight-through gradient approximation and a novel regularizer.
result Improves BNNs' robustness to bit errors without overfitting.
Improved 2-bit covariance estimator with reduced operator norm error and no tuning needed.
problem Improving 2-bit covariance estimation with reduced operator norm error and no tuning needed.
method Proposed a new 2-bit covariance matrix estimator using triangular dithering scales.
result Improved operator norm error rate that depends on effective rank of covariance matrix, closing theoretical gap.
AdaBoost improves binary classification in robust one-bit compressed sensing with adversarial errors.
problem Binary classification in robust one-bit compressed sensing with adversarial errors.
method AdaBoost and max--margin-classifier approach, with convergence rates improved under certain feature conditions.
result Improved convergence rates and explanation for harmless interpolating adversarial noise.
High error rates improve neural network performance and reduce power consumption.
problem Training neural networks with high Bit Error Rates (BERs).
method Trained three Binarized Convolutional Neural Network architectures on various datasets with high BERs.
result High BERs do not significantly degrade test accuracy, enabling more efficient hardware.
Paper improves DNN accelerator robustness against bit errors with energy savings.
problem Bit errors in quantized DNN weights reduce energy efficiency.
method Combines robust fixed-point quantization, weight clipping, and random bit error training.
result Significantly improves robustness against random bit errors with high energy savings.
In this paper, we consider the problem of reducing the bit error rate of flash-based solid state drives (SSDs) when cells are subject to inter-cell interference (ICI). By observing that the outputs of adjacent victim cells can be correlated due to common aggressors, we propose a novel channel model to accurately repres…
Paper proposes a CNN-based method for estimating intra frame bits and quality.
problem Efficient video delivery and bit allocation in video coding.
method Deep learning approach using CNNs trained on original frames and encoded distortions.
result Accurate estimation of intra frame bits and quality for better bit allocation.
This paper develops novel deep learning-based architectures and design methodologies for an orthogonal frequency division multiplexing (OFDM) receiver under the constraint of one-bit complex quantization. Single bit quantization greatly reduces complexity and power consumption, but makes accurate channel estimation and…
The paper offers error bounds for quantized dynamical models.
problem Accuracy of dynamical models from dependent data sequences.
method Developed uniform error bounds for quantized models and imperfect optimization algorithms.
result Unified bounds for slow and fast rates, scaling with model encoding bits.
Paper studies distributed learning with limited communication bits, achieving optimal error exponents.
problem Distributed hypothesis testing with constant communication bits.
method Geometric approach in distribution spaces, encoding empirical distributions to transmission bits.
result Optimal achievable error exponents and coding schemes for various communication constraints.
While the channel capacity reflects a theoretical upper bound on the achievable information transmission rate in the limit of infinitely many bits, it does not characterise the information transfer of a given encoding routine with finitely many bits. In this note, we characterise the quality of a code (i. e. a given en…
New protocols show 1-bit mean estimation can be order-optimal without interaction.
problem Can 1-bit mean estimation be optimal without interaction?
method Adaptive and non-adaptive threshold and interval queries, with one adaptive transition.
result Arbitrary non-adaptive quantizers can match the adaptive rate, suggesting interaction is not necessary.
Bayesian Bits unifies quantization and pruning through gradient optimization.
problem Joint mixed precision quantization and pruning for efficient neural networks.
method Gradient-based optimization with a novel bit width decomposition and learnable stochastic gates.
result Bayesian Bits achieves better accuracy vs. efficiency trade-off compared to static bit width networks.
Bit-Swap: Recursive Bits-Back Coding for Lossless Compression with Hierarchical Latent Variablescs.LG
Bit-Swap improves lossless compression for hierarchical latent variable models.
problem Efficient lossless compression for latent variable models with hierarchical structure.
method Generalizes bits-back coding to hierarchical latent variable models with Markov chain structure.
result Achieves superior lossless compression rates for hierarchical latent variable models.
A deep learning autoencoder improves error correction for one-bit quantization.
problem Improving error correction for one-bit quantization in AWGN channels.
method Proposes a novel autoencoder-based coding scheme using turbo codes as implicit regularization.
result Empirically and theoretically shows nearly optimal performance of the proposed coding scheme.
We consider the problem of noisy 1-bit matrix completion under an exact rank constraint on the true underlying matrix . Instead of observing a subset of the noisy continuous-valued entries of a matrix , we observe a subset of noisy 1-bit (or binary) measurements generated according to a probabilistic model. W…
Paper tackles 1-bit compressed sensing, presenting efficient algorithm for sparse signal estimation.
problem Estimating sparse signals from binary measurements.
method Non-convex sparsity-constrained program with one-shot hard thresholding.
result Simple algorithm produces accurate signal approximation with high probability.
New MIMO constellation design for noncoherent communications reduces hardware complexity.
problem Designing efficient MIMO constellations for noncoherent communications over fading channels.
method Geodesic curves of the Grassmann manifold for structured constellation design.
result Achieves comparable error performance to unstructured designs with reduced hardware complexity.
One-bit quantization improves inference speed for Random Features models.
problem Efficient inference on resource-constrained devices.
method Analysis of one-bit quantization in Random Features model.
result Asymptotically, quantizing weights except the last incurs no loss in generalization error.
We consider the problem of estimating the mean of a symmetric log-concave distribution under the constraint that only a single bit per sample from this distribution is available to the estimator. We study the mean squared error as a function of the sample size (and hence the number of bits). We consider three settings:…
Moniqua improves SGD convergence with quantized communication.
problem Efficiently communicating in decentralized SGD with limited bandwidth.
method Modulo quantized communication in decentralized SGD.
result Moniqua converges at the same rate as full-precision communication with less bits.
Deep latent variable models have seen recent success in many data domains. Lossless compression is an application of these models which, despite having the potential to be highly useful, has yet to be implemented in a practical manner. We present `Bits Back with ANS' (BB-ANS), a scheme to perform lossless compression w…
In recent years Deep Neural Networks (DNNs) have been rapidly developed in various applications, together with increasingly complex architectures. The performance gain of these DNNs generally comes with high computational costs and large memory consumption, which may not be affordable for mobile platforms. Deep model q…
Paper proposes training deep neural networks with 8-bit floating point precision.
problem Challenges in training deep neural networks at 8-bit precision due to higher precision and dynamic range requirements.
method Proposes a method to train deep neural networks using 8-bit floating point for weights, activations, errors, and gradients. Introduces an enhanced loss scaling method and stochastic rounding technique.
result Demonstrates state-of-the-art accuracy across multiple datasets and workloads compared to full precision baseline.
Paper proposes a 1-bit quantization scheme for high-dimensional statistical estimation.
problem High-dimensional statistical estimation with limited data.
method Uniformly dithered 1-bit quantization for sparse covariance matrix estimation, sparse linear regression, and matrix completion.
result Near minimax rates in sub-Gaussian regime and improved rates in heavy-tailed regime.
Study Thompson Sampling in adversarial bit prediction, finding regret bounds and optimal sequences.
problem Adversarial bit prediction with varying error weights.
method Thompson Sampling, analyzing sequences with largest and smallest regret.
result Regret bounds for adversarial bit prediction sequences, including optimal and worst-case scenarios.
AdaptivFloat improves deep learning inference accuracy at low precision.
problem Low precision quantization issues in deep learning inference.
method Dynamic floating-point representation with adaptive clipping.
result Consistently higher inference accuracy at low precision compared to other methods.
Majority bit estimation in noisy random recursive DAGs.
problem Estimating the majority bit in a noisy random recursive DAG.
method Majority rule among nodes, with bit flipping and noisy channel.
result Identification of the threshold for at which majority rule yields errors.
In this work, we propose an end-to-end block-based auto-encoder system for image compression. We introduce novel contributions to neural-network based image compression, mainly in achieving binarization simulation, variable bit rates with multiple networks, entropy-friendly representations, inference-stage code optimiz…
We consider the problem of estimating the arithmetic average of a finite collection of real vectors stored in a distributed fashion across several compute nodes subject to a communication budget constraint. Our analysis does not rely on any statistical assumptions about the source of the vectors. This problem arises as…
Paper analyzes BIHT for noisy 1-bit CS, improving results with up to τ-fraction of incorrect measurements.
problem Estimating sparse vectors from noisy sign measurements in 1-bit compressed sensing.
method Binary Iterative Hard Thresholding (BIHT) algorithm, using Gaussian matrices and high-dimensional geometry analysis.
result BIHT provides estimates within ε+τ error with τ-fraction of incorrect measurements, maintaining universality of measurements.
Autoencoder shapes QAM symbols for improved GMI performance.
problem Improving Geometric Mean Information (GMI) in QAM constellation shapes.
method End-to-end learning of bit mappings using autoencoders.
result Achieved up to 0.2 bits/QAM symbol gain in GMI.
Deep learning improves SCMA decoding and codebook construction for 5G wireless networks.
problem Designing low complexity high accuracy decoding algorithms and constructing optimal SCMA codebooks.
method Proposed a deep neural network (DNN) called DL-SCMA and an autoencoder (AE-SCMA) to learn and reconstruct SCMA modulated signals.
result Deep learning SCMA decoder outperforms conventional algorithms in BER, SER, and computational complexity.
New framework recovers sparse vectors via ReLU networks, achieving near-optimal statistical rate.
problem Robust one-bit compressed sensing with implicit sparsity constraints.
method Unconstrained empirical risk minimization on a ReLU generative network.
result Achieves a statistical rate of m = ~kn log(d/ε^2) for uniform recovery of any G(x0).
In this paper, we continue our previous work on the Dirichlet mixture model (DMM)-based VQ to derive the performance bound of the LSF VQ. The LSF parameters are transformed into the LSF domain and the underlying distribution of the LSF parameters are modelled by a DMM with finite number of mixture components. The…
Diffusion models improve image compression at low bit-rates.
problem Efficiently compressing images at very low bit-rates.
method Encoding into an embedding, using diffusion models to refine the embedding iteratively.
result Realistic reconstructions can be generated at extremely low bit-rates.
Two methods for model adaptation compared; fine-tuning outperforms Best-of-N in realizable settings.
problem Comparing methods for adapting large language models to new tasks.
method Supervised fine-tuning vs. Best-of-N approach.
result Supervised fine-tuning outperforms Best-of-N in realizable settings.
Paper proves a new lower bound on calibration error for binary prediction.
problem Proving a strong lower bound on calibration error for binary prediction.
method Developed two new techniques: early stopping and sidestepping.
result Proves an lower bound on calibration error.
Deep neural networks approximate functions in shift-invariant spaces with controlled error.
problem Approximating functions in shift-invariant spaces with neural networks.
method Using deep ReLU neural networks, estimating approximation error bounds based on network width and depth.
result Deep neural networks achieve optimal approximation rates for Sobolev spaces up to a logarithmic factor.
We study an extreme scenario in multi-label learning where each training instance is endowed with a single one-bit label out of multiple labels. We formulate this problem as a non-trivial special case of one-bit rank-one matrix sensing and develop an efficient non-convex algorithm based on alternating power iteration. …
Study 1-bit compressive sensing with generative models, improving recovery accuracy.
problem Accurately recover sparse vectors from binary measurements with generative models.
method Analyzes noiseless and noisy 1-bit measurements with i.i.d.~Gaussian and Lipschitz continuous generative priors, proving sample complexity bounds and stability properties.
result Proves sample complexity bounds and stability properties for 1-bit compressive sensing with generative models.
We study the risk performance of distributed learning for the regularization empirical risk minimization with fast convergence rate, substantially improving the error analysis of the existing divide-and-conquer based distributed learning. An interesting theoretical finding is that the larger the diversity of each local…
Paper proposes IABF to improve NECST robustness.
problem Limited robustness of NECST learned coding networks.
method Infomax Adversarial-Bit-Flip (IABF) to improve stability and robustness.
result IABF achieves state-of-the-art performances on compression and error correction benchmarks.
We consider in this paper the problem of noisy 1-bit matrix completion under a general non-uniform sampling distribution using the max-norm as a convex relaxation for the rank. A max-norm constrained maximum likelihood estimate is introduced and studied. The rate of convergence for the estimate is obtained. Information…
Study quantile reward identification with 1-bit feedback constraints.
problem Best arm identification with quantile reward and 1-bit communication.
method Proposes an algorithm using noisy binary search for quantile reward estimation.
result Derives upper and lower bounds on sample complexity for 1-bit feedback.
We propose a novel receiver for orthogonal frequency division multiplexing (OFDM) transmissions in impulsive noise environments. Impulsive noise arises in many modern wireless and wireline communication systems, such as Wi-Fi and powerline communications, due to uncoordinated interference that is much stronger than the…
Neural network VQ-VAE with WaveNet decodes speech at 1.6 kbps with high quality.
problem Efficiently transmitting and storing speech signals at low bit-rates.
method VQ-VAE and WaveNet architecture for speech coding.
result Speech coding at 1.6 kbps with perceptual quality between MELP and AMR-WB.