End-to-end auto-encoder for neural image compression with variable bit rates.
problem Efficient image compression with variable bit rates.
method Block-based auto-encoder system with novel contributions.
result Incremental performance improvement of each contribution.
Bit-Swap improves lossless compression for hierarchical latent variable models.
problem Efficient lossless compression for latent variable models with hierarchical structure.
method Generalizes bits-back coding to hierarchical latent variable models with Markov chain structure.
result Achieves superior lossless compression rates for hierarchical latent variable models.
A new method compresses MNIST dataset with VAE at near optimal rate.
problem Practical lossless compression with latent variable models.
method Bits Back with ANS (BB-ANS) for near optimal compression rate.
result Achieved superior compression rates to standard methods.
Paper proposes a CNN-based method for estimating intra frame bits and quality.
problem Efficient video delivery and bit allocation in video coding.
method Deep learning approach using CNNs trained on original frames and encoded distortions.
result Accurate estimation of intra frame bits and quality for better bit allocation.
Develops a method for lossless compression using latent variable models.
problem Lossless compression of large datasets.
method Bits back with asymmetric numeral systems (BB-ANS) using latent variable models.
result Achieves state-of-the-art lossless compression of full-size colour images.
Wide residual networks achieve low error rates with single-bit weights.
problem Deploying deep neural networks on resource-constrained hardware with low memory.
method Binarizing weights using sign function and scaling factors, applying warm-restart learning rate schedule.
result Achieved error rates of 3.9% on CIFAR-10, 18.5% on CIFAR-100, and 26.0% on ImageNet with 1-bit-per-weight.
New protocols show 1-bit mean estimation can be order-optimal without interaction.
problem Can 1-bit mean estimation be optimal without interaction?
method Adaptive and non-adaptive threshold and interval queries, with one adaptive transition.
result Arbitrary non-adaptive quantizers can match the adaptive rate, suggesting interaction is not necessary.
Moniqua improves SGD convergence with quantized communication.
problem Efficiently communicating in decentralized SGD with limited bandwidth.
method Modulo quantized communication in decentralized SGD.
result Moniqua converges at the same rate as full-precision communication with less bits.
Improved 2-bit covariance estimator with reduced operator norm error and no tuning needed.
problem Improving 2-bit covariance estimation with reduced operator norm error and no tuning needed.
method Proposed a new 2-bit covariance matrix estimator using triangular dithering scales.
result Improved operator norm error rate that depends on effective rank of covariance matrix, closing theoretical gap.
New VAE limits latent layer information rate for better performance.
problem Improving latent layer information in VAEs.
method Imposes a signal-to-noise ratio on latent layer information.
result BIR-VAE provides meaningful latent representation with specified information rate.
Paper proposes a 1-bit quantization scheme for high-dimensional statistical estimation.
problem High-dimensional statistical estimation with limited data.
method Uniformly dithered 1-bit quantization for sparse covariance matrix estimation, sparse linear regression, and matrix completion.
result Near minimax rates in sub-Gaussian regime and improved rates in heavy-tailed regime.
Improves bit error tolerance in RRAM-based BNNs without overfitting.
problem Bit errors in RRAM-based BNNs reduce accuracy and overfit to training error rates.
method Proposes straight-through gradient approximation and a novel regularizer.
result Improves BNNs' robustness to bit errors without overfitting.
This paper introduces new methods to improve 1-bit matrix completion by considering cluster effects.
problem Improving 1-bit matrix completion for clustered data.
method Group-Specific 1-bit Matrix Completion (GS1MC) and Cluster Developing Matrix Completion (CDMC).
result GS1MC and CDMC outperform existing methods in synthetic and real-world data.
BEGIN network models binary data without parametric assumptions.
problem Conditional independence in non-parametric families of binary data.
method BEGIN network models binary data using sparse linear representations and block factorizations.
result BEGIN network captures conditional independence for arbitrary binary and multinomial variables.
Autoencoder shapes QAM symbols for improved GMI performance.
problem Improving Geometric Mean Information (GMI) in QAM constellation shapes.
method End-to-end learning of bit mappings using autoencoders.
result Achieved up to 0.2 bits/QAM symbol gain in GMI.
Diffusion models improve image compression at low bit-rates.
problem Efficiently compressing images at very low bit-rates.
method Encoding into an embedding, using diffusion models to refine the embedding iteratively.
result Realistic reconstructions can be generated at extremely low bit-rates.
Two methods for model adaptation compared; fine-tuning outperforms Best-of-N in realizable settings.
problem Comparing methods for adapting large language models to new tasks.
method Supervised fine-tuning vs. Best-of-N approach.
result Supervised fine-tuning outperforms Best-of-N in realizable settings.
The paper proposes a nonparametric test for incomplete samples quantized to B bits.
problem Statistical inference with lossy or incomplete samples.
method Nonparametric testing procedure based on B-bit quantized samples.
result The proposed test achieves the classical minimax rate of testing for spline models when B exceeds a threshold.
Paper proposes a new detector for SSDs with ICI, reducing bit error rates significantly.
problem Reducing bit error rate in SSDs with inter-cell interference.
method Proposes a novel channel model and uses sum-product algorithm for accurate posterior distributions, then passes them to a soft LDPC decoder. Introduces an iterative scheme to improve performance in quantized cases.
result Significant reduction in bit error rate, reaching 99.99% in simulations with strong diagonal coupling.
We consider in this paper the problem of noisy 1-bit matrix completion under a general non-uniform sampling distribution using the max-norm as a convex relaxation for the rank. A max-norm constrained maximum likelihood estimate is introduced and studied. The rate of convergence for the estimate is obtained. Information…
Neural network VQ-VAE with WaveNet decodes speech at 1.6 kbps with high quality.
problem Efficiently transmitting and storing speech signals at low bit-rates.
method VQ-VAE and WaveNet architecture for speech coding.
result Speech coding at 1.6 kbps with perceptual quality between MELP and AMR-WB.
One-bit proximal method speeds up nonconvex stochastic optimization.
problem Reducing communication in distributed SGD for large datasets.
method Stochastic proximal gradient method using one-bit per update.
result The method achieves convergence rates similar to uncompressed SGD.
AdaBoost improves binary classification in robust one-bit compressed sensing with adversarial errors.
problem Binary classification in robust one-bit compressed sensing with adversarial errors.
method AdaBoost and max-ℓ1-margin-classifier approach, with convergence rates improved under certain feature conditions. result Improved convergence rates and explanation for harmless interpolating adversarial noise.
Binary sequence correlation estimation fails but trinary data succeeds.
problem Estimating correlation in binary sequences generated by thresholding a hidden continuous sequence.
method Formal analysis and numerical experiments on likelihood maximization and discretization effects.
result Consistent estimation of correlation is possible with trinary data but not with binary data.
Deep learning improves one-bit OFDM receiver performance.
problem One-bit quantization complicates accurate channel estimation and data detection in OFDM receivers.
method Developed deep neural networks for channel estimation and data detection, using a two-step training policy.
result Deep learning-based designs achieve lower BER than unquantized OFDM at moderate SNRs.
New method improves image compression using bits-back coding.
problem Lossy image compression with deep latent variable models.
method Iterative inference, stochastic annealing, bits-back coding.
result New state-of-the-art performance on lossy image compression.
One-bit clustering method for two-component sub-Gaussian mixture models
problem Clustering in sub-Gaussian mixture models
method One-bit clustering using dithered quantization
result Decaying misclassification rate with exponential signal-to-noise ratio
New insights into optimizing latent representations in hierarchical VAEs.
problem Optimizing latent representations in hierarchical VAEs for various applications.
method Identifying a general class of inference models and deriving theoretical bounds on performance.
result Guidance for practitioners on optimal regions in rate-space for different applications.
Study finds optimal learning rate schedules for sub-100M quantization-aware training across bit-widths.
problem Optimal learning rate schedules for quantization-aware training depend on bit-width.
method Factorial grid testing over bit-width, warmdown fraction, LR magnitude, model size, and seed.
result INT6 QAT requires a different schedule than higher-precision training, falsifying the primary hypothesis.
A new efficient test addresses limitations of knockoffs for conditional independence testing.
problem Testing conditional independence under model-X assumptions.
method Leave-One-Covariate-Out Conditional Randomization Test (LOCO-CRT)
result LOCO-CRT produces valid p-values for familywise error rate control with minimal variability. Paper predicts VQ performance for LSF using DMM in the ΔLSF domain.
problem Predicting VQ performance for LSF parameters.
method Transform LSF parameters to ΔLSF domain, model with DMM, calculate MSE, estimate bit rate.
result Estimates minimum bit rate for transparent coding of LSF.
New method compresses neural networks using random code, improving efficiency.
problem Large memory footprint of deep neural networks.
method Training a variational distribution over weights, encoding using Kullback-Leibler divergence.
result Achieves state-of-the-art compression rates and test performance.
FibQuant improves KV-cache compression for long-context inference.
problem Memory traffic bottleneck in long-context inference due to KV cache growth.
method Introduces FibQuant, a universal vector quantizer that combines Beta-quantile radii, Fibonacci/Roberts-Kronecker directions, and Lloyd-Max refinement.
result FibQuant achieves high compression rates with minimal loss in attention cosine similarity.
T-BFA targets and misleads specific DNN inputs to a chosen output.
problem Targeted attack on DNN weight parameters to hijack function.
method Identifies critical weight bits, ranks them by class dependence, and flips them to mislead inputs.
result Successfully misclassifies images from 'Hen' to 'Goose' class with 100% success rate, maintaining 59.35% validation accuracy.
High error rates improve neural network performance and reduce power consumption.
problem Training neural networks with high Bit Error Rates (BERs).
method Trained three Binarized Convolutional Neural Network architectures on various datasets with high BERs.
result High BERs do not significantly degrade test accuracy, enabling more efficient hardware.
We consider the problem of noisy 1-bit matrix completion under an exact rank constraint on the true underlying matrix M∗. Instead of observing a subset of the noisy continuous-valued entries of a matrix M∗, we observe a subset of noisy 1-bit (or binary) measurements generated according to a probabilistic model. W…
Paper tackles 1-bit compressed sensing, presenting efficient algorithm for sparse signal estimation.
problem Estimating sparse signals from binary measurements.
method Non-convex sparsity-constrained program with one-shot hard thresholding.
result Simple algorithm produces accurate signal approximation with high probability.
A new algorithm for compressing latent representations in deep models.
problem Compressing continuous latent representations in deep models.
method Separates model design and training from quantization; uses adaptive quantization based on posterior uncertainty.
result Image compression with the proposed algorithm outperforms JPEG over a wide range of bit rates.
New algorithm sets record for model evasion with minimal queries.
problem Evasion of machine learning models in black-box settings.
method Sign-based gradient estimation for binary black-box optimization.
result Achieves highest evasion rates and lowest query efficiency.
ActNN reduces neural network training memory by 2-bit quantization.
problem Limited memory in training neural networks.
method Randomly quantizes activations to 2 bits, proving convergence and proposing mixed-precision strategies.
result Reduces activation memory footprint by 12x with negligible accuracy loss.
Paper shows interaction not necessary for optimal 1-bit mean estimation.
problem Optimal one-bit mean estimation with minimal interaction.
method Developed a fully non-adaptive protocol that avoids interaction.
result Achieved optimal sample complexity without interaction.
Paper introduces adversarial lossy compression for video artifacts reduction.
problem Unpleasant reconstruction artifacts in standard video coding schemes at low bit-rates.
method Adversarial lossy video compression model minimizing an adversarial distortion objective.
result Reduction of perceptual artifacts and detail reconstruction under extreme compression.
AdaptivFloat improves deep learning inference accuracy at low precision.
problem Low precision quantization issues in deep learning inference.
method Dynamic floating-point representation with adaptive clipping.
result Consistently higher inference accuracy at low precision compared to other methods.
New scheme optimizes BMI through probabilistic and geometric shaping.
problem Optimizing bit-wise mutual information (BMI) for coded modulation.
method Joint optimization of BMI through probabilistic and geometric shaping.
result Joint optimization enables a continuum of constellation geometries and probability distributions.
New techniques improve 16-bit training accuracy without 32-bit units.
problem Training deep learning models with only 16-bit floating-point units.
method Studied BFloat16 units and applied stochastic rounding and Kahan summation techniques.
result Up to 7% absolute validation accuracy gain in 16-bit-FPU training.
The one-bit quantization is implemented by one single comparator that operates at low power and a high rate. Hence one-bit compressive sensing (1bit-CS) becomes attractive in signal processing. When measurements are corrupted by noise during signal acquisition and transmission, 1bit-CS is usually modeled as minimizing …
The property of perfectness plays an important role in the theory of Bayesian networks. First, the existence of perfect distributions for arbitrary sets of variables and directed acyclic graphs implies that various methods for reading independence from the structure of the graph (e.g., Pearl, 1988; Lauritzen, Dawid, La…
New framework recovers sparse vectors via ReLU networks, achieving near-optimal statistical rate.
problem Robust one-bit compressed sensing with implicit sparsity constraints.
method Unconstrained empirical risk minimization on a ReLU generative network.
result Achieves a statistical rate of m = ~kn log(d/ε^2) for uniform recovery of any G(x0).