Optimizes neural computation by combining multiple constraints using maximum entropy method.
problem Defines physical limits of neuron-like engineering by optimizing multiple performance criteria.
method Uses Jaynes' maximum entropy method to combine and optimize multiple constraints.
result Identifies a Shannon bits/joule statement as a result of combining constraints.
New techniques improve 16-bit training accuracy without 32-bit units.
problem Training deep learning models with only 16-bit floating-point units.
method Studied BFloat16 units and applied stochastic rounding and Kahan summation techniques.
result Up to 7% absolute validation accuracy gain in 16-bit-FPU training.
Bayesian Bits unifies quantization and pruning through gradient optimization.
problem Joint mixed precision quantization and pruning for efficient neural networks.
method Gradient-based optimization with a novel bit width decomposition and learnable stochastic gates.
result Bayesian Bits achieves better accuracy vs. efficiency trade-off compared to static bit width networks.
Low-bit training framework reduces energy consumption in CNNs.
problem Reducing energy consumption in convolutional neural networks.
method Low-bit training framework using MLS tensor format with dynamic quantization.
result Achieves superior trade-off between accuracy and bit-width.
Bit-Swap improves lossless compression for hierarchical latent variable models.
problem Efficient lossless compression for latent variable models with hierarchical structure.
method Generalizes bits-back coding to hierarchical latent variable models with Markov chain structure.
result Achieves superior lossless compression rates for hierarchical latent variable models.
Paper improves DNN accelerator robustness against bit errors with energy savings.
problem Bit errors in quantized DNN weights reduce energy efficiency.
method Combines robust fixed-point quantization, weight clipping, and random bit error training.
result Significantly improves robustness against random bit errors with high energy savings.
Training deep neural networks with 8-bit floating point numbers is now possible and more efficient.
problem Challenges in training DNNs with reduced precision, especially for gradient computations.
method Introduction of chunk-based accumulation and floating point stochastic rounding to reduce arithmetic precision to 16 bits.
result Successful training of DNNs using 8-bit floating point numbers, maintaining accuracy on various models and datasets.
Bit-slice sparsity improves ReRAM-based DNN acceleration.
problem Limited ADC power and area constraints in ReRAM-based DNN accelerators.
method Proposed bit-slice L1 algorithm to induce sparsity during training.
result 2x sparsity improvement compared to previous methods.
Estimating mean from one-bit samples of symmetric log-concave distributions.
problem Estimating the mean of a symmetric log-concave distribution with limited one-bit measurements.
method Analyzes mean squared error in three settings: centralized, adaptive, and distributed, with and without quantization.
result One round of adaptivity is sufficient to achieve optimal mean-square error in the adaptive setting.
Majority bit estimation in noisy random recursive DAGs.
problem Estimating the majority bit in a noisy random recursive DAG.
method Majority rule among nodes, with bit flipping and noisy channel.
result Identification of the threshold for p at which majority rule yields errors. Paper proposes a CNN-based method for estimating intra frame bits and quality.
problem Efficient video delivery and bit allocation in video coding.
method Deep learning approach using CNNs trained on original frames and encoded distortions.
result Accurate estimation of intra frame bits and quality for better bit allocation.
Improves matrix multiplication throughput for asymmetric bit-width operands.
problem Matrix multiplications between asymmetric bit-width operands, especially 8- and 4-bit, are not efficiently handled by existing SIMD instructions.
method Proposes a new SIMD matrix multiplication instruction that uses mixed precision on inputs (8- and 4-bit) and accumulates into 16-bit output, improving throughput.
result Offers 2x improvement in throughput compared to existing symmetric-operand-size instructions, with negligible overflow.
A new algorithm learns a dictionary for blind one-bit compressed sensing.
problem Reconstructing sparse signals from one-bit compressed measurements without prior knowledge of the sparsity domain.
method Dictionary learning to learn the matrix $\Db=\AbΦ$ using a steepest-descent method.
result The proposed algorithm outperforms the no-dictionary-learning case, especially with more training signals and measurements.
Bit threads prove holographic monogamy of mutual information.
problem Proving the monogamy of mutual information in holographic entanglement.
method Using bit threads and multicommodity flow adapted from network theory, combined with convex optimization tools.
result Proved the monogamy of mutual information property of holographic entanglement entropies.
Paper proposes a hybrid model-based and data-driven approach for one-bit compressive autoencoding.
problem Designing efficient one-bit compressive autoencoding models for complex systems.
method Hybrid model-based and data-driven methodology for one-bit sparse signal recovery.
result Significant improvement in one-bit compressive autoencoding compared to state-of-the-art algorithms.
Sparse diffusion steepest-descent for one-bit CS in sensor networks.
problem Estimating sparse vectors from sign measurements in wireless sensor networks.
method Diffusion strategy combined with steepest-descent optimization for cooperative sparse vector estimation.
result Simulation results show the proposed algorithm outperforms non-distributive methods.
Paper proposes a hybrid model-based and data-driven method for one-bit compressive variational autoencoding.
problem Designing efficient one-bit compressive sensing systems.
method Hybrid model-based and data-driven approach for one-bit compressive variational autoencoding.
result Significant improvement in one-bit compressive sensing compared to state-of-the-art methods.
Paper proposes training deep neural networks with 8-bit floating point precision.
problem Challenges in training deep neural networks at 8-bit precision due to higher precision and dynamic range requirements.
method Proposes a method to train deep neural networks using 8-bit floating point for weights, activations, errors, and gradients. Introduces an enhanced loss scaling method and stochastic rounding technique.
result Demonstrates state-of-the-art accuracy across multiple datasets and workloads compared to full precision baseline.
New protocols show 1-bit mean estimation can be order-optimal without interaction.
problem Can 1-bit mean estimation be optimal without interaction?
method Adaptive and non-adaptive threshold and interval queries, with one adaptive transition.
result Arbitrary non-adaptive quantizers can match the adaptive rate, suggesting interaction is not necessary.
Topological data analysis classifies encrypted bits with success.
problem Classifying encrypted data with traditional machine learning methods.
method Persistent homology for generating topological features, machine learning pipeline.
result Successfully classifies encrypted data, outperforming classical models.
New method reduces deep learning complexity on IoT devices.
problem High computational complexity limits deep learning on IoT devices.
method Local quantization region for low-bit data representation.
result Models retain accuracy with reduced computational complexity.
2-bit coding scheme improves near neighbor search efficiency.
problem Efficiently estimating data similarities in near neighbor search.
method Developed 2-bit quantization and nonlinear estimators.
result 2-bit strategy significantly reduces computational complexity.
DeepRec uses deep learning to recover signals from one-bit measurements.
problem Signal recovery from one-bit noisy measurements.
method Deep unfolding of inference optimization into deep neural network layers.
result DeepRec improves accuracy and computational efficiency.
Paper offers robust recovery for 1-bit sensing with partial Gaussian circulant matrices.
problem Accurately recovering vectors from 1-bit measurements using structured matrices.
method Correlation-based optimization with randomly signed partial Gaussian circulant matrices and generative models.
result Recovery guarantees match those for i.i.d. Gaussian matrices but with faster computation.
A non-convex algorithm recovers low-rank matrices from one-bit labels efficiently.
problem Learning with one-bit labels in multi-label scenarios.
method Formulated as one-bit rank-one matrix sensing, developed an alternating power iteration algorithm.
result Achieves linear convergence and nearly optimal sampling complexity.
One-bit feedback suffices for a bandit problem's optimal strategy.
problem Optimal strategy for multi-armed bandit problem with limited feedback.
method Coding and decoding schemes for one-bit feedback to mimic full-reward feedback.
result Regret ratio approaches 1 with one-bit feedback.
New algorithm tackles batched stochastic linear bandits with 1-bit communication constraints.
problem Stochastic linear bandits with 1-bit communication constraints.
method Phased-elimination algorithms based on G-optimal designs and 1-bit mean estimation.
result Achieves near-optimal regret bounds for broad scaling regimes.
Wide residual networks achieve low error rates with single-bit weights.
problem Deploying deep neural networks on resource-constrained hardware with low memory.
method Binarizing weights using sign function and scaling factors, applying warm-restart learning rate schedule.
result Achieved error rates of 3.9% on CIFAR-10, 18.5% on CIFAR-100, and 26.0% on ImageNet with 1-bit-per-weight.
Moniqua improves SGD convergence with quantized communication.
problem Efficiently communicating in decentralized SGD with limited bandwidth.
method Modulo quantized communication in decentralized SGD.
result Moniqua converges at the same rate as full-precision communication with less bits.
Study 1-bit compressive sensing with generative models, improving recovery accuracy.
problem Accurately recover sparse vectors from binary measurements with generative models.
method Analyzes noiseless and noisy 1-bit measurements with i.i.d.~Gaussian and Lipschitz continuous generative priors, proving sample complexity bounds and stability properties.
result Proves sample complexity bounds and stability properties for 1-bit compressive sensing with generative models.
ADD embeds a 48-bit message into images, achieving high accuracy and speed.
problem Embedding high-fidelity messages into images to detect authenticity and source.
method Two-stage process: linear combination and addition of watermark to image, followed by decoding.
result ADD achieves 100% decoding accuracy for 48-bit watermarking, with minimal performance drop under various distortions.
This paper introduces new methods to improve 1-bit matrix completion by considering cluster effects.
problem Improving 1-bit matrix completion for clustered data.
method Group-Specific 1-bit Matrix Completion (GS1MC) and Cluster Developing Matrix Completion (CDMC).
result GS1MC and CDMC outperform existing methods in synthetic and real-world data.
End-to-end auto-encoder for neural image compression with variable bit rates.
problem Efficient image compression with variable bit rates.
method Block-based auto-encoder system with novel contributions.
result Incremental performance improvement of each contribution.
FleXOR trains fractional quantization for neural networks, improving accuracy and size.
problem Quantization limits to integer bits restricts compression and accuracy.
method Encryption algorithm with XOR gates for fractional bits during inference.
result FleXOR achieves high accuracy with fractional sub-1-bit weights.
Generalizes bits back coding for time-series models with latent Markov structures.
problem Efficiently compressing time-series data with latent Markov structures.
method Extends bits back coding to time-series models with latent Markov structures, including HMMs and LGSSMs.
result Effective for small scale models, promising for larger scale settings like video compression.
Paper studies distributed learning with limited communication bits, achieving optimal error exponents.
problem Distributed hypothesis testing with constant communication bits.
method Geometric approach in distribution spaces, encoding empirical distributions to transmission bits.
result Optimal achievable error exponents and coding schemes for various communication constraints.
WrapNet optimizes inference for low-resolution neural networks by using 8-bit additions.
problem Reducing multiplication complexity in low-resolution neural networks.
method Adapting neural networks to use low-resolution (8-bit) additions in accumulators, with a cyclic activation layer and overflow penalty regularizer.
result Achieves comparable classification accuracy to 32-bit counterparts using low-resolution additions.
Kernel Quantization improves CNN compression without sacrificing performance.
problem Efficiently compressing CNN models without significant performance loss.
method Quantizes convolution kernels as the unit, learning a codebook for low-bit indexes.
result Significant compression ratio achieved with minimal accuracy loss.
Minwise hashing is the standard technique in the context of search and databases for efficiently estimating set (e.g., high-dimensional 0/1 vector) similarities. Recently, b-bit minwise hashing was proposed which significantly improves upon the original minwise hashing in practice by storing only the lowest b bits of e…
Paper proposes a 1-bit quantization scheme for high-dimensional statistical estimation.
problem High-dimensional statistical estimation with limited data.
method Uniformly dithered 1-bit quantization for sparse covariance matrix estimation, sparse linear regression, and matrix completion.
result Near minimax rates in sub-Gaussian regime and improved rates in heavy-tailed regime.
Study quantile reward identification with 1-bit feedback constraints.
problem Best arm identification with quantile reward and 1-bit communication.
method Proposes an algorithm using noisy binary search for quantile reward estimation.
result Derives upper and lower bounds on sample complexity for 1-bit feedback.
Deep learning improves one-bit OFDM receiver performance.
problem One-bit quantization complicates accurate channel estimation and data detection in OFDM receivers.
method Developed deep neural networks for channel estimation and data detection, using a two-step training policy.
result Deep learning-based designs achieve lower BER than unquantized OFDM at moderate SNRs.
One-bit quantization improves inference speed for Random Features models.
problem Efficient inference on resource-constrained devices.
method Analysis of one-bit quantization in Random Features model.
result Asymptotically, quantizing weights except the last incurs no loss in generalization error.
The article applies Occam's Razor to non-parametric model building, minimizing the number of bits for data encoding.
problem Overlooking the role of model parameters in data encoding leads to inefficient probability density estimators.
method Extends bit counting to model parameters, providing a true measure of complexity for parametric models.
result Minimizing total bit requirement leads to smoother, more efficient probability density estimates and fewer relevant parameters.
Flexpoint improves deep learning training efficiency by using adaptive 16-bit format.
problem Training deep neural networks in low bit-width formats is challenging.
method Flexpoint uses a shared exponent dynamically adjusted to minimize overflows and maximize dynamic range.
result 16-bit Flexpoint tensors closely match 32-bit floating point in training deep networks without tuning.
A new method for 1-bit matrix completion that is faster and more accurate.
problem Estimating a low-rank matrix from binary observations.
method Majorization-Minimization Gauss-Newton (MMGN) method.
result MMGN outperforms existing methods in accuracy and speed.
This paper introduces a differentiable, scalable quantization method for neural networks.
problem Previous quantization methods lacked differentiability and scalability.
method The approach is differentiable and scalable, using bit-shifting and logarithmic quantization.
result The method achieves comparable accuracy to state-of-the-art approaches with less training time and lower inference cost.
Predict social trust using 1-bit measurements and non-uniform sampling.
problem Predict social trust in social networks with sign measurements and non-uniform sampling.
method Propose a 1-bit max-norm constrained formulation and use a projected gradient decent algorithm.
result Demonstrated superior performance on benchmark datasets.