APoT quantization improves neural network efficiency and accuracy.
problem Efficiently quantizing weights and activations in neural networks.
method Constraining quantization levels as sums of Powers-of-Two terms, applying reparameterization, and weight normalization.
result 4-bit quantized ResNet-50 achieves 76.6% top-1 accuracy, 22% computational cost reduction.
New method achieves faster calibration without randomization.
problem Calibrating probabilistic forecasts in adversarial settings.
method Using interval forecasts and the power of two choices.
result Achieves O(1/T) calibration error rate without randomization. Efficiently detects anomalies in videos with reduced computation.
problem Anomaly detection in videos for robust and efficient systems.
method Powers-of-two weights and denoising for CNN simplification and robustness.
result 10% faster detection with comparable accuracy and robustness.
Let K be a connected finite complex. This paper studies the problem of whether one can attach a cell to some iterated suspension S^j K so that the resulting space satisfies Poincare duality. When this is possible, we say that S^j K is a spine. We introduce the notion of quadratic self duality and show that if K is quad…
We show that the only rational homology spheres which can admit almost complex structures occur in dimensions two and six. Moreover, we provide infinitely many examples of six-dimensional rational homology spheres which admit almost complex structures, and infinitely many which do not. We then show that if a closed alm…
The use of low-precision fixed-point arithmetic along with stochastic rounding has been proposed as a promising alternative to the commonly used 32-bit floating point arithmetic to enhance training neural networks training in terms of performance and energy efficiency. In the first part of this paper, the behaviour of …
Focused quantization reduces CNN models by 18x while maintaining accuracy.
problem Memory and compute resource constraints in deploying CNNs on constrained devices.
method Focused quantization, exploiting weight distributions after pruning, dynamically discovers optimal numerical representations.
result Achieved a 18.08x compression ratio with only 0.24% loss in accuracy in ResNet-50.
In this paper, we study how the mean shift algorithm can be used to denoise a dataset. We introduce a new framework to analyze the mean shift algorithm as a denoising approach by viewing the algorithm as an operator on a distribution function. We investigate how the mean shift algorithm changes the distribution and sho…
Randomly initialized ReLU networks of depth two can approximate smooth functions well.
problem Approximation power of two-layer networks of random ReLUs.
method Harmonic analysis and ridgelet representation theory for upper bounds, dimensionality arguments for lower bounds.
result Near-matching upper and lower bounds for L2-approximation and Sobolev norms. Let Gamma be a group generated by two positive multi-twists. We give some sufficient conditions for Gamma to be free or have no `unexpectedly reducible' elements. For a group Gamma generated by two Dehn twists, we classify the elements in Gamma which are multi-twists. As a consequence we are able to list all the lanter…
We consider derivative-free algorithms for stochastic and non-stochastic convex optimization problems that use only function values rather than gradients. Focusing on non-asymptotic bounds on convergence rates, we show that if pairs of function values are available, algorithms for d-dimensional optimization that use …
Study shows Transformer and Neural GPU are Turing complete without external memory.
problem Exploring computational power of modern neural network architectures.
method Analyzing computational properties of Transformer and Neural GPU.
result Transformer and Neural GPU are Turing complete without external memory.
HMQ improves quantization for edge devices with mixed precision.
problem Efficient quantization for edge devices with uniform, power-of-two thresholds.
method Introduces HMQ, a mixed precision quantization block that repurposes Gumbel-Softmax for searching over quantization schemes.
result Achieves competitive and state-of-the-art results on ImageNet despite restrictions.
Paper introduces LUT-Q for efficient neural network quantization.
problem Reducing memory and computational requirements for deep neural networks.
method LUT-Q method learns a dictionary to assign weights to values, allowing for various constraints.
result LUT-Q consistently achieves better performance than other methods with the same quantization bitwidth.
Generative adversarial networks (GANs) are innovative techniques for learning generative models of complex data distributions from samples. Despite remarkable recent improvements in generating realistic images, one of their major shortcomings is the fact that in practice, they tend to produce samples with little divers…
Legendrian knot representations linked to colored Kauffman polynomial.
problem Relating Legendrian knot representations to colored Kauffman polynomial.
method Introducing ungraded n-colored ruling polynomial and relating it to the n-colored Kauffman polynomial. result Ungraded representation numbers of Legendrian knot DG-algebra agree with n-colored Kauffman polynomial specialization. Paper presents CeNN quantization for efficient CPS applications.
problem Efficient processing for CPS applications, especially in telemedicine and ADAS.
method Incremental quantization of CeNNs with various strategies.
result Achieved up to 7.8x speedup with no performance loss.
DJPQ optimizes neural network pruning and quantization for hardware efficiency.
problem Efficiently compress neural networks for hardware inference.
method Joint gradient-based optimization of pruning and quantization into a differentiable loss function.
result Significant reduction in Bit-Operations (BOPs) with minimal accuracy loss.
Bayesian Bits unifies quantization and pruning through gradient optimization.
problem Joint mixed precision quantization and pruning for efficient neural networks.
method Gradient-based optimization with a novel bit width decomposition and learnable stochastic gates.
result Bayesian Bits achieves better accuracy vs. efficiency trade-off compared to static bit width networks.
We consider the problem of deep neural net compression by quantization: given a large, reference net, we want to quantize its real-valued weights using a codebook with K entries so that the training loss of the quantized net is minimal. The codebook can be optimally learned jointly with the net, or fixed, as for bina…
Paper details Hilbert-curve for high-performance data mining.
problem Efficiently mapping multi-dimensional data to one dimension.
method Defines Hilbert-curve using finite automaton and context-free grammar.
result Cache-oblivious algorithms for matrix operations and clustering.
MAGDiff detects data shifts in neural networks without retraining.
problem Neural networks' sensitivity to data distribution shifts.
method Extracts MAGDiff representations from neural networks to detect shifts.
result MAGDiff representations improve data set shift detection.
GTC improves neural network compression and accuracy without multiplications.
problem Balancing accuracy and computational complexity in deep neural networks.
method Generalizes ternary connect to allow arbitrary levels and integer powers of two, learning optimal levels and weights end-to-end.
result GTC achieves comparable accuracy to binary networks with superior compression and hardware benefits.
A new iterative low complexity algorithm has been presented for computing the Walsh-Hadamard transform (WHT) of an N dimensional signal with a K-sparse WHT, where N is a power of two and K=O(Nα), scales sub-linearly in N for some 0<α<1. Assuming a random support model for the non-zero transform domain…
RED-2400 is a public benchmark of trading events from a Solana exchange, labeled by algorithmic rejection.
problem Analyzing algorithmically-rejected trading events for insights into market dynamics.
method Public dataset of 6,660 algorithmically-rejected trading events, linked to post-rejection price and liquidity trajectories.
result First window of a planned series of datasets extending the time horizon and enabling regime-stratified analysis.
A new compression technique reduces communication costs in distributed deep learning.
problem Communication bottlenecks in distributed deep learning training.
method Natural Compression (NC) and Natural Dithering.
result Natural Compression increases communication savings by 3-4 times with negligible convergence speed impact.
Computes volumes of metric maps on surfaces, linking to Weil-Petersson volumes.
problem Computing volumes of specific metric maps on surfaces.
method Using recent results on discrete maps with irreducibility constraints, computes volumes as homogeneous polynomials.
result Identifies volumes as homogeneous polynomials and satisfies string and dilaton equations.
This paper proposes a learning framework for n-bit quantized neural networks that improves accuracy and speed on FPGAs.
problem Efficiently implementing quantized neural networks on FPGAs to maintain accuracy and speed.
method A novel learning framework for n-bit QNNs, constrained weights, reconstructed gradient function, n-BQ-NN structure, and SVPE array.
result Quantized models achieve almost the same accuracy as full-precision models and outperform typical low-precision QNNs.