arXiv research

Model compresses event-like contexts using gated surprise signals.

A new compressed sensing system speeds up PSTE detection.

In this paper, we propose and analyze SPARQ-SGD, which is an event-triggered and compressed algorithm for decentralized training of large-scale machine learning models. Each node can locally compute a condition (event) which triggers a communication where quantized and sparsified local model parameters are sent. In SPA…

An increasing amount of civil engineering applications are utilising data acquired from infrastructure instrumented with sensing devices. This data has an important role in monitoring the response of these structures to excitation, and evaluating structural health. In this paper we seek to monitor pedestrian-events (su…

In this paper, we consider linear state-space models with compressible innovations and convergent transition matrices in order to model spatiotemporally sparse transient events. We perform parameter and state estimation using a dynamic compressed sensing framework and develop an efficient solution consisting of two nes…

This work optimizes DNN inference for energy-harvesting devices by compressing and selectively executing neural network exits.

Footfall based biometric system is perhaps the only person identification technique which does not hinder the natural movement of an individual. This is a clear edge over all other biometric systems which require a formidable amount of human intervention and encroach upon an individual's privacy to some extent or the o…

New guarantees for asymmetric sketching in compressive learning.

We propose a graph spectral representation of time series data that 1) is parsimoniously encoded to user-demanded resolution; 2) is unsupervised and performant in data-constrained scenarios; 3) captures event and event-transition structure within the time series; and 4) has near-linear computational complexity in both …

We propose a simple but efficient method termed Guided Learning for weakly-labeled semi-supervised sound event detection (SED). There are two sub-targets implied in weakly-labeled SED: audio tagging and boundary detection. Instead of designing a single model by considering a trade-off between the two sub-targets, we de…

Quantum computing speeds up analysis of financial stochastic processes.

Spatio-temporal data compression method reduces memory usage.

Kolmogorov-Arnold network improves GW catalog posterior construction.

This paper improves understanding of knowledge distillation and its effects.

Flexible framework compresses models using LC algorithm.

Reduces multiclass and regression compression schemes to binary ones.

Proposes a link between randomness and compression in deep learning.

Paper introduces a new adaptive gradient method with gradient compression for distributed training.

Galen algorithm compresses neural networks for specific hardware with reduced latency.

The (isothermic) compressibility of lattice knots can be examined as a model of the effects of topology and geometry on the compressibility of ring polymers. In this paper, the compressibility of minimal length lattice knots in the simple cubic, face centered cubic and body centered cubic lattices are determined. Our r…

Neural NCD reveals LLMs don't compress well for classification.

BDC compresses both sample size and dimensionality of large datasets.

DoCoFL compresses model updates for cross-device federated learning.

Compressing neural nets is an active research problem, given the large size of state-of-the-art nets for tasks such as object recognition, and the computational limits imposed by mobile devices. We give a general formulation of model compression as constrained optimization. This includes many types of compression: quan…

Proposes a method to train neural networks directly on compressed text data.

Study shows LLC correlates with neural network compressibility.

Paper proposes SCALLION and SCAFCOM for compressed FL with reduced communication.

New uncertainty principle limits compression in distributed learning, suggesting optimal methods.

One of the biggest issues in deep learning theory is the generalization ability of networks with huge model size. The classical learning theory suggests that overparameterized models cause overfitting. However, practically used large deep models avoid overfitting, which is not well explained by the classical approaches…

Recurrent Neural Networks (RNN) can be difficult to deploy on resource constrained devices due to their size.As a result, there is a need for compression techniques that can significantly compress RNNs without negatively impacting task accuracy. This paper introduces a method to compress RNNs for resource constrained e…

Proposes a new linearity-based neural network compression method.

Randomized matrix compression techniques, such as the Johnson-Lindenstrauss transform, have emerged as an effective and practical way for solving large-scale problems efficiently. With a focus on computational efficiency, however, forsaking solutions quality and accuracy becomes the trade-off. In this paper, we investi…

Can textual data be compressed intelligently without losing accuracy in evaluating sentiment? In this study, we propose a novel evolutionary compression algorithm, PARSEC (PARts-of-Speech for sEntiment Compression), which makes use of Parts-of-Speech tags to compress text in a way that sacrifices minimal classification…

Unified framework compresses GANs up to 47x with minimal quality loss.

New compression theory justifies model pruning for neural networks.

SHVC improves image compression with fewer parameters.

Model compression has been introduced to reduce the required hardware resources while maintaining the model accuracy. Lots of techniques for model compression, such as pruning, quantization, and low-rank approximation, have been suggested along with different inference implementation characteristics. Adopting model com…

Compressive learning is a framework where (so far unsupervised) learning tasks use not the entire dataset but a compressed summary (sketch) of it. We propose a compressive learning classification method, and a novel sketch function for images.

COIN++ compresses multiple data types efficiently.

Paper introduces adversarial lossy compression for video artifacts reduction.

LLMs compress financial texts, but distort decision-making.

A new method, REC, compresses images by encoding their latent representations efficiently.

Wavelets help compress neural networks efficiently.

This work compresses sequences by treating them as continuous-time processes, enabling efficient discretization.

Auto-Compressing Subset Pruning reduces model size for faster inference.

DP-Net uses dynamic programming for efficient deep neural network compression.

LEAD algorithm speeds up decentralized optimization with compression.

We show that model compression can improve the population risk of a pre-trained model, by studying the tradeoff between the decrease in the generalization error and the increase in the empirical risk with model compression. We first prove that model compression reduces an information-theoretic bound on the generalizati…