COIN++ compresses multiple data types efficiently.
problem Handling diverse data modalities in neural compression.
method Implicit neural representations and modulations quantization.
result Significant compression gains with reduced encoding time.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Data Compression”
Can textual data be compressed intelligently without losing accuracy in evaluating sentiment? In this study, we propose a novel evolutionary compression algorithm, PARSEC (PARts-of-Speech for sEntiment Compression), which makes use of Parts-of-Speech tags to compress text in a way that sacrifices minimal classification…
Model compression is eminently suited for deploying deep learning on IoT-devices. However, existing model compression techniques rely on access to the original or some alternate dataset. In this paper, we address the model compression problem when no real data is available, e.g., when data is private. To this end, we p…
Paper proposes SCALLION and SCAFCOM for compressed FL with reduced communication.
problem Reducing communication overhead in Federated Learning with data heterogeneity and partial participation.
method Revisit and simplify stochastic controlled averaging, proposing SCALLION and SCAFCOM for unbiased and biased compression.
result SCALLION and SCAFCOM outperform existing methods in communication and computation complexities.
L21 SNF compresses mixed-sign data robustly.
problem Compression of mixed-sign data with high fidelity.
method Regularized L21 Semi-NonNegative Matrix Factorization (L21 SNF).
result Rigorous proof of convergence and use-case advantages demonstrated.
A new method compresses conditional distributions of labelled data.
problem No existing method directly compresses the conditional distribution of labelled data.
method Introduce Average Maximum Conditional Mean Discrepancy (AMCMD), derive a closed form estimator, and extend Kernel Herding (KH) to Average Conditional Kernel Herding (ACKH).
result Directly compressing conditional distributions outperforms joint distribution compression and greedy selection.
Neural networks using numerous text data have been successfully applied to a variety of tasks. While massive text data is usually compressed using techniques such as grammar compression, almost all of the previous machine learning methods assume already decompressed sequence data as their input. In this paper, we propo…
Deep learning accelerators efficiently train over vast and growing amounts of data, placing a newfound burden on commodity networks and storage devices. A common approach to conserve bandwidth involves resizing or compressing data prior to training. We introduce Progressive Compressed Records (PCRs), a data format that…
Data compression is a popular technique for improving the efficiency of data processing workloads such as SQL queries and more recently, machine learning (ML) with classical batch gradient methods. But the efficacy of such ideas for mini-batch stochastic gradient descent (MGD), arguably the workhorse algorithm of moder…
Unified strategy for efficient data compression and model estimation.
problem Limited interactive exploration and data interaction in linear model development and deployment.
method Conditionally sufficient statistics for optimal data compression and estimation of linear models.
result Linear models can be estimated from compressed data without loss of parameters or covariances.
This paper speeds up SVC clustering by compressing data while preserving key properties.
problem Efficiently clustering large-scale real-world data sets.
method Spectrum-preserving data compression for fast support vector clustering.
result Achieved 100X and 115X speedups on real-world data sets while maintaining clustering quality.
BDC compresses both sample size and dimensionality of large datasets.
problem Large datasets in both sample size and dimensionality.
method Two-stage framework using Decoded MMD, Reconstruction MMD, and Encoded MMD.
result BDC achieves comparable or superior performance with lower cost and higher compression rates.
One of the biggest issues in deep learning theory is the generalization ability of networks with huge model size. The classical learning theory suggests that overparameterized models cause overfitting. However, practically used large deep models avoid overfitting, which is not well explained by the classical approaches…
Compressed Factorization: Fast and Accurate Low-Rank Factorization of Compressively-Sensed Datacs.LG
What learning algorithms can be run directly on compressively-sensed data? In this work, we consider the question of accurately and efficiently computing low-rank matrix or tensor factorizations given data compressed via random projections. We examine the approach of first performing factorization in the compressed dom…
Compressed sensing in MRI enables high subsampling factors while maintaining diagnostic image quality. This technique enables shortened scan durations and/or improved image resolution. Further, compressed sensing can increase the diagnostic information and value from each scan performed. Overall, compressed sensing has…
LEAD algorithm speeds up decentralized optimization with compression.
problem Slow convergence and stability issues in decentralized optimization with compression.
method Proposes the first linearly convergent decentralized algorithm with compression.
result First consensus error bound for coupled dynamics of primal and dual updates.
New framework compresses and recovers scientific data efficiently.
problem Efficiently managing and recovering from large scientific datasets.
method Grounded in learning exponential families, preserves uncertainty and supports trade-offs.
result Preserves physical features and quantities of interest in compressed representations.
Distributed sensors compress and send features to a fusion center for linear regression.
problem Efficiently compress and transmit features from distributed sensors to a fusion center under varying communication constraints.
method Designs a distributed and adaptive feature compression scheme using optimal quantizers and simple adaptive strategies.
result Demonstrates improved inference performance through simulated experiments.
Motivated by the ever-increasing demands for limited communication bandwidth and low-power consumption, we propose a new methodology, named joint Variational Autoencoders with Bernoulli mixture models (VAB), for performing clustering in the compressed data domain. The idea is to reduce the data dimension by Variational…
Real-world data typically contain repeated and periodic patterns. This suggests that they can be effectively represented and compressed using only a few coefficients of an appropriate basis (e.g., Fourier, Wavelets, etc.). However, distance estimation when the data are represented using different sets of coefficients i…
Autoencoders fail to capture sparse structure in 1-bit data compression.
problem Proving the performance of shallow autoencoders on sparse data compression.
method Gradient descent analysis and approximate message passing.
result Gradient descent minimizer for sparse data is the identity (up to permutation) above critical sparsity.
Compressed Regressionstat.ML
Recent research has studied the role of sparsity in high dimensional regression and signal reconstruction, establishing theoretical limits for recovering sparse models from sparse data. This line of work shows that -regularized least squares regression can accurately estimate a sparse linear model from nois…
DACE estimates covariance from compressed data, improving accuracy.
problem Estimating covariance from large, distributed data.
method Data-aware weighted sampling for unbiased estimation.
result DACE provides more accurate covariance estimation under compression.
Deep Neural Networks (DNNs) have recently been achieving state-of-the-art performance on a variety of computer vision related tasks. However, their computational cost limits their ability to be implemented in embedded systems with restricted resources or strict latency constraints. Model compression has therefore been …
New method compresses large sample data for faster discriminant analysis.
problem Large sample sizes in discriminant analysis increase computational burden.
method Proposes a new compression approach for reducing training samples.
result Significant computational gains and superior predictive ability compared to random sub-sampling.
Designing a data sharing mechanism without sacrificing too much privacy can be considered as a game between data holders and malicious attackers. This paper describes a compressive adversarial privacy framework that captures the trade-off between the data privacy and utility. We characterize the optimal data releasing …
A new method learns graph compression from data.
problem Graphs lack ordering, making conventional compression algorithms ineffective.
method Partition and Code framework: decompose, learn, encode.
result PnC achieves compression gains that grow with graph size.
Principal component analysis, dictionary learning, and auto-encoders are all unsupervised methods for learning representations from a large amount of training data. In all these methods, the higher the dimensions of the input data, the longer it takes to learn. We introduce a class of neural networks, termed RandNet, f…
Lossless compression methods shorten the expected representation size of data without loss of information, using a statistical model. Flow-based models are attractive in this setting because they admit exact likelihood optimization, which is equivalent to minimizing the expected number of bits per message. However, con…
New video compression method outperforms traditional approaches.
problem Efficient video compression in low latency mode.
method Feedback Recurrent Autoencoder network architecture.
result State of the art MS-SSIM/rate performance on UVG dataset.
"Lossless" Compression of Deep Neural Networks: A High-dimensional Neural Tangent Kernel Approachstat.ML
Lossless compression of deep neural networks using NTK and RMT.
problem Compressing large-scale deep neural networks for low-power devices.
method High-dimensional neural tangent kernel approach.
result Asymptotic spectral equivalence between NTK matrices of wide DNNs enables lossless compression.
This paper proposes a new method for efficient data compression using Bayesian neural networks.
problem Efficient compression of data represented as functions mapping coordinates to signal values.
method Overfitting variational Bayesian neural networks to the data and compressing an approximate posterior weight sample using relative entropy coding.
result Our method achieves strong performance on image and audio compression while retaining simplicity.
Paper proposes a compression principle for neural networks using Bayesian optimization.
problem Finding methods for making generalizable predictions in machine learning.
method Compression principle and Bayesian optimization approach.
result Optimal predictive models minimize total compressed message length of data and model definition.
Lossy compression of statistical data using quantum annealing.
problem Efficiently compressing statistical floating-point data.
method Representation learning with binary variables, classical optimization of basis vectors, quantum annealing for coefficients, bias correction.
result Quantum annealing shows promising results with 3.5x better compression than neural-network autoencoders.
This paper develops a method to estimate the rate-distortion function for general data sources.
problem Estimating the rate-distortion function for general data sources.
method Develops an algorithm for sandwiching the R-D function of a general (not necessarily discrete) source using i.i.d. data samples.
result Estimates R-D sandwich bounds for various data sources, including natural images, indicating potential for improving compression methods.
The usage of deep generative models for image compression has led to impressive performance gains over classical codecs while neural video compression is still in its infancy. Here, we propose an end-to-end, deep generative modeling approach to compress temporal sequences with a focus on video. Our approach builds upon…
We present a new similarity measure based on information theoretic measures which is superior than Normalized Compression Distance for clustering problems and inherits the useful properties of conditional Kolmogorov complexity. We show that Normalized Compression Dictionary Size and Normalized Compression Dictionary En…
Data compression speeds up machine learning loss calculations.
problem Computational demand in calculating mean squared error for large datasets.
method Use rank-1 lattices to compress data, assigning weights based on original data and responses.
result Our QMC data compression algorithms can lead to arbitrary high convergence rates for smooth functions.
New compression theory justifies model pruning for neural networks.
problem Improving neural network performance with reduced model size.
method Information-theoretic rate-distortion theory applied to NN compression.
result Pruning improves model performance on CIFAR-10 and ImageNet datasets.
Wavelets help compress neural networks efficiently.
problem Efficiently compressing linear layers in neural networks.
method Learnable wavelet transforms to compress RNNs.
result Wavelet compressed RNNs have fewer parameters and perform competitively.
Diffusion Generative Models Meet Compressed Sensing, with Applications to Imaging and Financestat.ML
CSDM integrates compressed sensing into diffusion models for faster data generation.
problem Efficiently generating synthetic data in high-dimensional spaces.
method Integrating compressed sensing into diffusion models (CSDM) to reduce dimensionality and accelerate inference.
result Achieves provably faster convergence and better latent space dimension selection.
New method preserves privacy while improving machine learning accuracy.
problem Privacy-preserving machine learning for daily data.
method Compressive Privacy and multi-kernel method.
result Improved utility classification accuracy with privacy preservation.
Compressed LLM embeddings improve noisy regression tasks without overfitting.
problem Noisy regression tasks with high signal-to-noise ratios.
method Comparison of embedding compression techniques using autoencoder hidden representations.
result Compression improves performance on noisy tasks like financial return prediction.
Data-independent pruning method reduces neural network size with accuracy guarantees.
problem Limited computational and memory resources for neural networks.
method Structured pruning using coresets.
result First efficient algorithm with worst-case guarantees on compression and accuracy.
A new deep learning framework for efficient IoT data compression and inference.
problem Limited bandwidth and power resources in IoT sensors.
method Co-designed deep learning framework that maximizes sensing goal accuracy.
result Superior performance compared to benchmark models.
In this paper, we present a novel approach for fine-tuning a decoder-side neural network in the context of image compression, such that the weight-updates are better compressible. At encoder side, we fine-tune a pre-trained artifact removal network on target data by using a compression objective applied on the weight-u…
This work compresses sequences by treating them as continuous-time processes, enabling efficient discretization.
problem Efficient compression of sequences, especially with deep learning models that scale with sequence length.
method Treat sequences as continuous-time processes, learn efficient discretization, and decode at different time intervals.
result Automatic bit rate reductions in video and motion capture sequences using learned discretization.
Telemonitoring of electroencephalogram (EEG) through wireless body-area networks is an evolving direction in personalized medicine. Among various constraints in designing such a system, three important constraints are energy consumption, data compression, and device cost. Conventional data compression methodologies, al…