HSRL learns network embeddings capturing both local and global topology.
problem Capturing both local and global topological information in network analysis.
method HSRL recursively compresses networks into smaller ones, then learns embeddings using existing methods.
result HSRL outperforms state-of-the-art methods in link prediction.
NoNN compresses deep networks into distributed IoT modules with minimal communication.
problem Memory and communication constraints in IoT devices for deep learning inference.
method NoNN compresses a large pretrained network into disjoint, highly-compressed student modules, optimizing for memory and communication.
result NoNN achieves higher accuracy than baselines and similar to the teacher model with minimal communication.
This paper optimizes AI inference on edge devices with reduced communication and computation costs.
problem Efficiently performing AI inference on resource-constrained edge devices with reduced communication and computation costs.
method A three-step framework for effective inference: model split point selection, communication-aware model compression, and task-oriented encoding of intermediate features.
result Our proposed framework achieves a better trade-off and significantly reduces inference latency compared to baseline methods.
New algorithms reduce communication costs in collaborative learning.
problem Reducing communication costs in collaborative learning.
method Distributed boosting and adaptation to classification noise.
result Communication-efficient algorithms for collaborative PAC learning robust to noise.
EdgeAI aims to deploy deep learning on IoT devices.
problem Deep learning's high computational demands on IoT devices.
method Addressing data-independent deployment and communication-aware distributed inference.
result New directions to enable deep learning on IoT devices.
The paper addresses challenges in edge deep learning for IoT, proposing new directions.
problem Challenges in large-scale deep learning adoption for IoT devices.
method Unified view targeting three research directions: federated learning, data-independent deployment, and communication-aware inference.
result A network-centric approach is needed for edge intelligence.
ECG improves graph clustering and resolves resolution limit issues.
problem Graph clustering resolution limit issue.
method ECG uses consensus clustering to improve graph clustering.
result ECG alleviates the resolution limit issue and improves partition stability.
Proposes CCCVAE for better single-cell clustering with cell-cell communication.
problem Improving single-cell RNA sequencing clustering by incorporating cell-cell communication.
method Integrates cell-cell communication into a variational autoencoder framework.
result Empirical results show CCCVAE outperforms standard VAEs in clustering performance.
Flexible framework compresses models using LC algorithm.
problem Efficiently compressing neural networks for resource constraints.
method Decouples learning and compression steps with alternating L and C phases.
result Compressed models maintain performance and accuracy.
Reduces multiclass and regression compression schemes to binary ones.
problem Developing efficient learning algorithms for multiclass and regression problems.
method Reduces sample compression schemes for binary classes to multiclass and regression settings.
result Establishes new compression schemes for multiclass and regression problems.
Proposes a link between randomness and compression in deep learning.
problem Improving efficiency in deep learning training.
method Introduces a novel tomographic compression framework called Dual Tomographic Compression (DTC).
result Demonstrates high correlation between learning performance and Gibbs entropy over compression ratios.
A new method classifies images using a compressed summary.
problem Classifying images efficiently.
method Compressive learning with a novel sketch function.
result Improved image classification with less data.
Unified framework for non-compressed networks from compression bounds.
problem Generalization of large deep neural networks with huge model size.
method Unified framework converting compression based bounds to non-compressed networks.
result Unified data-dependent generalization error bound with tighter evaluation.
Paper introduces a new adaptive gradient method with gradient compression for distributed training.
problem Communication overhead in distributed machine learning systems.
method Adaptive gradient method with gradient compression, scalable system BytePS-Compress.
result Convergence rate of O ( 1 / T ) \mathcal{O}(1/\sqrt{T}) O ( 1/ T ) for non-convex problems. Galen algorithm compresses neural networks for specific hardware with reduced latency.
problem Finding optimal compression policies for neural networks on specific hardware.
method Reinforcement learning using pruning and quantization to optimize inference latency.
result Compressed ResNet18 for ARM processor reduced inference latency by 80%.
The (isothermic) compressibility of lattice knots can be examined as a model of the effects of topology and geometry on the compressibility of ring polymers. In this paper, the compressibility of minimal length lattice knots in the simple cubic, face centered cubic and body centered cubic lattices are determined. Our r…
Paper compresses neural network weight-updates for image artifacts removal.
problem Efficiently compressing neural network weight-updates for image artifacts removal.
method Fine-tuning a pre-trained artifact removal network on target data with a compression objective that encourages sparse and quantized weight-updates.
result Achieves reconstruction quality comparable to traditional codecs at comparable bitrates.
HiLLoC compresses large images losslessly using VAEs.
problem Lossless compression of large color photographs.
method Fully convolutional VAE models trained on ImageNet are applied to lossless compression.
result Achieves state-of-the-art compression for full-size ImageNet images.
Neural NCD reveals LLMs don't compress well for classification.
problem The disconnect between compression and classification in neural networks.
method Developed Neural NCD to compare LLMs to classic algorithms, finding classification accuracy not correlated with compression rate.
result Classification accuracy is not predictable by compression rate alone, challenging current understanding.
BDC compresses both sample size and dimensionality of large datasets.
problem Large datasets in both sample size and dimensionality.
method Two-stage framework using Decoded MMD, Reconstruction MMD, and Encoded MMD.
result BDC achieves comparable or superior performance with lower cost and higher compression rates.
This paper compresses RNNs for IoT devices by 15-38x using Kronecker products.
problem Resource constraints on IoT devices make RNNs difficult to deploy.
method Kronecker product (KP) for compressing RNN layers by 15-38x with minimal accuracy loss.
result Kronecker product can compress RNNs by 50x when quantized to 8-bits.
DoCoFL compresses model updates for cross-device federated learning.
problem Downlink compression for cross-device federated learning where clients may appear only once.
method Proposes DoCoFL framework for downlink compression in cross-device federated learning.
result Significant bi-directional bandwidth reduction with competitive accuracy.
Compressing neural nets is an active research problem, given the large size of state-of-the-art nets for tasks such as object recognition, and the computational limits imposed by mobile devices. We give a general formulation of model compression as constrained optimization. This includes many types of compression: quan…
Paper presents a new video compression method using autoencoders.
problem Efficient video compression with reduced quality loss.
method 3D autoencoder with discrete latent space and autoregressive prior trained jointly.
result Method outperforms state-of-the-art learned video compression networks.
New taxonomy and evaluation of neural network compression methods.
problem Efficiency of deep neural networks in real-world applications.
method Categorization and evaluation of tensor factorization and probabilistic compression methods.
result SVD and probabilistic compression methods are complementary and give the best results.
Proposes a method to train neural networks directly on compressed text data.
problem Training neural networks on compressed text data without decompression.
method Introduces composer modules to encode symbols from grammar compression rules into vector representations.
result Demonstrates that the proposed method can achieve both memory and computational efficiency while maintaining moderate performance.
Unified framework improves model compression while maintaining robustness.
problem Achieving high compression ratios without sacrificing adversarial robustness.
method Adversarially Trained Model Compression (ATMC) framework integrating multiple compression techniques.
result ATMC achieves better trade-off between model size, accuracy, and robustness.
Study shows LLC correlates with neural network compressibility.
problem Evaluating limits of neural network compression.
method Extended minimum description length principle using singular learning theory.
result Complexity estimates based on LLC are linearly correlated with compressibility.
Paper proposes SCALLION and SCAFCOM for compressed FL with reduced communication.
problem Reducing communication overhead in Federated Learning with data heterogeneity and partial participation.
method Revisit and simplify stochastic controlled averaging, proposing SCALLION and SCAFCOM for unbiased and biased compression.
result SCALLION and SCAFCOM outperform existing methods in communication and computation complexities.
Gradient descent with compressed iterates reduces model size for federated learning.
problem Efficiently compressing model sizes in federated learning.
method Gradient descent with iterates compressed using lossy randomized techniques.
result Improves model compression and aggregation in federated learning.
New uncertainty principle limits compression in distributed learning, suggesting optimal methods.
problem Minimizing communication cost while maintaining message quality in distributed learning.
method Formalized uncertainty principle for compression operators, introduced Kashin compression.
result Explicit formula for Kashin compression's variance bound, dimension independent.
Proposes a new linearity-based neural network compression method.
problem Reduction of neural network model size while maintaining accuracy.
method Integrates linearity-based intuition with ReLU activation functions to merge layers.
result Achieves up to 75% reduction in model size without loss of accuracy.
Paper introduces compressibility loss for learning sparse neural network weights.
problem Learning highly compressible neural network weights.
method Applying a compressibility loss to minimize the negated sparsity of the signal.
result At critical points, weight vectors are ternary signals with a sparsity directly related to the objective value.
Model compression reduces generalization error and increases empirical risk, potentially improving population risk.
problem Improving population risk with model compression.
method Information-theoretic analysis and rate distortion theory.
result Model compression can improve population risk if the decrease in generalization error exceeds the increase in empirical risk.
Paper compresses RNNs for resource-constrained devices.
problem Difficulty deploying RNNs on resource-constrained devices.
method Uses Kronecker product (KP) to compress RNN layers.
result KP compresses RNN layers by 16-38x with minimal accuracy loss.
Randomized matrix compression techniques, such as the Johnson-Lindenstrauss transform, have emerged as an effective and practical way for solving large-scale problems efficiently. With a focus on computational efficiency, however, forsaking solutions quality and accuracy becomes the trade-off. In this paper, we investi…
A new framework maximizes influence spread in social networks by accounting for inter-community diffusion.
problem Real-world social networks have inter-community influence that is often overlooked in community-based IM approaches.
method Community-IM++ uses a heuristic based on community-based diffusion degree and progressive budgeting to model and prioritize cross-community diffusion.
result Community-IM++ achieves near-greedy influence spread at up to 100 times lower runtime than existing methods.
Can textual data be compressed intelligently without losing accuracy in evaluating sentiment? In this study, we propose a novel evolutionary compression algorithm, PARSEC (PARts-of-Speech for sEntiment Compression), which makes use of Parts-of-Speech tags to compress text in a way that sacrifices minimal classification…
Unified framework compresses GANs up to 47x with minimal quality loss.
problem High parameter complexity of GANs for resource-constrained devices.
method Unified optimization framework combining model distillation, channel pruning, and quantization.
result 47x compression of CartoonGAN with minimal quality degradation.
Dream Distillation compresses models without data, achieving high accuracy.
problem Model compression without real data.
method Data-independent model compression framework.
result Achieves 88.5% accuracy on CIFAR-10 test set.
New compression theory justifies model pruning for neural networks.
problem Improving neural network performance with reduced model size.
method Information-theoretic rate-distortion theory applied to NN compression.
result Pruning improves model performance on CIFAR-10 and ImageNet datasets.
SHVC improves image compression with fewer parameters.
problem Challenges in VAE compression, especially with bits-back coding.
method Introduces autoregressive sub-pixel convolution and autoregressive initial bits.
result Achieves state-of-the-art compression performance with fewer model parameters.
Model compression has been introduced to reduce the required hardware resources while maintaining the model accuracy. Lots of techniques for model compression, such as pruning, quantization, and low-rank approximation, have been suggested along with different inference implementation characteristics. Adopting model com…
New methods show deep networks can compress info without saturating activations.
problem Understanding how neural networks generalize and compress information.
method Adaptive mutual information estimation techniques for neural networks.
result Compression occurs in networks with non-saturating activation functions.
COIN++ compresses multiple data types efficiently.
problem Handling diverse data modalities in neural compression.
method Implicit neural representations and modulations quantization.
result Significant compression gains with reduced encoding time.
PCRs compress data for deep learning, reducing training time.
problem Efficiently training deep learning models over large datasets.
method Combining progressive compression with an efficient storage layout.
result PCRs can tolerate up to 50% compression without significantly affecting training accuracy.
Paper introduces adversarial lossy compression for video artifacts reduction.
problem Unpleasant reconstruction artifacts in standard video coding schemes at low bit-rates.
method Adversarial lossy video compression model minimizing an adversarial distortion objective.
result Reduction of perceptual artifacts and detail reconstruction under extreme compression.
LLMs compress financial texts, but distort decision-making.
problem LLMs compress financial texts, altering decision-making.
method Analyzed two diagnostic patterns: decontextualization and model dependency. Proposed Agentic Context Compression.
result LLM-compressed financial texts alter decision-making.