This paper automates deep model compression using reinforcement learning.
problem Efficiently compressing deep neural networks without sacrificing accuracy.
method Reinforcement learning-based actor-critic structure for automated compression.
result 4-fold reduction in FLOP with 2.8% higher accuracy for VGG-16.
This work automates CNN model compression for mobile devices.
problem Deploying trained CNNs to mobile devices requires balancing speed, memory, and accuracy.
method Reinforcement learning with architecture search and knowledge distillation.
result An automated model compression algorithm improves the trade-off between speed, memory, and accuracy.
Automated tool reduces FPGA inference latency to 5 μs for deep neural networks.
problem Deploying ultra low-latency, low-power deep neural networks on FPGAs.
method Extending hls4ml library, using model compression techniques like pruning and quantization-aware training.
result Achieved inference latency of 5 μs with 97% resource reduction.
Evolutionary method constructs CNNs for data compression and classification.
problem Creating efficient CNNs for data compression and classification.
method Two-step approach using evolutionary algorithms: 1) Convolutional autoencoder, 2) Convolutional neural network. Compression trade-off considered.
result Framework achieves comparable accuracy to hand-crafted networks, demonstrating effectiveness.
Automated Bayesian coreset construction for scalable inference.
problem Designing scalable and theoretically sound automated inference algorithms.
method Sparsity-constrained variational inference within an exponential family.
result Automated Bayesian coreset construction with improved KL divergence to the exact posterior.
Automates design of lightweight neural networks for image classification.
problem Designing efficient neural networks for edge devices with limited computational resources.
method Uses the Mesh Adaptive Direct Search (MADS) algorithm to optimize network architecture.
result Achieves comparable performance to standard methods with fewer design trials.
Gaussian Process upsampling boosts OCR accuracy from low-res images.
problem Low-quality and downsampled image data hinders OCR accuracy.
method Gaussian Process upsampling model for improving OCR on low-resolution documents.
result Upsampling improves OCR accuracy on low-resolution images.
Improved deep learning model deployment on tiny MCUs with mixed-precision quantization.
problem Memory limitations prevent accurate deployment of DNN models on tiny MCUs.
method Automated mixed-precision quantization using Reinforcement Learning for MCU constraints.
result Mixed-precision models achieve high accuracy with uniform quantization policies.
Paper explores AIaaS on SDI, offering services in smart sectors.
problem Complexity of infrastructures driving AIaaS.
method Proposes architectural scheme based on SDIs with AI-aaS applications.
result Experimental results for three AI-aaS applications.
This paper examines the problem of locating outlier columns in a large, otherwise low-rank, matrix. We propose a simple two-step adaptive sensing and inference approach and establish theoretical guarantees for its performance; our results show that accurate outlier identification is achievable using very few linear sum…
GANCS uses GANs to speed up MRI reconstruction while maintaining diagnostic quality.
problem Time and resource intensive MRI reconstruction and loss of diagnostic quality in compressed sensing.
method Generative adversarial networks (GAN) trained on historical patient data to learn diagnostic-quality MR images.
result GANCS reconstructs MRI images in a few milliseconds with high contrast and texture details.
Paper proposes an AutoML framework for efficient device-edge co-inference.
problem Finding optimal hyper-parameters for model sparsity and feature compression.
method Sequential decision problem solved using deep reinforcement learning (DRL).
result Achieves better communication-computation trade-off and significant speedup.
Machine learning recreates the periodic table from element properties.
problem Recreating the periodic table using machine learning.
method Unsupervised machine learning with GTM for feature embedding.
result PTG autonomously generates various periodic table layouts.
Prediction Factory automates predictive model development and evaluation.
problem Rapidly developing and sharing predictive models with domain experts.
method Data science automation system with three interfaces: baseline, full, and optional automation.
result Full automation interface generated reports funded 57.5% of the time, compared to 42.5% for baseline.
Generative models encode and decode 3D crystal structures from a large dataset.
problem Challenges in encoding and decoding 3D crystal structures from large datasets.
method Training two neural networks on a dataset of over 120,000 crystal structures to encode and decode 3D atom positions.
result Ability to generate compressed, continuous latent space representations and decode molecules accurately.
Automated theorem prover proves non-orderability of groups.
problem Proving non-orderability of groups.
method Generic automated theorem prover with tools like positive cones, torsions, generalised torsions, and cofinal elements.
result Demonstrated automated proof of non-orderability of groups.
Flexible framework compresses models using LC algorithm.
problem Efficiently compressing neural networks for resource constraints.
method Decouples learning and compression steps with alternating L and C phases.
result Compressed models maintain performance and accuracy.
Reduces multiclass and regression compression schemes to binary ones.
problem Developing efficient learning algorithms for multiclass and regression problems.
method Reduces sample compression schemes for binary classes to multiclass and regression settings.
result Establishes new compression schemes for multiclass and regression problems.
Proposes a link between randomness and compression in deep learning.
problem Improving efficiency in deep learning training.
method Introduces a novel tomographic compression framework called Dual Tomographic Compression (DTC).
result Demonstrates high correlation between learning performance and Gibbs entropy over compression ratios.
A new method classifies images using a compressed summary.
problem Classifying images efficiently.
method Compressive learning with a novel sketch function.
result Improved image classification with less data.
Unified framework for non-compressed networks from compression bounds.
problem Generalization of large deep neural networks with huge model size.
method Unified framework converting compression based bounds to non-compressed networks.
result Unified data-dependent generalization error bound with tighter evaluation.
PARSEC compresses text for sentiment analysis with minimal loss in accuracy.
problem Compressing text data for sentiment analysis without losing accuracy.
method Uses Parts-of-Speech tags to compress text intelligently.
result Accurate compression is possible with minimal loss in sentiment classification accuracy.
Shape adaptor learns flexible resizing factors for neural networks.
problem Fixed resizing layers limit network performance.
method Learnable reshaping factor for traditional resizing layers.
result Performance increases consistently across multiple datasets.
Paper introduces a new adaptive gradient method with gradient compression for distributed training.
problem Communication overhead in distributed machine learning systems.
method Adaptive gradient method with gradient compression, scalable system BytePS-Compress.
result Convergence rate of O ( 1 / T ) \mathcal{O}(1/\sqrt{T}) O ( 1/ T ) for non-convex problems. Study identifies what healthcare tasks can and should be automated.
problem Undesired consequences of automation in healthcare.
method Quantitative analysis using machine learning models trained on practitioner and expert ratings.
result Developed an analytical tool (Automatability-Desirability Matrix) for policymakers and leaders.
Generalizes model compression as constrained optimization, including various techniques.
problem Large neural nets for tasks like object recognition are too big for mobile devices.
method Formulated as constrained optimization, developed a general algorithm using augmented Lagrangian and alternating optimization.
result Guaranteed to find the best compressed model for the task locally.
Galen algorithm compresses neural networks for specific hardware with reduced latency.
problem Finding optimal compression policies for neural networks on specific hardware.
method Reinforcement learning using pruning and quantization to optimize inference latency.
result Compressed ResNet18 for ARM processor reduced inference latency by 80%.
This paper surveys scalable automated alignment methods for LLMs.
problem Scalability issues in traditional human-annotated alignment methods for LLMs.
method Categorizes and discusses various automated alignment methods.
result Emerging automated alignment methods are effective and scalable.
The (isothermic) compressibility of lattice knots can be examined as a model of the effects of topology and geometry on the compressibility of ring polymers. In this paper, the compressibility of minimal length lattice knots in the simple cubic, face centered cubic and body centered cubic lattices are determined. Our r…
This paper uses machine learning to assist automation engineers in decision making.
problem Imperfect decision making by automation engineers leads to multiple iterations and increased time for software development.
method Defined challenges and proposed solutions using machine learning for automation engineering, including code classification, finding similar code snippets, and hardware selection.
result Paragraph embedding techniques achieved an F1-score of 72% for classifying automation using code snippets, and autoencoder models for hardware recommendation achieved p@3 and p@5 of 0.79 and 0.95, respectively.
Paper compresses neural network weight-updates for image artifacts removal.
problem Efficiently compressing neural network weight-updates for image artifacts removal.
method Fine-tuning a pre-trained artifact removal network on target data with a compression objective that encourages sparse and quantized weight-updates.
result Achieves reconstruction quality comparable to traditional codecs at comparable bitrates.
HiLLoC compresses large images losslessly using VAEs.
problem Lossless compression of large color photographs.
method Fully convolutional VAE models trained on ImageNet are applied to lossless compression.
result Achieves state-of-the-art compression for full-size ImageNet images.
Neural NCD reveals LLMs don't compress well for classification.
problem The disconnect between compression and classification in neural networks.
method Developed Neural NCD to compare LLMs to classic algorithms, finding classification accuracy not correlated with compression rate.
result Classification accuracy is not predictable by compression rate alone, challenging current understanding.
BDC compresses both sample size and dimensionality of large datasets.
problem Large datasets in both sample size and dimensionality.
method Two-stage framework using Decoded MMD, Reconstruction MMD, and Encoded MMD.
result BDC achieves comparable or superior performance with lower cost and higher compression rates.
This paper compresses RNNs for IoT devices by 15-38x using Kronecker products.
problem Resource constraints on IoT devices make RNNs difficult to deploy.
method Kronecker product (KP) for compressing RNN layers by 15-38x with minimal accuracy loss.
result Kronecker product can compress RNNs by 50x when quantized to 8-bits.
Infinite diameter found in compression body graph.
problem Determining the diameter of compression body graphs.
method Analyzing the structure and connections within compression body graphs.
result The compression body graph has infinite diameter.
DoCoFL compresses model updates for cross-device federated learning.
problem Downlink compression for cross-device federated learning where clients may appear only once.
method Proposes DoCoFL framework for downlink compression in cross-device federated learning.
result Significant bi-directional bandwidth reduction with competitive accuracy.
Automated strategies improve model adaptation efficiency.
problem Manual adaptation strategies are time-consuming and costly.
method Flexible adaptive mechanism deployment for automated adaptation strategies.
result Automated strategies achieve better or comparable performance.
DeepTwist compresses models by occasionally distorting weights, improving accuracy and efficiency.
problem Challenges in model compression due to high design complexity and additional overhead.
method DeepTwist uses occasional weight distortion without changing training algorithms.
result Significantly improved compression rates for various techniques with reduced effort.
Paper presents a new video compression method using autoencoders.
problem Efficient video compression with reduced quality loss.
method 3D autoencoder with discrete latent space and autoregressive prior trained jointly.
result Method outperforms state-of-the-art learned video compression networks.
New taxonomy and evaluation of neural network compression methods.
problem Efficiency of deep neural networks in real-world applications.
method Categorization and evaluation of tensor factorization and probabilistic compression methods.
result SVD and probabilistic compression methods are complementary and give the best results.
Proposes a method to train neural networks directly on compressed text data.
problem Training neural networks on compressed text data without decompression.
method Introduces composer modules to encode symbols from grammar compression rules into vector representations.
result Demonstrates that the proposed method can achieve both memory and computational efficiency while maintaining moderate performance.
Unified framework improves model compression while maintaining robustness.
problem Achieving high compression ratios without sacrificing adversarial robustness.
method Adversarially Trained Model Compression (ATMC) framework integrating multiple compression techniques.
result ATMC achieves better trade-off between model size, accuracy, and robustness.
Paper proposes SCALLION and SCAFCOM for compressed FL with reduced communication.
problem Reducing communication overhead in Federated Learning with data heterogeneity and partial participation.
method Revisit and simplify stochastic controlled averaging, proposing SCALLION and SCAFCOM for unbiased and biased compression.
result SCALLION and SCAFCOM outperform existing methods in communication and computation complexities.
Study shows LLC correlates with neural network compressibility.
problem Evaluating limits of neural network compression.
method Extended minimum description length principle using singular learning theory.
result Complexity estimates based on LLC are linearly correlated with compressibility.
New uncertainty principle limits compression in distributed learning, suggesting optimal methods.
problem Minimizing communication cost while maintaining message quality in distributed learning.
method Formalized uncertainty principle for compression operators, introduced Kashin compression.
result Explicit formula for Kashin compression's variance bound, dimension independent.
Gradient descent with compressed iterates reduces model size for federated learning.
problem Efficiently compressing model sizes in federated learning.
method Gradient descent with iterates compressed using lossy randomized techniques.
result Improves model compression and aggregation in federated learning.
Automated machine learning simplifies model selection and tuning.
problem Manual tuning of machine learning models by data scientists is time-consuming and requires extensive expertise.
method Review of AutoML techniques including automated feature engineering, model learning, and deep learning.
result Current AutoML techniques can significantly reduce the burden of manual tuning.