The paper optimizes neural network inference on mobile GPUs.
problem Limited computing power and thermal constraints on mobile CPUs.
method Leverage mobile GPUs for neural network inference.
result Real-time inference of deep neural networks on Android and iOS devices.
This paper optimizes AI inference on edge devices with reduced communication and computation costs.
problem Efficiently performing AI inference on resource-constrained edge devices with reduced communication and computation costs.
method A three-step framework for effective inference: model split point selection, communication-aware model compression, and task-oriented encoding of intermediate features.
result Our proposed framework achieves a better trade-off and significantly reduces inference latency compared to baseline methods.
FixyNN splits CNN models into fixed and trainable parts for efficient on-device inference.
problem Energy inefficiency in on-device CNN inference for real-time computer vision.
method Co-designed hardware accelerator platform with transfer learning for training.
result Achieved nearly 2x better energy efficiency than a conventional accelerator.
The rising popularity of intelligent mobile devices and the daunting computational cost of deep learning-based models call for efficient and accurate on-device inference schemes. We propose a quantization scheme that allows inference to be carried out using integer-only arithmetic, which can be implemented more efficie…
Survey on-device ML challenges and future directions.
problem Training machine learning models on-device with limited resources.
method Reformulated as resource constrained learning, comparing techniques from various AI areas.
result Identification of open challenges and future research directions.
Personalized stress model using transfer learning from 20 participants.
problem Limited generalizability of machine learning models due to individual physiological differences.
method Transfer learning from a base model trained on 20 participants' physiological data collected in real-time.
result Improved model personalization and cross-domain performance.
WEST compresses word embeddings and softmax layers for memory efficiency.
problem Memory constraints in large vocabulary models.
method WEST encodes words with sequences of sub-units, improving compression without performance loss.
result WEST achieves significant compression without sacrificing performance.
FedZKT enables resource-constrained devices to participate in federated learning with heterogeneous models.
problem Inequality in resource allocation hinders participation from resource-constrained devices in federated learning.
method Zero-shot knowledge transfer through a server-assigned distillation process.
result FedZKT effectively transfers knowledge across heterogeneous on-device models without requiring comparable local training efforts.
Paper proposes federated learning for SNNs to enable low-power, online training.
problem Limited data at each device for on-device SNN training.
method Federated Learning (FL) for cooperative SNN training, leveraging local and global feedback.
result FL-SNN achieves significant advantages over separate training and offers a flexible trade-off between accuracy and communication load.
VoiceFilter-Lite separates speech from background in real-time for on-device speech recognition.
problem Separate speech from background in real-time for on-device speech recognition.
method Asymmetric loss, adaptive runtime suppression, quantization to 8-bit.
result VoiceFilter-Lite achieves real-time speech separation and maintains speech recognition performance.
Paper proposes an AutoML framework for efficient device-edge co-inference.
problem Finding optimal hyper-parameters for model sparsity and feature compression.
method Sequential decision problem solved using deep reinforcement learning (DRL).
result Achieves better communication-computation trade-off and significant speedup.
Secure and efficient distributed learning on devices with limited communication.
problem Limited communication and security in distributed on-device learning.
method Proposes SLSGD, a robust distributed optimization algorithm with efficient communication and attack tolerance.
result Stabilizes convergence and tolerates data poisoning on a small number of workers.
A new quantization strategy reduces Transformer model size and inference time.
problem Heavy computation load and memory overhead in Transformer models for mobile devices.
method Mixed precision quantization with varying bits per word in embedding blocks.
result 11.8x smaller model size and 3.5x speed up for on-device NMT.
ActiveHARNet improves resource efficiency in deep learning for HAR and fall detection.
problem Resource efficiency and real-time learning for HAR models.
method Deep ensembled model with incremental learning and active learning.
result Significant efficiency boost during inference and reduction in acquired pool points.
On-device federated learning updates edge models by exchanging trained results.
problem Limited training data at edge devices due to model drift.
method OS-ELM for sequential training and autoencoder for anomaly detection, combined with federated learning.
result The proposed approach produces a merged model as accurately as traditional methods with lower costs.
Federated learning framework extended for personalizing global models.
problem Evaluate personalization strategies for global models on-device.
method Extend federated learning framework, develop tools for analysis and evaluation.
result Personalization yields significant benefits for a large user population.
FD and FAug reduce communication in on-device ML with non-IID data.
problem Minimize communication overhead in on-device ML with non-IID data.
method Federated distillation (FD) and federated augmentation (FAug).
result FD with FAug reduces communication by 26x while maintaining high accuracy.
User-specific KWS system learns new keywords on-device.
problem Out-of-vocabulary problem in traditional KWS systems.
method Query-by-example enrollment and testing, using phonetic posteriors and FST.
result Promising performance on two keywords, preserving simplicity.
NeuZip compresses neural network weights to save memory during training and inference.
problem Memory constraints in neural network training and inference.
method Entropy-based dynamic weight compression.
result Significant reduction in memory usage without performance loss.
MDLdroid improves mobile deep learning for personal sensing with faster training.
problem Continuous local changes and resource constraints in personal mobile sensing affect global model performance.
method ChainSGD-reduce approach to reduce overhead and balance resources.
result 2x to 3.5x faster training on off-the-shelf mobile devices compared to single-device training.
BottleNet++ compresses deep learning features for efficient mobile inference.
problem Balancing computation and communication in mobile devices with deep learning models.
method End-to-end architecture with encoder, non-trainable channel layer, and decoder.
result Achieves high compression ratio and bandwidth reduction with minimal accuracy loss.
This work reduces computation cost for on-device CNN training.
problem High computation cost during on-device CNN training.
method Self-supervised instance filtering and error map pruning.
result Substantial computation saving without significant accuracy loss.
A lightweight FPGA-based reinforcement learning approach for edge devices.
problem Resource constraints and inefficiency of DQN on edge devices.
method OS-ELM based training algorithm and L2 regularization for stability.
result 29.77x and 89.40x faster than conventional DQN-based approach for CartPole-v0 task.
Mobile training improves speech recognition for users with unique speech characteristics.
problem Limited generalization of speaker-independent speech recognition models for users with very different speech characteristics.
method Securely training personalized end-to-end speech recognition models on mobile devices, splitting gradient computation to reduce memory usage.
result On-device personalization achieved 58.1% relative word error rate reduction compared to 63.7% in a server environment, with 18.7% performance degradation.
This paper tackles energy-efficient machine learning on low-power devices.
problem Energy consumption in machine learning due to data communication.
method Dynamic averaging for integer exponential families on low-power processors.
result Achieves comparable model quality with significantly less communication and energy.
Paper proposes low-rank gradient approximation to save memory for deep neural network training.
problem Memory limitation on mobile devices for deep neural network training.
method Approximating gradient matrices using low-rank parameterization.
result Reduces training memory by about 33.0% for Adam optimization and 4.5% relative lower word error rate on ASR personalization task.
Paper proposes efficient weight updates for edge nodes with minimal communication.
problem Inefficient and resource-intensive full weight updates for edge nodes.
method Deep partial updating, selecting a subset of weights to update.
result Achieves similar performance with fewer weight updates.
SPINN optimizes neural network inference on devices and cloud.
problem Inference on mobile devices is challenging due to high computational demands and dynamic connectivity.
method Synergistic progressive inference with a novel scheduler.
result SPINN achieves up to 2x higher throughput and reduces server cost by up to 6.8x.
Much of the focus in the design of deep neural networks has been on improving accuracy, leading to more powerful yet highly complex network architectures that are difficult to deploy in practical scenarios, particularly on edge devices such as mobile and other consumer devices given their high computational and memory …
Framework prevents data leakage in mobile cloud DNNs.
problem Data leakage from cloud DNNs poses privacy risks.
method Privacy-preserving reinforcement learning framework.
result Framework successfully defends against various privacy attacks.
Two new methods reduce communication costs in federated learning.
problem Heavy communication costs in federated learning, especially with non-IID data.
method FedMMD uses MMD constraint for two-stream model training; FedFusion aggregates local and global features.
result FedMMD and FedFusion reduce communication costs by 20% and 60% respectively.
Improves E2E ASR performance on numeric sequences with additional training data and denormalization.
problem Challenges in recognizing numeric sequences out-of-vocabulary in ASR systems.
method Uses text-to-speech for additional numeric training data and a small-footprint neural network for denormalization.
result Reduction of WER by up to a factor of 8 in the longest numeric sequences.
Efficient equivariant MobileNetV2 for medical applications on mobile devices.
problem Limited computational resources for deep learning models in medical applications.
method Design and optimize an equivariant version of MobileNetV2 with model quantization.
result Achieved close-to state-of-the-art performance on medical dataset with improved efficiency.
E2-Train reduces training energy by 80%+ for state-of-the-art CNNs.
problem Efficient training of energy-hungry CNNs on edge devices.
method Selective layer update, stochastic mini-batch dropping, and sign prediction for low-precision backpropagation.
result Achieves >90% energy savings for training ResNet-74 on CIFAR-10.
PairNets optimize AI models for fast IoT applications.
problem Slow training and high memory usage of deep neural networks.
method Developed Pairwise Neural Networks (PairNets) with low memory and fast training.
result PairNets achieve faster training (one epoch) and lower prediction errors.
Federated learning is viewed as a hierarchical latent variable model for new algorithm development.
problem Training models privately across multiple clients while maintaining privacy and efficiency.
method Viewing federated learning as a hierarchical latent variable model and applying Expectation-Maximization (EM) algorithm.
result Proposes FedSparse, a federated learning algorithm that promotes sparsity and reduces communication and inference costs.
Method certifies edge predictions with cloud-level reliability.
problem Ensuring reliability of edge intelligence models.
method Conformal alignment-based cascading mechanism.
result Certifies conditional coverage with user control over risk level.
The increasing demand for on-device deep learning services calls for a highly efficient manner to deploy deep neural networks (DNNs) on mobile devices with limited capacity. The cloud-based solution is a promising approach to enabling deep learning applications on mobile devices where the large portions of a DNN are of…
New method FedEx accelerates federated hyperparameter tuning.
problem Federated hyperparameter tuning challenges in distributed learning.
method FedEx method connecting to weight-sharing, adapted for federated optimization.
result FedEx outperforms natural baselines on various benchmarks.
A hybrid framework reduces ML complexity on edge devices.
problem Limited memory and energy on edge devices.
method Compressed data collection and tailored deep learning network.
result Significant reduction in computational complexity and memory.
Paper tackles intermittent learning for energy-constrained machine learning tasks.
problem Energy-constrained machine learning tasks on intermittently powered systems.
method Developed an algorithm and heuristics for efficient learning under energy constraints.
result Improves energy efficiency by up to 100% and reduces learning examples by up to 50%.
Single-Path NAS designs efficient ConvNets for mobile devices in hours.
problem Designing efficient ConvNets for mobile devices under latency constraints.
method Single-Path NAS, a differentiable method that reduces search cost and inference time.
result Achieves state-of-the-art accuracy on ImageNet with 79ms inference latency.
MoDeGPT compresses large language models without accuracy loss, saving 98% compute costs.
problem Compression of large language models for resource-constrained devices.
method Structured compression framework using modular decomposition and matrix pair reduction.
result MoDeGPT achieves 90-95% zero-shot performance with 25-30% compression rates.
DONNA rapidly finds optimal neural networks across diverse spaces.
problem Efficient scaling and handling of diverse architectural search-spaces in NAS.
method Three-phase pipeline: accuracy predictor, rapid evolutionary search, and optimal model finetuning.
result 100x faster than MNasNet in finding state-of-the-art architectures on-device.
EdgeSpeechNets improve speech recognition on mobile devices.
problem Deploying deep learning for speech recognition on edge devices is challenging.
method Human-machine collaboration for designing efficient DNN architectures.
result EdgeSpeechNets achieve higher accuracy with smaller network size and lower computational cost.
This paper proposes a new weight representation scheme for efficient model compression and performance enhancement.
problem Challenges in achieving performance enhancement on devices due to irregular sparse matrix representations.
method Fine-grained and unstructured pruning method combined with structured weight encryption.
result Achieved high compression ratios and performance on various deep learning models.
ONLAD Core detects anomalies in edge devices with fast learning and low power.
problem Anomaly detection in edge devices with concept drift and data transfers.
method Highly optimized neural network-based anomaly detection on edge devices.
result ONLAD Core achieves fast anomaly detection and low power consumption.
A framework for privacy-preserving DNN pruning and acceleration.
problem Privacy concerns in DNN weight pruning for mobile devices.
method ADMM-based iterative pruning with synthetic data, compiler optimizations.
result 4.2X, 2.5X, and 2.0X speedup with almost no accuracy loss.