A method for trust evaluation of devices in human-device coexistence systems.
problem Efficient trust evaluation of devices in systems with diverse physical and social attributes.
method Canonical correlation analysis-enhanced hypergraph self-supervised learning (HSLCCA).
result The proposed HSLCCA method significantly outperforms baseline algorithms in identifying trusted devices.
Deep learning detects pneumonia with 36x compression on low-power devices.
problem High accuracy pneumonia detection on low-power embedded devices.
method Structured weight pruning method for compression and maintaining accuracy.
result Up to 36x compression ratio with no accuracy loss.
Survey of knowledge distillation for resource-limited devices.
problem Deploying large deep learning models on resource-limited devices.
method Knowledge distillation using a smaller model trained with information from a larger model.
result A new metric (distillation metric) for comparing different knowledge distillation algorithms.
POET enables large neural network training on tiny devices with reduced energy.
problem Training large neural networks on memory-limited edge devices.
method Jointly optimizes rematerialization and paging for memory reduction, formulating an MILP for energy-efficient training.
result POET trains ResNet-18 and BERT within Cortex-M memory constraints, outperforming current methods in energy efficiency.
New approach for sharing deep learning costs between devices and cloud.
problem Prohibitive deep learning computational requirements for embedded devices.
method Study of representation compressibility in MobileNetV2 for balancing computation, bandwidth, and accuracy.
result An optimal splitting layer for network can be found with a simple PCA-based compression scheme.
Placeto learns efficient device placements for any neural network graph.
problem Finding efficient device placements for neural network training.
method Reinforcement learning approach with iterative placement improvements and graph embeddings.
result Placeto requires up to 6.1x fewer training steps and generalizes to unseen graphs.
Mobile devices learn audio representations using self-supervised methods.
problem Learning general-purpose audio representations on mobile devices.
method Temporal context exploitation in spectrogram domain, inspired by Word2Vec.
result Self-supervised models can produce embeddings that perform similarly to fully supervised models.
Deep neural networks show great potential as solutions to many sensing application problems, but their excessive resource demand slows down execution time, pausing a serious impediment to deployment on low-end devices. To address this challenge, recent literature focused on compressing neural network size to improve pe…
Recent breakthroughs in Neural Architectural Search (NAS) have achieved state-of-the-art performance in many tasks such as image classification and language understanding. However, most existing works only optimize for model accuracy and largely ignore other important factors imposed by the underlying hardware and devi…
New layer improves neural networks' flexibility and speed.
problem Designing efficient neural networks for real-time learning.
method Sparse Associative Memories for incremental learning.
result Improved Neural Networks with faster training and real-time support.
GEM detects malicious accounts using adaptive embeddings from heterogeneous graphs.
problem Detecting malicious accounts on a leading mobile payment platform.
method Adaptive learning of discriminative embeddings from heterogeneous account-device graphs with attention mechanism for node importance.
result GEM consistently outperforms competitive methods in detecting malicious accounts.
Deep belief network improves smartphone activity recognition.
problem Activity recognition on mobile devices.
method Categorization through deep belief network.
result 98.25% correct diagnosis in training data, 93.01% in test data.
A new quantization strategy reduces Transformer model size and inference time.
problem Heavy computation load and memory overhead in Transformer models for mobile devices.
method Mixed precision quantization with varying bits per word in embedding blocks.
result 11.8x smaller model size and 3.5x speed up for on-device NMT.
TASO optimizes CNN models for memory-constrained devices.
problem Large CNN models are too resource-intensive for mobile devices.
method Integer Linear Programming (ILP) for selecting operations and workspace allocation.
result 8x speedup and 2.2x reduction in memory usage compared to greedy algorithms.
Next generation of embedded Information and Communication Technology (ICT) systems are collaborative systems able to perform autonomous tasks. The remarkable expansion of the embedded ICT market, together with the rise and breakthroughs of Artificial Intelligence (AI), have put the focus on the Edge as it stands as one…
Python scripts analyze MRAM-based neuromorphic devices' process variation impacts on machine learning accuracy.
problem Impact of process variation on MRAM-based neuromorphic devices' performance in machine learning applications.
method Developed transportable Python scripts to analyze output variation under changes in device dimensions.
result Revealed impacts and limits for processing variation of device fabrication on energy vs. accuracy tradeoffs.
PoET-BiN reduces power consumption in neural networks on embedded devices.
problem Power inefficiency in neural network implementations on embedded platforms.
method Look-Up Table based implementation with a modified Decision Tree approach.
result Near state-of-the-art results with up to 6 orders of magnitude energy reduction.
The recent advances in deep neural networks (DNNs) make them attractive for embedded systems. However, it can take a long time for DNNs to make an inference on resource-constrained computing devices. Model compression techniques can address the computation issue of deep inference on embedded devices. This technique is …
LAD-BNet improves real-time energy forecasting on edge devices.
problem Real-time energy forecasting on edge devices for smart grid optimization and intelligent buildings.
method Hybrid neural architecture combining temporal lag exploitation and TCN with dilated convolutions.
result 14.49% MAPE at 1-hour horizon with 18ms inference time on Edge TPU, 8-12x faster than CPU.
KM method reduces ConvNet parameters to 9% higher accuracy with minimal additional memory.
problem Expensive memory usage for training ConvNets on embedded devices.
method Kernel Modulation (KM) method that adapts all network parameters for each task.
result KM delivers up to 9% higher accuracy than other parameter-efficient methods.
Most of the parameters in large vocabulary models are used in embedding layer to map categorical features to vectors and in softmax layer for classification weights. This is a bottle-neck in memory constraint on-device training applications like federated learning and on-device inference applications like automatic spe…
Deep learning classifies animal behavior from wearable accelerometers.
problem Classifying animal behavior from accelerometer data.
method End-to-end deep neural network with IIR and FIR filters.
result Outperforms state-of-the-art algorithms in real-time classification.
There is growing interest in being able to run neural networks on sensors, wearables and internet-of-things (IoT) devices. However, the computational demands of neural networks make them difficult to deploy on resource-constrained edge devices. To meet this need, our work introduces a new recurrent unit architecture th…
Edge devices adapt pre-trained models to local data without backpropagation.
problem Adapting pre-trained models to edge devices' local data distributions.
method Feed-forward latent domain adaptation using cross-attention.
result Consistent improvements over ERM baselines and domain-supervised adaptation.
Improved cardiac arrhythmia detection in wearable devices with neural networks.
problem Resource constraints in low-power wearable devices for accurate arrhythmia detection.
method Adapted a convolutional-recurrent neural network to a low-power microcontroller, optimizing for precision and memory usage.
result Reduced F1 score from 0.8 to 0.784 in fixed-point precision, with a 195.6KB memory footprint and 33.98MOps/s throughput. Convolutional neural networks have recently achieved significant breakthroughs in various image classification tasks. However, they are computationally expensive,which can make their feasible mplementation on embedded and low-power devices difficult. In this paper convolutional neural network binarization is implemente…
Latent Replay reduces continual learning computation and storage needs.
problem Catastrophic forgetting in continual learning.
method Store activations volumes at intermediate layers instead of past data, slow down learning below the latent replay layer.
result Latent Replay achieves state-of-the-art performance on complex video benchmarks.
New method accelerates CNNs for mobile devices by approximating tensors and quantizing weights.
problem Efficiently compress and accelerate CNNs for mobile devices.
method Low-rank tensor approximation in Tucker format combined with quantization of weights and activations.
result Our method significantly improves CNN performance on various classification tasks.
Motion sensors such as accelerometers and gyroscopes measure the instant acceleration and rotation of a device, in three dimensions. Raw data streams from motion sensors embedded in portable and wearable devices may reveal private information about users without their awareness. For example, motion data might disclose …
A new framework for mobile authentication using deep metric learning.
problem Challenges in mobile authentication using behavioral biometrics.
method Deep metric learning, private data protection, flexible training scheduling.
result 95% authentication accuracy on public datasets, robust against attacks.
Cyber-Physical Systems (CPSs) have been pervasive including smart grid, autonomous automobile systems, medical monitoring, process control systems, robotics systems, and automatic pilot avionics. As usually implemented on embedded devices, CPS is typically constrained by computation capacity and energy consumption. In …
Convolutional neural networks have achieved astonishing results in different application areas. Various methods which allow us to use these models on mobile and embedded devices have been proposed. Especially binary neural networks seem to be a promising approach for these devices with low computational power. However,…
Survey of knowledge distillation for model compression.
problem Deploying large deep models on resource-limited devices.
method Knowledge distillation from large teacher models to small student models.
result Effective model compression and acceleration.
New HANNA architecture improves accuracy, latency, and energy efficiency for mobile devices.
problem Designing neural networks optimized for specific hardware devices.
method Differentiable NAS method with energy term added to loss function.
result HANNA achieves significant speedup and energy efficiency over MobileNetv2 and CondenseNet.
Personalized stress model using transfer learning from 20 participants.
problem Limited generalizability of machine learning models due to individual physiological differences.
method Transfer learning from a base model trained on 20 participants' physiological data collected in real-time.
result Improved model personalization and cross-domain performance.
FedZKT enables resource-constrained devices to participate in federated learning with heterogeneous models.
problem Inequality in resource allocation hinders participation from resource-constrained devices in federated learning.
method Zero-shot knowledge transfer through a server-assigned distillation process.
result FedZKT effectively transfers knowledge across heterogeneous on-device models without requiring comparable local training efforts.
Convolutional neural networks have achieved astonishing results in different application areas. Various methods that allow us to use these models on mobile and embedded devices have been proposed. Especially binary neural networks are a promising approach for devices with low computational power. However, training accu…
METEOR learns efficient representations from multi-modal data streams.
problem Efficiently interpreting multi-modal information in complex environments.
method METEOR learns compact representations by sharing parameters within semantically meaningful groups and preserving domain-agnostic semantics.
result METEOR reduces memory usage by around 80% compared to conventional methods.
APGE protects graph node representations from inference attacks.
problem Privacy leakage in graph embedding methods.
method Adversarial training framework with disentangling and purging mechanisms.
result APGE preserves structural and utility attributes while concealing private information.
Keyword spotting--or wakeword detection--is an essential feature for hands-free operation of modern voice-controlled devices. With such devices becoming ubiquitous, users might want to choose a personalized custom wakeword. In this work, we present DONUT, a CTC-based algorithm for online query-by-example keyword spotti…
Deep learning identifies unknown IoT devices in network traffic.
problem Unauthorized IoT devices pose security risks in BYOD environments.
method Deep learning applied to network traffic images for IoT device identification.
result Over 99% accuracy in identifying 10 IoT devices and non-white-listed devices.
On-device federated learning updates edge models by exchanging trained results.
problem Limited training data at edge devices due to model drift.
method OS-ELM for sequential training and autoencoder for anomaly detection, combined with federated learning.
result The proposed approach produces a merged model as accurately as traditional methods with lower costs.
WNQ uses weight normalization to reduce quantization error in deep neural networks.
problem High quantization error in deep neural networks due to long-tail distribution of weights.
method Weight normalization to avoid long-tail distribution and reduce quantization error.
result WNQ achieves state-of-the-art performance on CIFAR-100 and ImageNet.
Two approaches scale up DNN optimization for diverse edge devices.
problem Optimizing DNNs for edge devices with varying performance requirements.
method Reuse performance predictors on proxy devices and build scalable predictors.
result Optimized DNN designs for many different edge devices without lengthy optimization.
Improved IDS using LSTM and embedding for network traffic data.
problem Network security threats and need for IDS to detect attacks.
method Proposes IDS models using LSTM for time-series and embedding for categorical network traffic data.
result Improves binary classification accuracy to 99.72%.
Paper proposes FTT-NAS to create fault-tolerant CNNs for edge devices.
problem Faults in edge devices affect deep learning applications.
method Formalized fault models, implemented FTT-NAS, incorporated FTT.
result Discovered CNNs outperform baseline architectures with fault tolerance.
This paper optimizes how deep learning models are distributed across different devices.
problem Optimizing how large, complex neural networks are split across multiple devices.
method Identified and solved an optimization problem for device placement of DNN operators.
result Automated algorithms that solve the device placement problem for modern pipelined settings.
Automated VOC classification using embedded machine learning.
problem Classifying volatile organic compounds (VOCs) from electronic nose data.
method Embedded machine learning algorithm on Arduino Uno for VOC detection.
result 99% classification accuracy achieved in embedded solution.