Study develops sign recognition system for DHH users.
problem Accessibility of voice-controlled devices for Deaf and Hard-of-Hearing users.
method Multimodal data (RGB video and skeletal data) for sign language recognition using deep learning.
result Validation on GMUASL51 dataset of 12 users and 13107 samples across 51 signs.
Accurate on-device keyword spotting (KWS) with low false accept and false reject rate is crucial to customer experience for far-field voice control of conversational agents. It is particularly challenging to maintain low false reject rate in real world conditions where there is (a) ambient noise from external sources s…
Automatic voice-controlled systems have changed the way humans interact with a computer. Voice or speech recognition systems allow a user to make a hands-free request to the computer, which in turn processes the request and serves the user with appropriate responses. After years of research and developments in machine …
Keyword spotting--or wakeword detection--is an essential feature for hands-free operation of modern voice-controlled devices. With such devices becoming ubiquitous, users might want to choose a personalized custom wakeword. In this work, we present DONUT, a CTC-based algorithm for online query-by-example keyword spotti…
In a voice-controlled smart-home, a controller must respond not only to user's requests but also according to the interaction context. This paper describes Arcades, a system which uses deep reinforcement learning to extract context from a graphical representation of home automation system and to update continuously its…
The paper enhances a virtual assistant's humor to improve user satisfaction.
problem Improving a virtual assistant's ability to deliver humorous responses.
method Combines traditional NLP techniques with self-attentional networks and multi-task learning, using implicit feedback for labeling.
result Deep-learning models outperform heuristic methods in real-world user satisfaction.
FedZKT enables resource-constrained devices to participate in federated learning with heterogeneous models.
problem Inequality in resource allocation hinders participation from resource-constrained devices in federated learning.
method Zero-shot knowledge transfer through a server-assigned distillation process.
result FedZKT effectively transfers knowledge across heterogeneous on-device models without requiring comparable local training efforts.
Deep learning identifies unknown IoT devices in network traffic.
problem Unauthorized IoT devices pose security risks in BYOD environments.
method Deep learning applied to network traffic images for IoT device identification.
result Over 99% accuracy in identifying 10 IoT devices and non-white-listed devices.
On-device federated learning updates edge models by exchanging trained results.
problem Limited training data at edge devices due to model drift.
method OS-ELM for sequential training and autoencoder for anomaly detection, combined with federated learning.
result The proposed approach produces a merged model as accurately as traditional methods with lower costs.
A method for trust evaluation of devices in human-device coexistence systems.
problem Efficient trust evaluation of devices in systems with diverse physical and social attributes.
method Canonical correlation analysis-enhanced hypergraph self-supervised learning (HSLCCA).
result The proposed HSLCCA method significantly outperforms baseline algorithms in identifying trusted devices.
Two approaches scale up DNN optimization for diverse edge devices.
problem Optimizing DNNs for edge devices with varying performance requirements.
method Reuse performance predictors on proxy devices and build scalable predictors.
result Optimized DNN designs for many different edge devices without lengthy optimization.
This paper optimizes how deep learning models are distributed across different devices.
problem Optimizing how large, complex neural networks are split across multiple devices.
method Identified and solved an optimization problem for device placement of DNN operators.
result Automated algorithms that solve the device placement problem for modern pipelined settings.
Survey on-device ML challenges and future directions.
problem Training machine learning models on-device with limited resources.
method Reformulated as resource constrained learning, comparing techniques from various AI areas.
result Identification of open challenges and future research directions.
Distributed learning adapts to diverse devices, improving performance.
problem Training neural networks on devices with varying capabilities and resources.
method Each device trains a customized neural network, sharing parameters with others.
result Achieves higher rewards on more powerful devices without sacrificing weaker ones.
Flexible device participation improves federated learning convergence.
problem Strict device participation limits federated learning reach.
method Analytical results and new aggregation scheme for flexible participation.
result Convergence improved with flexible device participation.
Semi-decentralized federated learning combines device-to-server and device-to-device communications for faster convergence.
problem Faster convergence in federated learning with decentralized model training.
method Two timescale hybrid federated learning (TT-HF) with cooperative D2D model aggregations.
result Achieves sublinear convergence rate of O(1/t) with adaptive control algorithm.
This paper optimizes AI inference on edge devices with reduced communication and computation costs.
problem Efficiently performing AI inference on resource-constrained edge devices with reduced communication and computation costs.
method A three-step framework for effective inference: model split point selection, communication-aware model compression, and task-oriented encoding of intermediate features.
result Our proposed framework achieves a better trade-off and significantly reduces inference latency compared to baseline methods.
AutoML has become a popular service that is provided by most leading cloud service providers today. In this paper, we focus on the AutoML problem from the \emph{service provider's perspective}, motivated by the following practical consideration: When an AutoML service needs to serve {\em multiple users} with {\em multi…
We propose a real-time context-aware learning system along with the architecture that runs on the mobile devices, provide services to the user and manage the IoT devices. In this system, an application running on mobile devices collected data from the sensors, learned about the user-defined context, made predictions in…
SplitEasy trains ML models on mobile devices without server data transfer.
problem Training complex DL models on resource-limited mobile devices.
method Split learning approach where sensitive layers are trained locally, computationally intensive layers on server.
result SplitEasy trains models on mobile devices with minimal data transfer, near-constant time per sample.
Fog learning distributes ML model training across heterogeneous devices and networks.
problem Challenges with conventional federated learning in heterogeneous networks.
method Intelligent distribution of ML model training across nodes from edge devices to cloud servers.
result Enhanced federated learning with multi-layer hybrid framework considering network, heterogeneity, and proximity.
FedCluster accelerates federated learning convergence by cycling device groups.
problem Federated learning convergence issues with device-level data heterogeneity.
method FedCluster groups devices into clusters that cycle through learning rounds, boosting convergence with meta-updates.
result FedCluster achieves faster convergence in nonconvex optimization compared to FedAvg.
A new federated learning framework for handling device heterogeneity.
problem Handling device heterogeneity in federated learning.
method Superquantile-based objective with parameterized levels of conformity, optimized using secure aggregation.
result The optimization algorithm converges to a stationary point.
SEFR is a fast, energy-efficient classifier for ultra-low power devices.
problem Running machine learning on battery-powered devices is challenging due to time and energy constraints.
method SEFR is an ultra-low power classifier with linear time complexity for training and testing.
result SEFR is 63 times faster and 70 times more energy efficient than state-of-the-art classifiers.
Automates detection of electric devices in 3D x-ray images of luggage.
problem Detecting electric devices in cluttered 3D baggage images.
method Unpack, Predict, eXtract, Repack (UXPR) algorithm using segmentation and ensemble learning.
result System can accurately detect electric devices in 3D baggage images.
A lightweight FPGA-based reinforcement learning approach for edge devices.
problem Resource constraints and inefficiency of DQN on edge devices.
method OS-ELM based training algorithm and L2 regularization for stability.
result 29.77x and 89.40x faster than conventional DQN-based approach for CartPole-v0 task.
AMS improves video inference on edge devices by adapting a small model with online knowledge distillation.
problem High computation cost of Deep Neural Networks for real-time video inference on edge devices.
method AMS uses a remote server to continually train and adapt a small model on edge devices, using online knowledge distillation from a large model.
result 0.4--17.8 percent mean Intersection-over-Union improvement in video semantic segmentation.
Federated learning enables private model training across devices.
problem Private and collaborative machine learning across multiple devices.
method Designing scalable, privacy-preserving FL systems using graph-based optimization.
result Personalized models for each device while maintaining data privacy.
Flower framework simplifies federated learning experiments on edge devices.
problem Realistic implementation of Federated Learning on edge devices is challenging.
method Developed a comprehensive federated learning framework, Flower, supporting large-scale experiments on heterogeneous devices.
result Flower enables federated learning experiments with up to 15M client size using only two high-end GPUs.
Recent breakthroughs in Neural Architectural Search (NAS) have achieved state-of-the-art performance in many tasks such as image classification and language understanding. However, most existing works only optimize for model accuracy and largely ignore other important factors imposed by the underlying hardware and devi…
Recurrent neural networks (RNNs) are powerful constructs capable of modeling complex systems, up to and including Turing Machines. However, learning such complex models from finite training sets can be difficult. In this paper we empirically show that RNNs can learn models of computer peripheral devices through input a…
POET enables large neural network training on tiny devices with reduced energy.
problem Training large neural networks on memory-limited edge devices.
method Jointly optimizes rematerialization and paging for memory reduction, formulating an MILP for energy-efficient training.
result POET trains ResNet-18 and BERT within Cortex-M memory constraints, outperforming current methods in energy efficiency.
On-device inference of machine learning models for mobile phones is desirable due to its lower latency and increased privacy. Running such a compute-intensive task solely on the mobile CPU, however, can be difficult due to limited computing power, thermal constraints, and energy consumption. App developers and research…
Survey of knowledge distillation for resource-limited devices.
problem Deploying large deep learning models on resource-limited devices.
method Knowledge distillation using a smaller model trained with information from a larger model.
result A new metric (distillation metric) for comparing different knowledge distillation algorithms.
MetaDVFS uses device and application metadata to improve DVFS efficiency.
problem Improving energy efficiency in mobile platforms with diverse applications and hardware.
method Formulates DVFS as a multi-task reinforcement learning problem and introduces MetaDVFS, leveraging metadata for knowledge transfer.
result MetaDVFS achieves up to 26% improvement in Quality of Experience and up to 17% improvement in Performance-Power Ratio.
Training deep learning models on mobile devices recently becomes possible, because of increasing computation power on mobile hardware and the advantages of enabling high user experiences. Most of the existing work on machine learning at mobile devices is focused on the inference of deep learning models (particularly co…
DarkneTZ protects edge devices from DNN model leaks using TEE and model partitioning.
problem Privacy risks of pre-trained DNNs on edge devices through membership inference attacks.
method Model partitioning into sensitive and untrusted parts, leveraging TEE.
result DarkneTZ provides reliable model privacy with minimal performance overhead.
TOCO framework compresses neural networks based on tolerance analysis.
problem Deploying large neural networks on edge devices with limited resources.
method TOCO uses tolerance analysis to perform fine-grained compression, allowing flexibility to hardware changes.
result Fine-grained compression of neural networks on edge devices.
Paper proposes low-rank gradient approximation to save memory for deep neural network training.
problem Memory limitation on mobile devices for deep neural network training.
method Approximating gradient matrices using low-rank parameterization.
result Reduces training memory by about 33.0% for Adam optimization and 4.5% relative lower word error rate on ASR personalization task.
Paper proposes an AutoML framework for efficient device-edge co-inference.
problem Finding optimal hyper-parameters for model sparsity and feature compression.
method Sequential decision problem solved using deep reinforcement learning (DRL).
result Achieves better communication-computation trade-off and significant speedup.
New algorithm eliminates symmetry requirement for training neural networks on resistive device arrays.
problem Training accuracy on resistive device arrays depends on device switching symmetry.
method Developed 'Tiki-Taka' algorithm to minimize unintentional cost term due to device asymmetry.
result Achieves same accuracy with non-symmetric devices as with symmetric devices.
Paper proposes a human-algorithm approach to reduce medical device recall risk and workload.
problem High recall rate and regulatory workload in FDA's 510(k) pathway.
method Developed machine learning models to estimate recall risk and proposed a data-driven clearance policy.
result Conservative evaluation of policy shows a 32.9% improvement in recall rate and 40.5% reduction in workload.
New algorithm extracts device profiles for short-term power predictions in commercial buildings.
problem Short-term power prediction in commercial buildings with high accuracy.
method Unsupervised extraction of device profiles from aggregate power measurements, disaggregation using particle swarm optimization, and state changes forecast by artificial neural networks.
result Developed approach outperforms existing methods with high accuracy.
This paper introduces a novel low-cost device prototype for the automatic diagnosis of diseases, utilizing inputted symptoms and personal background. The engineering goal is to solve the problem of limited healthcare access with a single device. Diagnosing diseases automatically is an immense challenge, owing to their …
Mobile training improves speech recognition for users with unique speech characteristics.
problem Limited generalization of speaker-independent speech recognition models for users with very different speech characteristics.
method Securely training personalized end-to-end speech recognition models on mobile devices, splitting gradient computation to reduce memory usage.
result On-device personalization achieved 58.1% relative word error rate reduction compared to 63.7% in a server environment, with 18.7% performance degradation.
Federated learning on edge devices achieves high accuracy with minimal data exchange.
problem Training deep neural networks on edge devices while maintaining user privacy.
method Training CNN, LSTM, and MLP on MNIST data using federated learning on edge devices (Raspberry Pi4s). Experimentally tested on IID and non-IID samples.
result Up to 85% test accuracy achieved with 2 minutes of training time and <10 MB data exchange per device.
On-device machine learning (ML) enables the training process to exploit a massive amount of user-generated private data samples. To enjoy this benefit, inter-device communication overhead should be minimized. With this end, we propose federated distillation (FD), a distributed model training algorithm whose communicati…
Paper presents a lightweight, unobtrusive method to protect edge device data privacy.
problem Protecting inference data privacy in IoT edge devices with limited compute power.
method A lightweight neural network at edge devices to obfuscate inference data without indicating obfuscation.
result Effectively protects inference data confidentiality while preserving backend accuracy.