Most edge AI focuses on prediction tasks on resource-limited edge devices while the training is done at server machines. However, retraining or customizing a model is required at edge devices as the model is becoming outdated due to environmental changes over time. To follow such a concept drift, a neural-network based…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Edge Devices”
This paper optimizes AI inference on edge devices with reduced communication and computation costs.
problem Efficiently performing AI inference on resource-constrained edge devices with reduced communication and computation costs.
method A three-step framework for effective inference: model split point selection, communication-aware model compression, and task-oriented encoding of intermediate features.
result Our proposed framework achieves a better trade-off and significantly reduces inference latency compared to baseline methods.
Two approaches scale up DNN optimization for diverse edge devices.
problem Optimizing DNNs for edge devices with varying performance requirements.
method Reuse performance predictors on proxy devices and build scalable predictors.
result Optimized DNN designs for many different edge devices without lengthy optimization.
AMS improves video inference on edge devices by adapting a small model with online knowledge distillation.
problem High computation cost of Deep Neural Networks for real-time video inference on edge devices.
method AMS uses a remote server to continually train and adapt a small model on edge devices, using online knowledge distillation from a large model.
result 0.4--17.8 percent mean Intersection-over-Union improvement in video semantic segmentation.
Executing deep neural networks for inference on the server-class or cloud backend based on data generated at the edge of Internet of Things is desirable due primarily to the limited compute power of edge devices and the need to protect the confidentiality of the inference neural networks. However, such a remote inferen…
DarkneTZ protects edge devices from DNN model leaks using TEE and model partitioning.
problem Privacy risks of pre-trained DNNs on edge devices through membership inference attacks.
method Model partitioning into sensitive and untrusted parts, leveraging TEE.
result DarkneTZ provides reliable model privacy with minimal performance overhead.
Mobile edge learning is an emerging technique that enables distributed edge devices to collaborate in training shared machine learning models by exploiting their local data samples and communication and computation resources. To deal with the straggler dilemma issue faced in this technique, this paper proposes a new de…
POET enables large neural network training on tiny devices with reduced energy.
problem Training large neural networks on memory-limited edge devices.
method Jointly optimizes rematerialization and paging for memory reduction, formulating an MILP for energy-efficient training.
result POET trains ResNet-18 and BERT within Cortex-M memory constraints, outperforming current methods in energy efficiency.
Paper proposes an AutoML framework for efficient device-edge co-inference.
problem Finding optimal hyper-parameters for model sparsity and feature compression.
method Sequential decision problem solved using deep reinforcement learning (DRL).
result Achieves better communication-computation trade-off and significant speedup.
Developing active inference agents for edge devices with limited resources.
problem Creating effective active inference agents on edge devices with limited computational resources.
method Introducing a software toolbox to accelerate the development of active inference agents by non-experts.
result Accelerates the democratization of active inference agents for edge devices.
Resource-constrained IoT devices, such as sensors and actuators, have become ubiquitous in recent years. This has led to the generation of large quantities of data in real-time, which is an appealing target for AI systems. However, deploying machine learning models on such end-devices is nearly impossible. A typical so…
Flower framework simplifies federated learning experiments on edge devices.
problem Realistic implementation of Federated Learning on edge devices is challenging.
method Developed a comprehensive federated learning framework, Flower, supporting large-scale experiments on heterogeneous devices.
result Flower enables federated learning experiments with up to 15M client size using only two high-end GPUs.
LAD-BNet improves real-time energy forecasting on edge devices.
problem Real-time energy forecasting on edge devices for smart grid optimization and intelligent buildings.
method Hybrid neural architecture combining temporal lag exploitation and TCN with dilated convolutions.
result 14.49% MAPE at 1-hour horizon with 18ms inference time on Edge TPU, 8-12x faster than CPU.
EDCompress optimizes energy efficiency of CNN models on edge devices.
problem Low energy consumption for edge devices with diverse dataflow types.
method Energy-aware model compression using reinforcement learning.
result Improves energy efficiency by 20X, 17X, 37X in various networks.
New method improves grouped convolutions on edge devices.
problem Efficiently implementing grouped convolutions on edge devices.
method Grouped Spatial Pack Convolutions (GSPC) in TVM.
result GSPC outperforms existing implementations by 3.4x, 8x, and 4x on average.
A new framework reduces data upload for image classification while protecting user privacy.
problem Data upload limitations and privacy concerns in cloud-based image classification.
method Unsupervised autoencoder training at edge devices, followed by latent vector transmission to server for classifier training.
result The framework reduces communications overhead and protects user data privacy.
Survey on pruning CNN models to reduce size for edge devices.
problem Reducing large CNN models for edge deployment.
method Comprehensive review of pruning strategies, criteria, and techniques.
result Pruning accelerates CNN models for edge applications.
This paper studies communication efficiency in federated learning by optimizing the sum-rate-distortion function for indirect multiterminal source coding.
problem Indirect multiterminal source coding in federated learning where edge devices send noisy gradients to the server.
method Analyzes the rate region for the quadratic vector Gaussian CEO problem under unbiased estimator and derives an explicit formula for the sum-rate-distortion function.
result Derives an explicit formula for the sum-rate-distortion function in the special case of identical gradients over edge devices.
Low-rank training improves neural network training on edge devices with non-volatile memory.
problem Training neural networks on edge devices with non-volatile memory, especially in terms of write density and auxiliary memory.
method Low-rank training scheme to address write density and auxiliary memory limitations.
result The low-rank training technique outperforms standard SGD in accuracy and weight writes.
Semi-supervised anomaly detection is an approach to identify anomalies by learning the distribution of normal data. Backpropagation neural networks (i.e., BP-NNs) based approaches have recently drawn attention because of their good generalization capability. In a typical situation, BP-NN-based models are iteratively op…
A lightweight FPGA-based reinforcement learning approach for edge devices.
problem Resource constraints and inefficiency of DQN on edge devices.
method OS-ELM based training algorithm and L2 regularization for stability.
result 29.77x and 89.40x faster than conventional DQN-based approach for CartPole-v0 task.
Orpheus simplifies deep learning deployment on edge devices.
problem Optimizing deep learning inference on edge devices for efficiency.
method Orpheus is a new framework with a small codebase, minimal dependencies, and easy integration.
result Preliminary results show the effectiveness of Orpheus for inference optimisations.
Edge devices learn a global model collaboratively over wireless channels.
problem Learning a global model from edge devices with imperfect channel state information.
method Proposed analog aggregation scheme, receive beamforming at PS, and convergence analysis.
result Performance improvement with more PS antennas, even with imperfect CSI.
Privacy is What We Care About: Experimental Investigation of Federated Learning on Edge Devicescs.LG
Federated Learning enables training of a general model through edge devices without sending raw data to the cloud. Hence, this approach is attractive for digital health applications, where data is sourced through edge devices and users care about privacy. Here, we report on the feasibility to train deep neural networks…
Machine learning (ML), especially deep learning is made possible by the availability of big data, enormous compute power and, often overlooked, development tools or frameworks. As the algorithms become mature and efficient, more and more ML inference is moving out of datacenters/cloud and deployed on edge devices. This…
Machine learning at the edge offers great benefits such as increased privacy and security, low latency, and more autonomy. However, a major challenge is that many devices, in particular edge devices, have very limited memory, weak processors, and scarce energy supply. We propose a hybrid hardware-software framework tha…
Supermarkets need to ensure clean and safe environments for both shoppers and employees. Slips, trips, and falls can result in injuries that have a physical as well as financial cost. Timely detection of hazardous conditions such as spilled liquids or fallen items on supermarket floors can reduce the chances of serious…
Edge devices adapt pre-trained models to local data without backpropagation.
problem Adapting pre-trained models to edge devices' local data distributions.
method Feed-forward latent domain adaptation using cross-attention.
result Consistent improvements over ERM baselines and domain-supervised adaptation.
Much of the focus in the design of deep neural networks has been on improving accuracy, leading to more powerful yet highly complex network architectures that are difficult to deploy in practical scenarios, particularly on edge devices such as mobile and other consumer devices given their high computational and memory …
While neural network hardware accelerators provide a substantial amount of raw compute throughput, the models deployed on them must be co-designed for the underlying hardware architecture to obtain the optimal system performance. We present a class of computer vision models designed using hardware-aware neural architec…
Paper proposes a multi-phase pruning pipeline for deep ensemble learning on IIoT devices.
problem Computational limitations of IoT devices for deep learning models.
method Generates diverse pruned models, applies integer quantization, and uses clustering-based pruning.
result Significant reduction in model size (up to 90%) and improved performance (up to 7%) on IIoT devices.
Neural network compression methods have enabled deploying large models on emerging edge devices with little cost, by adapting already-trained models to the constraints of these devices. The rapid development of AI-capable edge devices with limited computation and storage requires streamlined methodologies that can effi…
Edge-Efficient Deep Learning Models for Automatic Modulation Classification: A Performance Analysiseess.SP
Optimized CNNs for AMC on edge devices reduce complexity without sacrificing accuracy.
problem Developing efficient DL models for AMC on resource-constrained edge devices.
method Pruning, quantization, and knowledge distillation techniques applied to CNNs.
result Optimized models maintain or improve AMC accuracy with reduced complexity.
This paper proposes a communication-efficient deep anomaly detection framework for industrial IoT.
problem Accurately detecting anomalies in time-series data from edge devices in industrial IoT.
method A federated learning-based approach with an Attention Mechanism-based Convolutional Neural Network-Long Short Term Memory (AMCNN-LSTM) model and gradient compression.
result The proposed framework accurately and timely detects anomalies with reduced communication overhead.
Despite showing state-of-the-art performance, deep learning for speech recognition remains challenging to deploy in on-device edge scenarios such as mobile and other consumer devices. Recently, there have been greater efforts in the design of small, low-footprint deep neural networks (DNNs) that are more appropriate fo…
This concept paper highlights a recently opened opportunity for large scale analytical algorithms to be trained directly on edge devices. Such approach is a response to the arising need of processing data generated by natural person (a human being), also known as personal data. Spiking Neural networks are the core meth…
From Federated to Fog Learning: Distributed Machine Learning over Heterogeneous Wireless Networkscs.DC
Fog learning distributes ML model training across heterogeneous devices and networks.
problem Challenges with conventional federated learning in heterogeneous networks.
method Intelligent distribution of ML model training across nodes from edge devices to cloud servers.
result Enhanced federated learning with multi-layer hybrid framework considering network, heterogeneity, and proximity.
Edge language models show bias over time, especially on resource-constrained devices.
problem Bias in edge language models on resource-constrained devices.
method Comparative analysis of text-based bias across edge, cloud, and desktop environments; optimized Llama-2 model on Raspberry Pi 4; feedback loop mechanism to correct bias.
result Llama-2 on Raspberry Pi 4 shows 43.23% and 21.89% more bias over time compared to cloud and desktop models.
Semi-decentralized federated learning combines device-to-server and device-to-device communications for faster convergence.
problem Faster convergence in federated learning with decentralized model training.
method Two timescale hybrid federated learning (TT-HF) with cooperative D2D model aggregations.
result Achieves sublinear convergence rate of O(1/t) with adaptive control algorithm.
HMQ improves quantization for edge devices with mixed precision.
problem Efficient quantization for edge devices with uniform, power-of-two thresholds.
method Introduces HMQ, a mixed precision quantization block that repurposes Gumbel-Softmax for searching over quantization schemes.
result Achieves competitive and state-of-the-art results on ImageNet despite restrictions.
Energy-efficient DL inference for IoT devices reduces power consumption and improves performance.
problem Energy inefficiency in deep learning models for IoT devices.
method Energy-aware early exiting policy to balance energy consumption and inference accuracy.
result Accuracy and service rate improved up to 25% and 35% respectively.
This paper optimizes object tracking on edge devices with small matrices.
problem Efficiently tracking objects in video sequences on edge devices with small matrices.
method Parallelized a Simple Online and Real-time Tracking (SORT) application on shared-memory multicores.
result Throughput-based parallelization technique outperforms multi-threading for small matrices.
Deep Neural Network (DNN) workloads are quickly moving from datacenters onto edge devices, for latency, privacy, or energy reasons. While datacenter networks can be protected using conventional cybersecurity measures, edge neural networks bring a host of new security challenges. Unlike classic IoT applications, edge ne…
BottleNet++: An End-to-End Approach for Feature Compression in Device-Edge Co-Inference Systemscs.LG
The emergence of various intelligent mobile applications demands the deployment of powerful deep learning models at resource-constrained mobile devices. The device-edge co-inference framework provides a promising solution by splitting a neural network at a mobile device and an edge computing server. In order to balance…
Federated learning (FL) allows model training from local data collected by edge/mobile devices while preserving data privacy, which has wide applicability to image and vision applications. A challenge is that client devices in FL usually have much more limited computation and communication resources compared to servers…
Federated learning enables resource-constrained edge compute devices, such as mobile phones and IoT devices, to learn a shared model for prediction, while keeping the training data local. This decentralized approach to train models provides privacy, security, regulatory and economic benefits. In this work, we focus on …
The paper addresses challenges in edge deep learning for IoT, proposing new directions.
problem Challenges in large-scale deep learning adoption for IoT devices.
method Unified view targeting three research directions: federated learning, data-independent deployment, and communication-aware inference.
result A network-centric approach is needed for edge intelligence.
SLIP secures LLMs on edge devices by splitting computation and protecting sensitive parts.
problem Protecting LLMs on edge devices from theft and unauthorized use.
method SLIP uses matrix decomposition to split model computation between secure and vulnerable resources, ensuring zero accuracy degradation and minimal latency.
result SLIP is the first practical, secure hybrid protocol for protecting LLMs on edge devices.