Offline RL tackles resource-constrained online deployment with improved policy transfer.
problem Training policies with limited online features using a rich offline dataset.
method Introduce a policy transfer algorithm that first trains a teacher agent with full offline features and then transfers knowledge to a student agent with limited online features.
result Consistent improvement in performance over baseline methods on resource-constrained datasets.
FAVANO improves federated learning for resource-constrained environments.
problem Asynchronous communication in federated learning leads to bias and scalability issues.
method FAVANO is a novel asynchronous federated learning framework for resource-constrained environments.
result FAVANO outperforms existing methods on standard benchmarks.
Energy savings for DNN inference on resource-constrained devices.
problem Energy efficiency in deep learning inference for constrained devices.
method Efficiently searches through equivalent DNN graphs to find the one with the least execution cost.
result Achieves 24% energy savings with minimal performance impact.
Improved distributed learning with reduced communication costs.
problem Efficient communication in resource-constrained environments for distributed learning.
method Proposed a cost-effective partial communication protocol.
result Communication cost is reduced to O(logT), improving significantly on full communication. FedZKT enables resource-constrained devices to participate in federated learning with heterogeneous models.
problem Inequality in resource allocation hinders participation from resource-constrained devices in federated learning.
method Zero-shot knowledge transfer through a server-assigned distillation process.
result FedZKT effectively transfers knowledge across heterogeneous on-device models without requiring comparable local training efforts.
Adaptive Bayesian Optimization for resource-constrained experiments with switching costs.
problem Sequential experimental design with varying costs for changing design variables.
method Adapted batch algorithms to sequential problem, proposing cost-aware and cost-ignorant methods.
result Cost-aware algorithm outperforms tuned process-constrained algorithms in all settings considered.
Paper proposes DP-PASGD for efficient, private IoT learning.
problem Privacy and resource constraints in IoT.
method Differentially private federated learning (DP-PASGD) for resource-constrained IoT.
result DP-PASGD achieves efficient training while maintaining privacy.
Algorithm improves resource allocation for food outreach to homeless.
problem Resource-constrained outreach for homeless individuals and food rescue.
method Thompson sampling with Markov chain recovery (via Stein variational gradient descent) for partially-observed episodic restless bandits.
result Significantly outperforms baselines in both organizations' problems.
Hydra boosts efficiency for long-context reasoning in resource-constrained settings.
problem Quadratic complexity of transformers limits long-context reasoning in resource-constrained systems.
method Hydra uses a modular architecture with adaptive routing between sparse global attention, mixture-of-experts, and dual memories.
result Hydra achieves significant throughput and accuracy improvements for long-context reasoning.
Review of efficient neural networks for TinyML on resource-constrained devices.
problem Resource constraints on ultra-low power MCUs for deep learning models.
method Model compression, quantization, low-rank factorization, model pruning, hardware acceleration, algorithm-architecture co-design.
result Optimized neural network architectures for minimal resource utilization on MCUs.
DBQ quantizes lightweight networks efficiently for resource-constrained devices.
problem High computational and storage complexity of deep neural networks on resource-constrained devices.
method A differentiable non-uniform quantizer that can be mapped onto efficient ternary-based dot product engines.
result Achieves state-of-the-art results with minimal training overhead and best accuracy-complexity trade-off.
Recurrent Neural Networks (RNN) can be difficult to deploy on resource constrained devices due to their size. As a result, there is a need for compression techniques that can significantly compress RNNs without negatively impacting task accuracy. This paper introduces a method to compress RNNs for resource constrained …
This paper optimizes AI inference on edge devices with reduced communication and computation costs.
problem Efficiently performing AI inference on resource-constrained edge devices with reduced communication and computation costs.
method A three-step framework for effective inference: model split point selection, communication-aware model compression, and task-oriented encoding of intermediate features.
result Our proposed framework achieves a better trade-off and significantly reduces inference latency compared to baseline methods.
DQA efficiently quantizes deep neural network activations for resource-constrained devices.
problem Efficiently quantizing deep neural network activations for resource-constrained devices.
method DQA uses simple shifting-based operations and Huffman coding for sub-6-bit quantization.
result DQA achieves significantly better accuracy than direct quantization and state-of-the-art methods.
learn2mix trains neural nets faster by adjusting class proportions dynamically.
problem Training neural nets efficiently with limited resources and imbalanced classes.
method Adaptive class proportion adjustment during training.
result Neural nets trained with learn2mix converge faster than static methods.
Mobile V-MoEs scale down ViTs for resource-constrained vision tasks.
problem Scaling down Vision Transformers for resource-constrained applications.
method Sparse Mixture-of-Experts (MoEs) applied to entire images, with a stable training procedure.
result Mobile V-MoEs achieve better performance-efficiency trade-offs than dense ViTs.
We propose to prune a random forest (RF) for resource-constrained prediction. We first construct a RF and then prune it to optimize expected feature cost & accuracy. We pose pruning RFs as a novel 0-1 integer program with linear constraints that encourages feature re-use. We establish total unimodularity of the constra…
EC2T creates sparse and ternary neural networks for resource-constrained devices.
problem Deploying deep neural networks on resource-constrained devices.
method Entropy-Constrained Trained Ternarization (EC2T) framework.
result EC2T creates sparse and ternary neural networks that are efficient in terms of storage and computation.
Federated Learning helps IoT devices learn without central servers.
problem Resource constraints in IoT devices hinder traditional ML approaches.
method Local training of IoT devices using global models.
result Federated Learning can be applied to IoT devices with varying resource capabilities.
Classification tasks are common across many fields and applications where the decision maker's action is limited by resource constraints. In direct marketing only a subset of customers is contacted; scarce human resources limit the number of interviews to the most promising job candidates; limited donated organs are pr…
Paper proposes efficient pruning method for neural networks.
problem Compressing deep neural networks for resource-constrained devices.
method Adaptive sparsity loss for budget-aware optimization during training.
result Demonstrated effectiveness on various architectures and datasets.
Data aggregation improves HAC for resource-constrained systems.
problem Resource constraints in embedded systems limit HAC's applicability.
method Data aggregation with BETULA algorithm reduces memory and runtime requirements.
result HAC can be applied to large datasets on resource-constrained systems.
A new pricing controller handles resource constraints to infer target prices effectively.
problem Resource constraints prevent fixed-price inference, leading to support exclusion.
method Formalizes support-exclusion failure, designs a target-aware controller, and uses a realized information clock.
result The controller can certify feasible target bands and log continuous local densities, leading to polynomial rates of inference.
Recurrent Neural Networks (RNN) can be difficult to deploy on resource constrained devices due to their size.As a result, there is a need for compression techniques that can significantly compress RNNs without negatively impacting task accuracy. This paper introduces a method to compress RNNs for resource constrained e…
Convolutional neural networks have recently achieved significant breakthroughs in various image classification tasks. However, they are computationally expensive,which can make their feasible mplementation on embedded and low-power devices difficult. In this paper convolutional neural network binarization is implemente…
Many Internet-of-Things (IoT) applications demand fast and accurate understanding of a few key events in their surrounding environment. Deep Convolutional Neural Networks (CNNs) have emerged as an effective approach to understand speech, images, and similar high dimensional data types. Algorithmic performance of modern…
New sparsity attacks degrade DNN efficiency, raising concerns for resource-constrained systems.
problem Vulnerabilities in DNNs through energy and latency attacks.
method Proposed sparsity attacks that modify DNN inputs to reduce activation sparsity, increasing execution time and energy consumption.
result Adversarial sparsity attacks can degrade DNN efficiency by up to 1.82x in image recognition DNNs.
CLEANN detects and mitigates neural network Trojans without labeled data.
problem Trojans in embedded neural networks that bypass detection during inference.
method Dictionary learning and sparse approximation for identifying Trojan triggers, lightweight algorithm/hardware co-design.
result Efficient real-time execution on resource-constrained platforms, competitive attack resiliency.
CHOOSE enhances shallow Transformers for wireless symbol detection.
problem Improving wireless symbol detection with shallow Transformers.
method Introducing autoregressive latent reasoning steps within hidden space.
result Lightweight Transformers achieve comparable performance to deep models.
Tiny Eats GRU detects eating episodes on a microcontroller.
problem Automatic dietary monitoring on low-power devices.
method Shallow gated recurrent unit (GRU) architecture on Arm Cortex M0+.
result Tiny Eats GRU achieves 95.15% accuracy with 4% memory usage and 6 ms latency.
Kolmogorov-Arnold Networks enable ultrafast online learning with fixed-point quantization.
problem Efficient online learning for high-frequency systems with strict memory constraints.
method Fixed-point online training on FPGAs exploiting B-spline locality in KANs.
result Kolmogorov-Arnold Networks are more efficient and expressive than MLPs for low-latency tasks.
ALF reduces network parameters and operations by 70% and 61%, respectively, on embedded hardware.
problem Efficient deployment of deep learning models on resource-constrained hardware.
method Autoencoder-based low-rank filter-sharing technique.
result ALF achieves significant compression with minimal accuracy loss.
T-Basis represents neural network tensors with fewer parameters.
problem Efficiently representing neural network tensors with fewer parameters.
method T-Basis uses Tensor Rings to represent tensors in a neural network, parameterizing them with a small number of coefficients.
result T-Basis achieves high compression rates with minimal performance loss.
A novel method for efficient CDRL over wireless networks.
problem Challenges in collaborative deep reinforcement learning over wireless networks.
method Semantic-aware heterogeneous federated deep reinforcement learning (HFDRL) algorithm.
result Superior performance compared to state-of-the-art baselines.
Post-training quantization saves resources for neural networks.
problem Implementing neural networks in resource-constrained hardware.
method Generalized post-training quantization method (GPFQ) with modifications for sparsity and error analysis.
result Error decays linearly with over-parametrization, showing minor loss of accuracy.
The paper uses information geometry to analyze model compression techniques, focusing on operator factorization.
problem Efficiently compressing deep learning models for resource-constrained devices.
method Information geometry applied to model compression, focusing on operator factorization.
result Iterative methods are crucial for fine-tuning models, especially when compression ratios are fixed.
This paper proposes IMU preintegrated features for efficient deep inertial odometry.
problem Efficient odometry from IMU data is challenging due to sensor imperfections and noise.
method Proposes IMU preintegrated features exploiting IMU motion model's manifold structure.
result Improves odometry performance and reduces computational burdens.
This paper develops the FastRNN and FastGRNN algorithms to address the twin RNN limitations of inaccurate training and inefficient prediction. Previous approaches have improved accuracy at the expense of prediction costs making them infeasible for resource-constrained and real-time applications. Unitary RNNs have incre…
Deep neural networks require large amounts of resources which makes them hard to use on resource constrained devices such as Internet-of-things devices. Offloading the computations to the cloud can circumvent these constraints but introduces a privacy risk since the operator of the cloud is not necessarily trustworthy.…
SWIFT improves time series forecasting on edge devices with wavelet decomposition.
problem Efficient time series forecasting on resource-constrained devices.
method Wavelet decomposition, cross-band fusion, and shared linear mapping.
result SWIFT achieves state-of-the-art performance on multiple datasets.
This work optimizes quantization of linear models to reduce memory usage.
problem Optimizing memory usage for high-dimensional linear models.
method Information-theoretic framework with randomized embedding-based algorithms.
result Matching upper and lower bounds for minimax risk under quantization constraints.
Implementing large-scale deep neural networks with high computational complexity on low-cost IoT devices may inevitably be constrained by limited computation resource, making the devices hard to respond in real-time. This disjunction makes the state-of-art deep learning algorithms, i.e. CNN (Convolutional Neural Networ…
Continuous representation of words is a standard component in deep learning-based NLP models. However, representing a large vocabulary requires significant memory, which can cause problems, particularly on resource-constrained platforms. Therefore, in this paper we propose an isotropic iterative quantization (IIQ) appr…
APQ jointly optimizes neural architecture, pruning, and quantization for efficient inference.
problem Efficient deep learning inference on resource-constrained hardware.
method Joint optimization of neural architecture, pruning, and quantization policy using a quantization-aware accuracy predictor.
result Joint optimization leads to 2.3% higher ImageNet accuracy with reduced latency and energy consumption.
Powerpropagation makes neural networks inherently sparse.
problem Training sparse neural networks to reduce computational footprint and model size.
method Introduces a new weight-parameterisation technique exploiting gradient descent dynamics.
result Models trained with Powerpropagation have a higher density of zero weights, allowing for more efficient pruning.
We present MorphNet, an approach to automate the design of neural network structures. MorphNet iteratively shrinks and expands a network, shrinking via a resource-weighted sparsifying regularizer on activations and expanding via a uniform multiplicative factor on all layers. In contrast to previous approaches, our meth…
Reinforcement learning algorithms are known to be sample inefficient, and often performance on one task can be substantially improved by leveraging information (e.g., via pre-training) on other related tasks. In this work, we propose a technique to achieve such knowledge transfer in cases where agent trajectories conta…
Optimizes random forest inference by defining step order to maximize accuracy.
problem Limited inference time in resource-constrained systems.
method Designs anytime random forest algorithm on step granularity, proposing optimal step order.
result Backward Squirrel Order performs nearly as well as the optimal step order.