Adaptive approach for cost-effective prediction models.
problem Resource-constrained prediction with accuracy and cost trade-offs.
method Bottom-up strategy to learn gating and prediction models.
result Method achieves higher accuracy with lower cost compared to state-of-the-art.
Dynamic model selection for cost-effective prediction.
problem Resource-constrained prediction with accuracy and cost trade-offs.
method Bottom-up recursive scheme for gating and prediction models.
result Method achieves higher accuracy for the same cost on benchmark datasets.
We propose to prune a random forest (RF) for resource-constrained prediction. We first construct a RF and then prune it to optimize expected feature cost & accuracy. We pose pruning RFs as a novel 0-1 integer program with linear constraints that encourages feature re-use. We establish total unimodularity of the constra…
FedZKT enables resource-constrained devices to participate in federated learning with heterogeneous models.
problem Inequality in resource allocation hinders participation from resource-constrained devices in federated learning.
method Zero-shot knowledge transfer through a server-assigned distillation process.
result FedZKT effectively transfers knowledge across heterogeneous on-device models without requiring comparable local training efforts.
Paper compresses RNNs for resource-constrained devices.
problem Difficulty deploying RNNs on resource-constrained devices.
method Uses Kronecker product (KP) to compress RNN layers.
result KP compresses RNN layers by 16-38x with minimal accuracy loss.
Offline RL tackles resource-constrained online deployment with improved policy transfer.
problem Training policies with limited online features using a rich offline dataset.
method Introduce a policy transfer algorithm that first trains a teacher agent with full offline features and then transfers knowledge to a student agent with limited online features.
result Consistent improvement in performance over baseline methods on resource-constrained datasets.
FastGRNN improves RNN accuracy while drastically reducing model size.
problem Inaccurate training and inefficient prediction in RNNs.
method FastGRNN uses a residual connection and gate to achieve state-of-the-art accuracy with a much smaller model.
result FastGRNN achieves state-of-the-art accuracy with models up to 35x smaller than existing RNNs.
FAVANO improves federated learning for resource-constrained environments.
problem Asynchronous communication in federated learning leads to bias and scalability issues.
method FAVANO is a novel asynchronous federated learning framework for resource-constrained environments.
result FAVANO outperforms existing methods on standard benchmarks.
This paper defines resource-constrained classifier performance and its impact on algorithm choice.
problem Classification tasks in resource-constrained settings where actions are limited.
method Defines resource-constrained classifier performance and discusses gains and lift.
result Gains and lift metrics can lead to different algorithm choices.
Federated Learning helps IoT devices learn without central servers.
problem Resource constraints in IoT devices hinder traditional ML approaches.
method Local training of IoT devices using global models.
result Federated Learning can be applied to IoT devices with varying resource capabilities.
Energy savings for DNN inference on resource-constrained devices.
problem Energy efficiency in deep learning inference for constrained devices.
method Efficiently searches through equivalent DNN graphs to find the one with the least execution cost.
result Achieves 24% energy savings with minimal performance impact.
Binarized CNNs improve GPU inference efficiency on resource-constrained devices.
problem Efficient inference on resource-constrained devices for image classification.
method Binarization of weights and computations in CNNs, implemented on GPUs.
result 7.4X speedup with 4.4% accuracy loss on embedded GPU platforms.
Improved distributed learning with reduced communication costs.
problem Efficient communication in resource-constrained environments for distributed learning.
method Proposed a cost-effective partial communication protocol.
result Communication cost is reduced to O ( log T ) O(\log T) O ( log T ) , improving significantly on full communication. We study the problem of structured prediction under test-time budget constraints. We propose a novel approach applicable to a wide range of structured prediction problems in computer vision and natural language processing. Our approach seeks to adaptively generate computationally costly features during test-time in ord…
SWIFT improves time series forecasting on edge devices with wavelet decomposition.
problem Efficient time series forecasting on resource-constrained devices.
method Wavelet decomposition, cross-band fusion, and shared linear mapping.
result SWIFT achieves state-of-the-art performance on multiple datasets.
Adaptive Bayesian Optimization for resource-constrained experiments with switching costs.
problem Sequential experimental design with varying costs for changing design variables.
method Adapted batch algorithms to sequential problem, proposing cost-aware and cost-ignorant methods.
result Cost-aware algorithm outperforms tuned process-constrained algorithms in all settings considered.
Paper proposes DP-PASGD for efficient, private IoT learning.
problem Privacy and resource constraints in IoT.
method Differentially private federated learning (DP-PASGD) for resource-constrained IoT.
result DP-PASGD achieves efficient training while maintaining privacy.
Algorithm improves resource allocation for food outreach to homeless.
problem Resource-constrained outreach for homeless individuals and food rescue.
method Thompson sampling with Markov chain recovery (via Stein variational gradient descent) for partially-observed episodic restless bandits.
result Significantly outperforms baselines in both organizations' problems.
Adaptive sampling method optimizes DNN compression for resource-constrained platforms.
problem Efficiently compressing DNNs for resource-constrained platforms with high accuracy.
method Adaptive sampling using genetic algorithm-inspired operations to optimize hyperparameters.
result Adaptive sampling outperforms rule-based and reinforcement learning methods in compression rate and accuracy.
Hydra boosts efficiency for long-context reasoning in resource-constrained settings.
problem Quadratic complexity of transformers limits long-context reasoning in resource-constrained systems.
method Hydra uses a modular architecture with adaptive routing between sparse global attention, mixture-of-experts, and dual memories.
result Hydra achieves significant throughput and accuracy improvements for long-context reasoning.
Review of efficient neural networks for TinyML on resource-constrained devices.
problem Resource constraints on ultra-low power MCUs for deep learning models.
method Model compression, quantization, low-rank factorization, model pruning, hardware acceleration, algorithm-architecture co-design.
result Optimized neural network architectures for minimal resource utilization on MCUs.
DBQ quantizes lightweight networks efficiently for resource-constrained devices.
problem High computational and storage complexity of deep neural networks on resource-constrained devices.
method A differentiable non-uniform quantizer that can be mapped onto efficient ternary-based dot product engines.
result Achieves state-of-the-art results with minimal training overhead and best accuracy-complexity trade-off.
Graph-based RKD improves knowledge distillation for resource-constrained systems.
problem Efficiently distill deep learning knowledge for resource-constrained systems.
method Graph-based relative knowledge distillation (RKD) to capture latent space geometry.
result The proposed method achieves better accuracy with the same resource budget.
AgileNet improves few-shot learning for resource-constrained devices.
problem Efficient few-shot learning on resource-constrained devices.
method Lightweight dictionary-based few-shot learning approach.
result AgileNet achieves superior accuracy compared to prior arts.
Distill-Net creates efficient CNNs for IoT by distilling complex models.
problem Efficient inference of deep CNNs on resource-constrained IoT platforms.
method Application-specific distillation of deep CNNs.
result Efficient inference on resource-constrained platforms with high accuracy.
Machine learning improves air pollution predictions in low-cost sensor networks.
problem Invalid predictions from low-cost sensors in resource-constrained countries.
method Probabilistic machine learning to model sensor calibration as distributions or Gaussian processes.
result Improves air pollution predictions and decision-making processes.
This paper optimizes AI inference on edge devices with reduced communication and computation costs.
problem Efficiently performing AI inference on resource-constrained edge devices with reduced communication and computation costs.
method A three-step framework for effective inference: model split point selection, communication-aware model compression, and task-oriented encoding of intermediate features.
result Our proposed framework achieves a better trade-off and significantly reduces inference latency compared to baseline methods.
Paper presents runtime-throttleable neural networks for resource-constrained devices.
problem Resource constraints in edge computing platforms.
method Runtime-throttleable neural networks using block-level gating.
result Smooth performance throttling with minimal accuracy loss.
No floating point, no multiplications, no problem! Training efficient networks for resource-constrained devices.
problem Designing efficient neural networks for resource-constrained devices without floating-point operations.
method Discretizing both in-network non-linearities and network weights to avoid floating-point and multiplication operations.
result Training networks without floating-point operations can achieve comparable performance to those using floating-point operations, with less memory usage.
Optimizes random forest inference by defining step order to maximize accuracy.
problem Limited inference time in resource-constrained systems.
method Designs anytime random forest algorithm on step granularity, proposing optimal step order.
result Backward Squirrel Order performs nearly as well as the optimal step order.
DQA efficiently quantizes deep neural network activations for resource-constrained devices.
problem Efficiently quantizing deep neural network activations for resource-constrained devices.
method DQA uses simple shifting-based operations and Huffman coding for sub-6-bit quantization.
result DQA achieves significantly better accuracy than direct quantization and state-of-the-art methods.
Mobile V-MoEs scale down ViTs for resource-constrained vision tasks.
problem Scaling down Vision Transformers for resource-constrained applications.
method Sparse Mixture-of-Experts (MoEs) applied to entire images, with a stable training procedure.
result Mobile V-MoEs achieve better performance-efficiency trade-offs than dense ViTs.
learn2mix trains neural nets faster by adjusting class proportions dynamically.
problem Training neural nets efficiently with limited resources and imbalanced classes.
method Adaptive class proportion adjustment during training.
result Neural nets trained with learn2mix converge faster than static methods.
A new method for growing neural networks by splitting neurons, improving efficiency.
problem Optimizing neural network structures, especially for lightweight architectures.
method A progressive training approach using steepest descent to adaptively grow and split neurons.
result The method provides a computationally efficient way to optimize neural network structures.
Proposes a bandit framework for dynamic user incentives.
problem Designing personalized incentives for users with evolving preferences.
method Combines greedy matching, UCB, and Markov chain theory.
result Algorithm provides theoretical regret bounds and practical examples.
EC2T creates sparse and ternary neural networks for resource-constrained devices.
problem Deploying deep neural networks on resource-constrained devices.
method Entropy-Constrained Trained Ternarization (EC2T) framework.
result EC2T creates sparse and ternary neural networks that are efficient in terms of storage and computation.
Efficient classifier with uncertainty bounds for safety-critical applications.
problem Lack of uncertainty bounds in high-accuracy classifiers for safety-critical tasks.
method Nadaraya-Watson estimator with frequentist bounds.
result Competitive accuracy and uncertainty bounds at reduced computational cost.
A new differentiable quantization method for neural networks.
problem Training neural networks for resource-constrained devices.
method Differentiable quantization procedure using categorical distributions over a quantization grid.
result The method can train networks that can be effectively discretized without loss of performance.
Study evaluates feature selection methods for emotion recognition in resource-constrained settings.
problem Reducing memory and computational requirements for emotion recognition in low-resource settings.
method Evaluation of three feature selection methods: ILFS, ReliefF, Fisher, and AFS.
result Smaller feature sets can achieve similar or better accuracy, reducing resource usage.
Novel LSTM network predicts pulsar timing residuals with few-shot data.
problem Predicting pulsar timing residuals with limited data.
method Long Short-Term Memory (LSTM) network optimized with model-agnostic meta-learning and particle swarm optimization.
result Robust generalization and accurate predictions across high-frequency test domains with minimal data.
Optimal query allocation improves extractive QA efficiency with LLMs.
problem Efficiency and reliability in extractive question answering with LLMs.
method Learning-to-Defer framework that allocates queries to specialized models.
result Enhanced answer reliability with reduced computational overhead.
Paper proposes efficient pruning method for neural networks.
problem Compressing deep neural networks for resource-constrained devices.
method Adaptive sparsity loss for budget-aware optimization during training.
result Demonstrated effectiveness on various architectures and datasets.
Paper uses imitation learning to create efficient insulin policies from MPC demonstrations.
problem Resource-constrained medical devices struggle with complex MPC optimizations and state estimation errors.
method Imitation learning of neural network policies from MPC-computed demonstrations, using Bayesian inference with Monte Carlo Dropout.
result Trained policies generalize well to different patient cohorts, outperforming traditional MPC with state estimation.
Study optimizes pricing under uncertainty and capacity constraints.
problem Optimizing pricing decisions under demand uncertainty and capacity constraints.
method Analyzes linear demand, stochastic noise, and finite capacity; uses certified demand forecasts and control variates.
result Certified demand forecasts reduce regret from O ( T ) O(\sqrt{T}) O ( T ) to O ( log T ) O(\log T) O ( log T ) under certain conditions. This paper proposes a method to automatically compress neural networks using Bayesian tensor decomposition.
problem Challenges in directly applying tensor compression in neural network training.
method Bayesian tensorized neural network with automatic rank selection.
result Produces significantly more compact neural networks (7.4x to 137x) directly from training.
This paper optimizes deep neural networks for resource-constrained devices.
problem Efficient deployment of deep neural networks on resource-constrained devices.
method Across-stack optimization of CNNs using weight pruning, channel pruning, quantization, and parallel execution.
result Comprehensive Pareto curves for trade-offs between accuracy, execution time, and memory space.
New method reduces deep learning complexity on IoT devices.
problem High computational complexity limits deep learning on IoT devices.
method Local quantization region for low-bit data representation.
result Models retain accuracy with reduced computational complexity.
Data aggregation improves HAC for resource-constrained systems.
problem Resource constraints in embedded systems limit HAC's applicability.
method Data aggregation with BETULA algorithm reduces memory and runtime requirements.
result HAC can be applied to large datasets on resource-constrained systems.