This paper identifies principles for scalable Bayesian inference.
problem Scaling Bayesian inference for growing datasets and complexity.
method Review of existing work on MCMC and variational approximation.
result Characterization of successful scalable inference procedures.
The paper explores how to balance accuracy and computational resources in learning to rank.
problem Balancing accuracy and computational resources in learning to rank for large datasets.
method Developed a hierarchy of rank-breaking mechanisms to trade off data points for computational resources.
result Theoretical guarantees on the proposed rank-breaking mechanisms provide trade-offs between accuracy and computational resources.
This study uses LSTM and SARIMA models to forecast CPU usage in cloud computing.
problem Predicting future CPU usage in cloud computing centers.
method Used LSTM and SARIMA models to forecast CPU usage over short and long time scales.
result SARIMA model performed better for long-term predictions but LSTM was more robust.
Novel framework reduces slow nodes' impact on distributed computing.
problem Mitigating slow nodes (stragglers) in distributed computing.
method Developed mathematical understanding and analyzed convergence behavior of encoded optimization algorithms.
result Characterized trade-offs between various parameters in encoded optimization.
The paper evaluates index-based allocation policies using data from randomized control trials.
problem Evaluating index-based allocation policies in resource-scarce scenarios.
method Using data from randomized control trials, the paper introduces an efficient estimator and methods for computing asymptotically correct confidence intervals.
result Valid statistical conclusions can be drawn for index-based allocation policies.
New algorithm optimizes resource allocation in non-stationary networks.
problem Optimal resource allocation in non-stationary RMABs is computationally hard.
method Sliding-Window Online Whittle (SW-Whittle) policy for non-stationary transition kernels.
result Sub-linear dynamic regret achieved with unknown variation budget.
Paper introduces MTGP for engineering tasks with sparse data.
problem Challenges of data sparsity and varying task correlations in engineering.
method Multi-Task Gaussian Processes (MTGP) framework.
result Improves predictive performance and reduces computational costs.
Distill-Net creates efficient CNNs for IoT by distilling complex models.
problem Efficient inference of deep CNNs on resource-constrained IoT platforms.
method Application-specific distillation of deep CNNs.
result Efficient inference on resource-constrained platforms with high accuracy.
Generative profiling improves real-time task timing for varied resource contexts.
problem Inaccurate task timing analysis for complex hardware architectures.
method Nonparametric, conditional multi-marginal Schrödinger Bridge (MSB) formulation for synthesizing context-dependent timing profiles.
result Maximum likelihood accurate execution profiles for unseen resource contexts.
The paper uses WeChat data to map urban cultural resource needs.
problem Optimizing cultural resource allocation in cities.
method Data-driven framework using WeChat user check-ins and temporal LDA model.
result Identifies urban regions lacking cultural resources.
BoostTransformer uses boosting to improve transformer efficiency and accuracy.
problem Heavy computational resources and hyperparameter tuning in transformer architectures.
method Augments transformers with boosting principles through subgrid token selection and importance-weighted sampling, incorporating a least square boosting objective directly into the pipeline.
result BoostTransformer demonstrates faster convergence and higher accuracy compared to standard transformers.
Teaches deep learning to statisticians.
problem Statisticians lack expertise in deep learning.
method Developed a program and taught DL to statistics graduate students.
result Provided tips and resources for teaching DL.
Unified framework for scalable black-box optimization.
problem Expensive black-box evaluations in scientific and engineering domains.
method Integrates active learning, multi-armed bandits, and distributed computing.
result Consistently outperforms state-of-the-art black-box optimizers.
Meta-DRL improves resource allocation in O-RAN networks.
problem Dynamic resource allocation in O-RAN networks.
method Meta Deep Reinforcement Learning (Meta-DRL) inspired by MAML.
result 19.8% improvement in network management performance.
Method learns software resource usage from snapshots.
problem Challenges in learning time-varying, correlated resource usage.
method Graph structured Schrödinger bridge problem for nonparametric learning.
result Predicts most-likely resource distributions.
New CNN layer selects important channels to improve model capacity.
problem Improving model capacity under resource constraints.
method Selective allocation of channels in convolutional layers.
result New layer allows new optima that generalize better.
Deep learning predicts hypoxemia from SpO2 data better than anesthesiologists.
problem Improving hypoxemia prediction for better patient outcomes.
method Trained deep learning model on SpO2 data to predict hypoxemia.
result Deep learning model outperforms anesthesiologists in predicting hypoxemia.
New framework extends linear scaling for kernel machines on GPUs.
problem Limited parallelization of large batch training on GPUs.
method Developed analytical framework for classical kernel machines.
result Extended linear scaling for faster training on GPUs.
RiskNet predicts penalties in unreliable communication networks using GNNs.
problem Predicting penalties in networks with unreliable resources.
method Graph Neural Network (GNN) based approach trained on random graphs.
result Precisely models penalties across various network topologies.
Reduces deep learning training data for faster testing.
problem Resource-intensive deep learning training with full data sets.
method Evaluated different training set reduction methods.
result Training set reduction is useful in resource-constrained environments.
This work explores feature learning tradeoffs in neural networks.
problem Resource tradeoffs in neural feature learning.
method Theoretical and experimental investigation of offline sparse parity learning.
result Width improves sample efficiency in sparse feature learning.
MLP outperforms other deep learning models for groundwater prediction.
problem Accurate groundwater level predictions under changing climatic conditions.
method Optimized hyperparameters of deep learning models using surrogate models.
result MLP performs best in terms of prediction accuracy and time-to-solution.
FOCuS detects changes in mean from high-frequency data efficiently.
problem Detecting changes in high-frequency data with limited resources.
method FOCuS algorithm that runs multiple window sizes and change sizes simultaneously.
result FOCuS achieves state-of-the-art performance in detecting anomalies.
Predicts and classifies computational jobs for efficient resource allocation in cloud centers.
problem Efficiently scheduling and assigning resources to computational jobs in cloud centers.
method Applied LSTM neural network for job arrival prediction and BIRCH clustering for job classification.
result Improved accuracy in predicting and classifying computational jobs compared to existing methods.
ARCO-BO optimizes multi-agent design under heterogeneity, improving efficiency and performance.
problem Heterogeneous multi-agent optimization challenges in resource use and information sharing.
method ARCO-BO integrates a consensus mechanism, budget-aware sampling, and partial input sharing for heterogeneous design spaces.
result ARCO-BO outperforms independent and collaborative BO methods in complex multi-agent settings.
Square loss performs comparably or better than cross-entropy in neural architectures for various tasks.
problem The superiority of cross-entropy loss over square loss in classification tasks is debated.
method Comparison of several neural architectures on NLP, ASR, and computer vision datasets using both loss functions.
result Square loss often produces better results in the majority of tasks, especially in NLP and ASR.
ATA optimizes task allocation in distributed machine learning.
problem Greedy task allocation leads to inefficiencies in distributed machine learning.
method Adaptive Task Allocation (ATA) adapts to unknown computation time distributions.
result ATA identifies optimal task allocation without prior knowledge of computation times.
This paper tackles efficient resource control in IoT edge computing using deep reinforcement learning.
problem Efficient allocation and scheduling of limited resources in IoT edge computing systems.
method Formulated as a CTMDP model, used deep reinforcement learning (RL) to approximate value functions and solve the MDP problem.
result Significant performance improvement over baseline algorithms and RL algorithms based on other architectures.
PePR scores assess DL model performance per resource unit, promoting smaller, more efficient models.
problem Limited access to large-scale resources hinders medical image analysis research.
method Introduced PePR score to measure DL model performance per resource unit.
result Small-scale, specialized models outperform large-scale models in resource-constrained settings.
PASHA optimizes model tuning for large datasets with limited resources.
problem Expensive HPO and NAS for large datasets.
method Dynamic resource allocation approach.
result Significantly reduces computational resources while maintaining performance.
Adaptive RL optimizes testing resource allocation for dynamic software environments.
problem Optimizing resource allocation for evolving software testing environments.
method Integrates Q-learning with hybrid reward design for sequential decision-making.
result Consistently outperforms static and optimization-based baselines in simulation studies.
Adaptive sampling method optimizes DNN compression for resource-constrained platforms.
problem Efficiently compressing DNNs for resource-constrained platforms with high accuracy.
method Adaptive sampling using genetic algorithm-inspired operations to optimize hyperparameters.
result Adaptive sampling outperforms rule-based and reinforcement learning methods in compression rate and accuracy.
Parallelizes word2vec for multi-core and many-core architectures.
problem Efficiently parallelize word2vec for modern multi-core/many-core architectures.
method Proposes HogBatch, improving reuse of data structures through minibatching and negative sample sharing, allowing matrix multiply operations.
result Demonstrates strong scalability up to 32 nodes and near linear scaling across cores and nodes.
New variational approach to deep learning via gradient descent.
problem Non-robustness and poor out-of-distribution generalization in deep learning.
method Regularize variational neural networks using gradient descent's implicit bias.
result Strong in- and out-of-distribution performance achieved without additional hyperparameter tuning.
FAVANO improves federated learning for resource-constrained environments.
problem Asynchronous communication in federated learning leads to bias and scalability issues.
method FAVANO is a novel asynchronous federated learning framework for resource-constrained environments.
result FAVANO outperforms existing methods on standard benchmarks.
Paper proposes a method to train neural networks incrementally using cloud computing despite disconnections and resource outages.
problem Frequent disconnections and resource outages in cloud computing and local machines hinder deep learning model training.
method Introduces an incremental learning framework that allows continuous training of neural networks even with interruptions.
result Demonstrates that incremental learning can maintain progress and train neural networks effectively despite interruptions.
Paper presents runtime-throttleable neural networks for resource-constrained devices.
problem Resource constraints in edge computing platforms.
method Runtime-throttleable neural networks using block-level gating.
result Smooth performance throttling with minimal accuracy loss.
Efficient neural networks for resource-constrained systems.
problem Reducing resource consumption in machine learning models for embedded systems.
method Quantized neural networks, network pruning, structural efficiency.
result Finding good trade-offs between resource efficiency and prediction quality is challenging.
Optimal resource allocation improves feature classification accuracy in noisy conditions.
problem Improving feature classification accuracy when features are noisy and resource allocation affects noise magnitude.
method Developed a method for computing optimal resource allocation in various scenarios.
result Non-uniform resource allocation can significantly enhance classification performance.
Federated Learning over wireless networks tackles resource allocation challenges.
problem Heterogeneity in UE data and resources in Federated Learning.
method Proposed FL algorithm for heterogeneous data, convergence rate analysis, and resource allocation optimization.
result The proposed algorithm outperforms vanilla FedAvg in convergence rate and accuracy.
NASIB adapts NAS to varying computation resources efficiently.
problem Constrained computation resources in enterprise environments.
method Adapts exploration vs. exploitation trade-off and uses Superkernels.
result Searches over a larger space with similar accuracy in less time.
It takes skill to build a meaningful predictive model even with the abundance of implementations of modern machine learning algorithms and readily available computing resources. Building a model becomes challenging if hundreds of terabytes of data need to be processed to produce the training data set. In a digital adve…
We analyze computational limits of modern Hopfield models based on pattern norms.
problem Understanding the efficiency of modern Hopfield models from a fine-grained complexity perspective.
method Fine-grained complexity analysis and upper bound criterion for pattern norms.
result Below a specific norm threshold, efficient variants of modern Hopfield models exist.
New graph attention operators improve performance and reduce computational costs.
problem Excessive computational resources in graph attention operators.
method Proposed hGAO and cGAO using hard and channel-wise attention mechanisms.
result Improved performance and computational savings with new operators.
This work tackles resource allocation in asynchronous and stochastic systems.
problem Distributed resource allocation in asynchronous and stochastic settings.
method Approximate stochastic primal-dual approach with asynchronous updates.
result The Asynchronous stochastic Primal-Dual (Asyn-PD) algorithm converges to the saddle point solution at a rate of O(1/t). PREREQ learns concept prerequisites from online educational resources.
problem Inferring prerequisite relations between educational concepts.
method PREREQ uses latent representations of concepts from Pairwise Latent Dirichlet Allocation and a Siamese neural network to learn from course prerequisites and labeled data.
result PREREQ outperforms state-of-the-art approaches and can learn from less data.
OpTorch optimizes deep learning for resource-limited environments.
problem Resource constraints in deep learning training.
method Optimized deep learning pipelines in training time and memory.
result Achieved similar accuracy to existing libraries with reduced memory usage.
Survey on discovering causal relationships from data.
problem Discover causal relationships from data.
method Modern, continuous optimization methods for structure learning.
result Survey of methods and resources for structure discovery.