Automates resource-efficient CNN design for IoT.
problem Designing custom CNNs for IoT applications is impractical and resource-intensive.
method Automated synthesis of resource scalable CNNs from an existing optimized baseline CNN.
result Synthesized CNNs are resource-efficient and competitive in accuracy.
A scalable parallel BO method for asynchronous settings.
problem Expensive-to-evaluate problems in machine learning.
method Simple and scalable Bayesian optimization method for asynchronous parallel settings.
result Demonstrated promising performance on benchmark functions and hyperparameter optimization.
Scalable3-BO tackles scalability issues in Bayesian optimization for big data and high dimensions.
problem Bayesian optimization scalability issues in big data and high dimensions.
method Sparse Gaussian process, random embedding, asynchronous parallelization.
result Scalable3-BO framework optimizes high-dimensional problems with 1 million data points and 10,000 dimensions.
This paper proposes a decentralized reinforcement learning method for multi-agent resource allocation.
problem Allocating heterogeneous resources among multiple agents in a decentralized manner.
method Liquid-Graph-Time Clustering-IPPO, integrating dynamic cluster consensus.
result LGTC-IPPO achieves more stable rewards, better coordination, and robust performance.
Optimizes data power control in cell-free networks for better spectral efficiency.
problem Maximizing overall spectral efficiency in cell-free networks with multi-objective optimisation.
method Applied scalable multi-objective Bayesian optimisation to solve convergence-time limitations.
result Improved radio resource management in cell-free networks.
We present MorphNet, an approach to automate the design of neural network structures. MorphNet iteratively shrinks and expands a network, shrinking via a resource-weighted sparsifying regularizer on activations and expanding via a uniform multiplicative factor on all layers. In contrast to previous approaches, our meth…
Unified framework for scalable black-box optimization.
problem Expensive black-box evaluations in scientific and engineering domains.
method Integrates active learning, multi-armed bandits, and distributed computing.
result Consistently outperforms state-of-the-art black-box optimizers.
ESAC combines genetic methods with RL to improve scalability and efficiency.
problem Combining genetic scalability with RL's data efficiency and optimal control.
method Combines Evolution Strategies (ES) with Soft Actor-Critic (SAC) to enable skill transfer and reduce hyperparameter sensitivity.
result Demonstrates improved performance and sample efficiency in challenging tasks.
Datasets are growing not just in size but in complexity, creating a demand for rich models and quantification of uncertainty. Bayesian methods are an excellent fit for this demand, but scaling Bayesian inference is a challenge. In response to this challenge, there has been considerable recent work based on varying assu…
HardCoRe-NAS finds fitting neural networks adhering to hard resource constraints.
problem Finding fitting neural networks that adhere to hard resource constraints.
method Accurate formulation of resource requirement and scalable search method.
result HardCoRe-NAS generates state-of-the-art architectures strictly satisfying hard resource constraints.
A scalable system learns acoustic models from 1 Million hours of untranscribed audio.
problem Learning acoustic models from large, untranscribed audio datasets.
method Semi-supervised learning with a student/teacher learning paradigm, focusing on the data and model pipelines.
result Relative accuracy improvements of 10-20% in noisy conditions, with no extensive hyper-parameter tuning.
Simple policy outperforms complex ones in cloud auto-scaling.
problem Predicting resource scaling for large-scale cloud applications with limited deployment throughput.
method Probabilistic workload forecast for auto-scaling decisions based on risk aversion.
result The proposed policy outperforms sophisticated and simple benchmark policies in real-world and synthetic data.
Analyzes self-attention in recurrent networks, proving it mitigates vanishing gradients.
problem Vanishing gradients in recurrent networks when capturing long-term dependencies.
method Formal analysis of self-attention's effect on gradient propagation, proposing a relevancy screening mechanism.
result Self-attention mitigates vanishing gradients in recurrent networks, providing guarantees.
Explosive growth in data and availability of cheap computing resources have sparked increasing interest in Big learning, an emerging subfield that studies scalable machine learning algorithms, systems, and applications with Big Data. Bayesian methods represent one important class of statistic methods for machine learni…
Unified framework for planning under uncertainty using variational inference.
problem Planning under uncertainty with separate objectives for exploration and exploitation.
method Variational inference on a generative model augmented with priors.
result EFE-based planning emerges as variational inference, enabling scalable, resource-aware policies.
HydroNets use river structure to improve hydrologic predictions.
problem Scalable and accurate hydrologic models are needed for climate change impacts.
method HydroNets are deep neural networks that incorporate river network structure.
result HydroNets improve predictions with fewer data, especially at longer horizons.
BINAS improves neural architecture search with interpretable models.
problem Finding efficient neural networks under resource constraints.
method BINAS uses a bilinear formulation for accuracy and resource estimation, with a scalable search method.
result BINAS generates comparable or better architectures than state-of-the-art methods, while strictly satisfying resource constraints.
AIDEL improves scalability of learned indexes in storage systems.
problem Expensive retraining and heavy inter-model dependency in learned indexes limit scalability.
method Construct different linear regression models based on data distribution, making them independent and easier to partition.
result AIDEL improves insertion performance by about 2x and comparable lookup performance.
Improved Thompson Sampling algorithms for bandits with tighter regret bounds.
problem Efficient and adaptive algorithms for stochastic bandits with bounded rewards.
method Proposed two parameterized Thompson Sampling-based algorithms: TS-MA-α and TS-TD-α.
result Achieved O(Kln^(α+1)(T)/Δ) regret bound, improving scalability and resource allocation.
Adaptive Bayesian Optimization for resource-constrained experiments with switching costs.
problem Sequential experimental design with varying costs for changing design variables.
method Adapted batch algorithms to sequential problem, proposing cost-aware and cost-ignorant methods.
result Cost-aware algorithm outperforms tuned process-constrained algorithms in all settings considered.
A new method optimizes complex engineering designs under uncertainty efficiently.
problem Optimizing large, uncertain engineering designs with limited resources.
method Multi-level informed optimization via decomposed Kriging.
result Significantly faster and more accurate optimization compared to state-of-the-art methods.
Optimal Volt/VAR control rules are designed using deep neural networks.
problem Designing optimal Volt/VAR control rules for distributed energy resources (DERs).
method Formulate optimal rule design as a bilevel program, then reformulate it as training a deep neural network (DNN). Use proximal gradient descent (PGD) iterations to emulate Volt/VAR dynamics.
result The proposed solution can be adapted to single/multi-phase feeders and achieves enhanced steady-state voltage profiles.
Two novel algorithms improve scalability and robustness of spectral clustering for large datasets.
problem Scalability and robustness of spectral clustering for large-scale datasets.
method Ultra-scalable spectral clustering (U-SPEC) and ultra-scalable ensemble clustering (U-SENC) algorithms.
result Robust and efficient clustering of ten-million-level datasets on a PC.
Graph neural networks optimize radio resource management policies for wireless networks.
problem Optimizing user selection and power control in wireless networks with fairness constraints.
method Formulated as a Lagrangian dual problem, RRM policies are parameterized by a GNN architecture trained on channel conditions.
result The method achieves superior tradeoff between average and 5th percentile rates, demonstrating fairness.
Federated Learning helps IoT devices learn without central servers.
problem Resource constraints in IoT devices hinder traditional ML approaches.
method Local training of IoT devices using global models.
result Federated Learning can be applied to IoT devices with varying resource capabilities.
We propose to prune a random forest (RF) for resource-constrained prediction. We first construct a RF and then prune it to optimize expected feature cost & accuracy. We pose pruning RFs as a novel 0-1 integer program with linear constraints that encourages feature re-use. We establish total unimodularity of the constra…
This paper tackles resource allocation in the Lightning Network using DRL.
problem Complex combinatorial problem of node selection and resource allocation in the Lightning Network.
method Attention-based Deep Reinforcement Learning framework.
result Improved resource allocation leads to better performance and decentralization in the LN.
TrueLearn uses Bayesian algorithms to match learners with educational resources.
problem Matching lifelong learners to open educational resources efficiently and effectively.
method TrueLearn uses a text ontology and Bayesian strategies to match learners with educational resources based on their background knowledge and material novelty.
result TrueLearn algorithms show promise in building an effective educational recommendation system.
VQC-MLPNet combines quantum and classical elements for scalable quantum machine learning.
problem Challenges in expressivity, trainability, and noise resilience of VQCs.
method Hybrid architecture with a VQC generating weights for a classical MLP during training.
result Improved expressivity, trainability, and robustness compared to standalone quantum or hybrid approaches.
A deep RL framework optimizes resource allocation in wireless networks.
problem Optimizing resource allocation and interference in wireless networks.
method Multi-agent deep reinforcement learning for distributed decision-making.
result Our approach outperforms decentralized and centralized baselines in terms of user rates.
LC-LMT makes certified defense more efficient and robust.
problem Scalable certified defense methods are resource-intensive and only achieve small robustness.
method LC-LMT, a lightweight Lipschitz margin training method.
result LC-LMT achieves higher robustness and ε-robustness more efficiently. Scalable Gaussian processes with latent Kronecker structure for large datasets.
problem Limited scalability of Gaussian processes for large datasets.
method Leveraging latent Kronecker structure, projecting kernel matrix onto latent Kronecker product, using iterative linear system solvers and pathwise conditioning.
result Outperforms state-of-the-art sparse and variational GPs on real-world datasets with up to five million examples.
Paper speeds up large foundation models for time series data.
problem Resource-intensive foundation models limit accessibility.
method Dimensionality reduction techniques, including PCA and neural network adapters.
result Up to 10x speedup and 4.5x more datasets fit on a single GPU.
This work simplifies data valuation for LLMs using Shapley value computation.
problem How to fairly distribute benefits from training superior LLMs with multiple data owners' resources.
method We leverage the specific mathematical structure of DPO to enable scalable Shapley value computation for LLMs.
result We demonstrate that Shapley value computation for LLMs trained with DPO is significantly simplified.
Paper introduces MTGP for engineering tasks with sparse data.
problem Challenges of data sparsity and varying task correlations in engineering.
method Multi-Task Gaussian Processes (MTGP) framework.
result Improves predictive performance and reduces computational costs.
The paper tackles scalability issues in Graph Representation Learning.
problem Prohibitive time and memory complexities in Graph Representation Learning.
method Leveraging the K-Core Decomposition property of Graphs to reduce time and memory consumption.
result Proposed techniques significantly reduce computational resources without compromising embedding quality.
A scalable portfolio approach speeds up Bayesian optimization for noisy functions.
problem Efficiently selecting multiple designs in parallel for noisy, expensive black-box optimization.
method A portfolio approach that balances exploration and exploitation, using a scalable allocation strategy.
result Significant speed improvements over existing methods, with similar or better performance.
SCAN divides deep neural networks into shallow classifiers for efficient deployment.
problem Explosive growth in storage and computation limits deep neural networks on edge devices.
method SCAN divides networks into shallow classifiers, uses attention modules and knowledge distillation, and employs a threshold-controlled scalable inference mechanism.
result SCAN achieves significant performance gain on CIFAR100 and ImageNet without hyper-parameter adjustments.
Mobile V-MoEs scale down ViTs for resource-constrained vision tasks.
problem Scaling down Vision Transformers for resource-constrained applications.
method Sparse Mixture-of-Experts (MoEs) applied to entire images, with a stable training procedure.
result Mobile V-MoEs achieve better performance-efficiency trade-offs than dense ViTs.
In AI research and industry, machine learning is the most widely used tool. One of the most important machine learning algorithms is Gradient Boosting Decision Tree, i.e. GBDT whose training process needs considerable computational resources and time. To shorten GBDT training time, many works tried to apply GBDT on Par…
CNN improves medium-range temperature forecasts with limited resources.
problem Limited computational resources for high-resolution temperature forecasts.
method CNN post-processing with ensemble NWP models for bias correction and spatial downscaling.
result High-resolution (5-km) surface temperature forecasts with lead times up to 5.5 days.
Snorkel DryBell uses weak supervision to speed up machine learning model development.
problem Costly label data in machine learning applications.
method Flexible ingestion of organizational knowledge, cross-feature production serving, scalable execution.
result Comparable quality to hand-labeled models, 52% performance improvement on average.
Unified scoring model improves efficiency and performance across multiple tasks.
problem Efficient and resource-efficient automated scoring for diverse tasks.
method Knowledge-distilled multi-task Mixture-of-Experts (MoE) approach.
result Comparable performance to task-specific models with significantly less storage and training resources.
Bayesian network approach for efficient cooperative MARL.
problem Leveraging inter-agent coupling information for scalable MARL algorithms.
method Modeling cooperative MARL via Bayesian networks, identifying value dependency sets, proposing P-DTDE paradigm.
result P-DTDE policy gradient estimator has lower total variance than CTDE.
Scalable method for regionalizing and extracting temporal patterns from time series data.
problem Static spatial snapshots and ad hoc regularization limit effective spatial analysis and resource management.
method Minimum description length principle for fully nonparametric spatial partitioning and time series archetypes.
result Accurately recovers planted regional structure and drivers in synthetic and empirical data.
Accelerates data loading in deep neural network training by 30x.
problem Data loading is a bottleneck in deep neural network training.
method Locality-aware data loading method using software caches.
result More than 30x speedup in data loading.
Survey of knowledge distillation for model compression.
problem Deploying large deep models on resource-limited devices.
method Knowledge distillation from large teacher models to small student models.
result Effective model compression and acceleration.
We propose a practical and scalable Gaussian process model for large-scale nonlinear probabilistic regression. Our mixture-of-experts model is conceptually simple and hierarchically recombines computations for an overall approximation of a full Gaussian process. Closed-form and distributed computations allow for effici…