Adaptive scheduling improves multilingual neural machine translation models.
problem Training models on multiple tasks with uniform or proportional sampling leads to poor performance trade-offs.
method Exploring non-adaptive and adaptive task scheduling, including implicit schedules.
result Adaptive schedules improve model performance for low-resource tasks without negatively affecting high-resource tasks.
Adaptive batch size schedules improve language model training efficiency and generalization.
problem Dilemma of choosing batch sizes in large-scale model training.
method General-purpose adaptive batch size schedules compatible with data and model parallelism.
result Adaptive batch size schedules outperform constant batch sizes and heuristic warmup schedules.
Optimal learning rate schedules derived for various tasks.
problem Inadequate learning rate schedules in practice compared to theory.
method Refined analysis of learning rate schedules for optimization algorithms.
result Derives new problem-adaptive learning rate schedules.
New method reduces Gibbs partition function estimation complexity.
problem Estimating partition functions of Gibbs distributions.
method Doubly-adaptive MCMC with adaptive cooling schedule and mean estimator.
result Outperforms state-of-the-art algorithms in computational complexity and robustness.
Study shows annealing with adaptive schedule reduces mode collapse in NFs for parameter estimation.
problem Mode collapse in normalizing flows for multimodal distributions.
method Annealing with an adaptive schedule based on effective sample size (ESS).
result Our approach reduces mode collapse and converges marginal likelihood faster than MCMC methods.
DAS3H optimizes skill-based spaced repetition schedules.
problem Optimizing adaptive and personalized spaced repetition schedules for skill-based learning.
method Developed a new student learning and forgetting model (DAS3H) that considers memory decay and multiple skills.
result DAS3H outperforms other models on real-world educational datasets.
ANT improves TS diffusion models by automatically determining noise schedules.
problem Suboptimal performance of TS diffusion models due to lack of domain-specific noise schedules.
method ANT proposes an adaptive noise schedule that automatically determines proper noise schedules for TS datasets based on their statistics.
result ANT achieves state-of-the-art performance on various TS tasks, including forecasting, refinement, and generation.
Adaptive learning rate improves model training flexibility.
problem Limited flexibility of hand-designed learning rate schedules.
method Reinforcement learning to automatically learn adaptive learning rates.
result Auto-learned learning rate controller achieves better test results.
This paper optimizes diffusion schedules for better sampling from data distributions.
problem Choosing an optimal discretization schedule for denoising diffusion models.
method Adaptive algorithm that selects an optimal schedule based on a work cost measure.
result The learned schedule recovers and outperforms manually tuned schedules.
Auto-Ensemble automates deep learning model ensembling with adaptive learning rate scheduling.
problem Difficulty in collecting diverse and accurate deep learning models through single training.
method Auto-Ensemble collects model checkpoints and uses adaptive learning rate scheduling to ensemble them.
result Ensembled models converge to various local optima, improving performance on few-shot learning.
New adaptive scheduler improves SAM for better model training.
problem Training machine learning models requires selecting a learning rate, which is often difficult and time-consuming.
method Derive Polyak schedulers tailored to SAM-style updates, proving linear convergence for strongly convex objectives and an O(1/T) rate for convex objectives.
result Polyak schedulers achieve comparable or better performance than tuned SAM baselines, reducing the need for learning-rate tuning.
Adaptive model scheduling boosts data labeling efficiency.
problem Efficiently labeling diverse data with limited resources.
method Adaptive Model Scheduling framework using deep reinforcement learning and heuristic algorithms.
result 53% reduction in execution time with no loss of labels.
This paper proposes a system-agnostic policy for dynamic scheduling.
problem Dynamic scheduling in changing systems is challenging due to system-specific optimal policies.
method Descriptive policy that learns a system-agnostic scheduling principle.
result System-agnostic meta-learning enables adaptation to unseen system characteristics.
Graph neural networks improve online planner selection for AI tasks.
problem Selecting the best planner for AI tasks.
method Graph Neural Network (GNN) for planner selection and adaptive scheduling.
result Improved planner selection and scheduling for cost-optimal planning.
Study separates learning rate effects from adaptive gradient methods.
problem Understanding the impact of learning rates on neural network training.
method Introduced a 'grafting' experiment to isolate learning rate effects.
result Many existing beliefs about adaptive gradient methods may be incorrect.
Untuned warmup of Adam performs similarly to RAdam in practical settings.
problem The stability of adaptive optimization algorithms like Adam requires a warmup schedule.
method Provided alternative explanation for warmup necessity based on update term magnitude.
result Untuned warmup of Adam is as effective as RAdam in typical practical settings.
Proposes an automatic cyclical scheduling for gradient-based discrete sampling.
problem Gradient-based sampling in high-dimensional models can get stuck in local modes.
method Cyclical step size and balancing schedules with automatic hyperparameter tuning.
result Proves non-asymptotic convergence and inference guarantees for general discrete distributions.
AdaPID optimizes diffusion-based samplers by dynamically adjusting schedules.
problem Optimizing the intermediate-time dynamics in diffusion-based samplers.
method Develops a time-varying stiffness schedule using Piece-Wise-Constant (PWC) parametrizations and a hierarchical refinement approach.
result QoS-driven PWC schedules consistently improve sampling fidelity and accuracy.
Deep learning approach for efficient IoT task scheduling in MEC networks.
problem Minimizing task latency in IoT users with large-scale MEC systems.
method Stacked auto-encoder for data compression, adaptive simulated annealing, experience replay.
result Near-optimal performance with significantly reduced computational time.
Seesaw optimizes training by balancing learning rate and batch size, accelerating model pretraining.
problem Optimizing training efficiency for large language models with adaptive optimizers.
method Develops a principled framework for batch-size scheduling, introducing Seesaw which multiplies learning rate by 1/√2 and doubles batch size.
result Empirically, Seesaw reduces wall-clock time by approximately 36% compared to cosine decay, matching theoretical limits.
AdaS adapts SGD learning rate based on knowledge gain metrics.
problem Empirical step-size selection in SGD optimization lacks consistency and insight.
method Introduces AdaS algorithm that adapts SGD learning rate based on knowledge gain metrics.
result AdaS outperforms existing adaptive learning methods in convergence and generalization.
New method adapts DLMs to intrinsic data dependence without prior knowledge.
problem Understanding how unmasking schedules affect DLM generation quality.
method Adapts unmasking schedule to target data distribution's dependence structure.
result Sampling convergence guarantees improve for low-complexity distributions.
AdaAnn optimizes annealing for efficient probability density approximation.
problem Efficiently approximating complex probability distributions with multiple modes.
method AdaAnn is an adaptive annealing scheduler that adjusts temperature increments based on KL divergence.
result AdaAnn improves computational efficiency in variational inference and parameter estimation.
Deployment of emerging technologies and rapid change in industries has created a lot of risk for initiating the new projects. Many techniques and suggestions have been introduced but still lack the gap from various prospective. This paper proposes a reliable project scheduling approach. The objectives of project schedu…
This work formalizes guidance in diffusion models and introduces a stochastic control framework.
problem Lack of a solid theoretical foundation for guidance scheduling in diffusion models.
method Introduces a stochastic optimal control framework to cast guidance scheduling as an adaptive optimization problem.
result Establishes a principled foundation for more effective guidance in diffusion models.
Meta-SAGE improves deep RL scalability for CO tasks by adapting pre-trained models to larger-scale problems.
problem Improving scalability of deep reinforcement learning models for combinatorial optimization tasks.
method Meta-SAGE combines a scale meta-learner and scheduled adaptation with guided exploration to adjust model parameters for larger-scale problems.
result Meta-SAGE outperforms previous methods and significantly improves scalability in CO tasks.
AdaLoss optimizes adaptive learning rates for efficient convergence in various models.
problem Efficiently optimizing adaptive learning rates for gradient descent methods.
method AdaLoss uses loss function information to dynamically adjust step sizes.
result AdaLoss achieves linear convergence in linear regression and robust global convergence in neural networks.
Proposes CDTD, a diffusion model for mixed-type tabular data.
problem Adapting diffusion models to mixed-type tabular data.
method Score matching and score interpolation for continuous features, adaptive noise schedules for categorical features.
result Consistently outperforms state-of-the-art models in mixed-type tabular data.
MLR-SNet learns flexible LR schedules for diverse tasks.
problem Adapting LR schedules for non-convex optimization problems.
method Parameterizes LR schedules with an explicit mapping formulation.
result Meta-learned LR schedules are transferable and adaptable.
New method learns optimal variance schedule for diffusion models.
problem Diffusion models' sensitivity to variance schedule.
method Probabilistic conditioning, learning schedule during training.
result Comparable or superior results in super-resolution microscopy and quantitative phase imaging.
Bayesian method optimizes rescheduling for multipurpose batch processes with incomplete look-ahead information.
problem Optimizing rescheduling for multipurpose batch processes under incomplete look-ahead information.
method Proposes a Bayesian dynamic scheduling method that learns from disturbances and updates schedules online.
result Achieves statistically better long-term costs and system nervousness compared to existing periodic rescheduling strategies.
The paper optimizes dynamic scheduling for ring architectures in deep learning training.
problem Optimizing deep learning training times with ring architectures.
method Formulated a non-convex, non-linear, NP-hard integer programming problem and developed a doubling heuristic.
result Dynamic scheduling can significantly reduce job completion times in ring architectures.
A heuristic minimizes tardy jobs' total weight on single-machine scheduling.
problem Minimizing tardy jobs' total weight on single-machine scheduling.
method Data-driven heuristic combining machine learning and problem-specific characteristics.
result Significantly outperforms state-of-the-art in optimality gap and adaptability.
CProp scales learning rate based on past gradient direction.
problem Adaptive learning rate scheduling during training.
method Gradient scaling method based on past gradient conformity.
result Significant gain in training speed on SGD and adaptive methods.
GALA adapts learning rates online by aligning gradients, improving deep learning model performance.
problem Fine-tuning learning rates for deep learning models requires extensive grid search.
method GALA dynamically adjusts learning rates by tracking gradient alignment and local curvature.
result GALA produces a flexible, adaptive learning rate schedule that increases when gradients align.
Bayesian optimisation for dynamically adjusting learning rates in machine learning models.
problem Dynamic adjustment of learning rates schedules in machine learning models.
method Probabilistic model based on latent Gaussian processes and auto-/regressive formulation.
result Flexibly adjusts learning rates schedules to abrupt changes of behaviours.
Novel Bayesian optimization framework improves portfolio management stability and efficiency.
problem Stable and sample-efficient optimization for black-box portfolio models under limited observation budgets.
method TPE-AS framework with adaptive scheduling and importance sampling.
result Demonstrated effectiveness across four backtest settings with three distinct models.
Cyber-physical systems, such as mobile robots, must respond adaptively to dynamic operating conditions. Effective operation of these systems requires that sensing and actuation tasks are performed in a timely manner. Additionally, execution of mission specific tasks such as imaging a room must be balanced against the n…
This work sets lower bounds on the number of score queries needed for diffusion sampling.
problem Establishing information-theoretic limits on the number of score evaluations required for diffusion sampling.
method Proving lower bounds on the number of adaptive score queries needed for sampling.
result Any sampling algorithm requires at least \(\widetilde{\Omega}(\sqrt{d})\) adaptive score queries for \(d\)-dimensional distributions.
CR-AIS improves AIS efficiency by constant rate annealing.
problem Efficiently sample from intractable distributions.
method Constant rate annealing schedule for AIS.
result CR-AIS outperforms existing Adaptive AIS methods.
ResQ uses reinforcement learning to optimize rescue efforts in disaster scenarios.
problem Optimizing emergency response in urban disaster zones.
method Heuristic multi-agent reinforcement learning scheduling algorithm.
result ResQ outperforms state-of-the-art methods in rescue efficiency.
Adaptive RL optimizes testing resource allocation for dynamic software environments.
problem Optimizing resource allocation for evolving software testing environments.
method Integrates Q-learning with hybrid reward design for sequential decision-making.
result Consistently outperforms static and optimization-based baselines in simulation studies.
SALSA automatically adjusts learning rates in stochastic gradient methods.
problem Automatic adjustment of learning rates in stochastic gradient methods.
method SALSA uses a line-search procedure to gradually increase the learning rate, then a statistical test to decrease it.
result SALSA matches the performance of best hand-tuned learning rate schedules in deep learning tasks.
AIHT improves online high-dimensional quantile regression by separating support discovery and refinement.
problem Online high-dimensional quantile regression with structural sparsity.
method Adaptive Iterative Hard Thresholding (AIHT) alternates stochastic updates with adaptive hard-thresholding steps.
result AIHT achieves logarithmic regret for the sliding-window objective in high-dimensional settings.
Generative models improve for multiscale scientific data with new noise and interpolation techniques.
problem Numerical challenges in generating high-fidelity samples for multiscale scientific data.
method Design of noise distributions and interpolation schedules in function space to ensure Lipschitz regularity and finite noise roughness.
result Scale-adaptive noise and interpolation schedules improve numerical efficiency and fidelity of generated samples.
Manifold Markov chain Monte Carlo algorithms have been introduced to sample more effectively from challenging target densities exhibiting multiple modes or strong correlations. Such algorithms exploit the local geometry of the parameter space, thus enabling chains to achieve a faster convergence rate when measured in n…
BLiE optimizes hyperparameters with theoretical guarantees and superior performance.
problem Hyperparameter optimization in machine learning.
method Lipschitz bandit approach exploiting Lipschitz continuity.
result BLiE finds ε-optimal hyperparameters with theoretical complexity.
Deep learning model classifies tweets for disaster rescue scheduling.
problem Efficiently processing and categorizing tweets for disaster rescue.
method Combining attention-based Bi-directional LSTM and CNN with pre-trained word vectors for classification and feature engineering.
result Proposed model outperforms other methods in Precision, Recall, F1-score, and Accuracy.