This paper proposes a method to select project schedules with the lowest risk.
problem Selecting schedules that meet project deadlines while minimizing risk.
method Integrating aleatory uncertainty into project scheduling to quantify and compare risks.
result Proposes a method to select schedules with the lowest risk.
New research shows many batch selection methods for training work just as well as full batch training.
problem Finding optimal batch selection methods for training.
method Analysis of mini-batch Gradient Descent (GD) and Stochastic GD (SGD) with various batch selection rules.
result All mini-batch schedules, including deterministic ones, generalize optimally for smooth Lipschitz-convex/nonconvex/strongly-convex loss functions.
Annealed importance sampling (AIS) is a common algorithm to estimate partition functions of useful stochastic models. One important problem for obtaining accurate AIS estimates is the selection of an annealing schedule. Conventionally, an annealing schedule is often determined heuristically or is simply set as a linear…
AdaPID optimizes diffusion-based samplers by dynamically adjusting schedules.
problem Optimizing the intermediate-time dynamics in diffusion-based samplers.
method Develops a time-varying stiffness schedule using Piece-Wise-Constant (PWC) parametrizations and a hierarchical refinement approach.
result QoS-driven PWC schedules consistently improve sampling fidelity and accuracy.
Study stability of selective SSMs with discontinuous gating.
problem Challenges in stability analysis of selective SSMs with discontinuous gating.
method Passivity and Input-to-State Stability (ISS) analysis of continuous-time selective SSMs.
result Derivation of sufficient conditions for global ISS with respect to the port input.
Automated planning is one of the foundational areas of AI. Since no single planner can work well for all tasks and domains, portfolio-based techniques have become increasingly popular in recent years. In particular, deep learning emerges as a promising methodology for online planner selection. Owing to the recent devel…
This paper optimizes diffusion schedules for better sampling from data distributions.
problem Choosing an optimal discretization schedule for denoising diffusion models.
method Adaptive algorithm that selects an optimal schedule based on a work cost measure.
result The learned schedule recovers and outperforms manually tuned schedules.
Recent research on Software-Defined Networking (SDN) strongly promotes the adoption of distributed controller architectures. To achieve high network performance, designing a scheduling function (SF) to properly dispatch requests from each switch to suitable controllers becomes critical. However, existing literature ten…
Two distinct phases of deep learning training improve model generalization.
problem Optimization and generalization in deep learning models.
method Isolating two phases of training: large-step and small-step, and tailoring training algorithms to each.
result Training algorithms optimized for each phase significantly simplify learning rate schedules.
A heuristic minimizes tardy jobs' total weight on single-machine scheduling.
problem Minimizing tardy jobs' total weight on single-machine scheduling.
method Data-driven heuristic combining machine learning and problem-specific characteristics.
result Significantly outperforms state-of-the-art in optimality gap and adaptability.
Robust Transformer-Based One-Step Stock Index Forecasting via Shifted Data Augmentation
problem Robust stock index forecasting
method Modified Transformer architecture with Shifted Data Augmentation
result Best performance on benchmark datasets
New adaptive scheduler improves SAM for better model training.
problem Training machine learning models requires selecting a learning rate, which is often difficult and time-consuming.
method Derive Polyak schedulers tailored to SAM-style updates, proving linear convergence for strongly convex objectives and an O(1/T) rate for convex objectives.
result Polyak schedulers achieve comparable or better performance than tuned SAM baselines, reducing the need for learning-rate tuning.
Dropout schedules can be optimized to significantly reduce model test loss.
problem Improving model performance in neural networks.
method Developed a mean-field theory of dropout at the edge of chaos, proposing front-loaded dropout schedules.
result Front-loaded dropout schedules reduce test loss by 18-35% over constant dropout.
This work formalizes guidance in diffusion models and introduces a stochastic control framework.
problem Lack of a solid theoretical foundation for guidance scheduling in diffusion models.
method Introduces a stochastic optimal control framework to cast guidance scheduling as an adaptive optimization problem.
result Establishes a principled foundation for more effective guidance in diffusion models.
EI-MTD defends edge intelligence against adversarial attacks with dynamic scheduling.
problem Adversarial attacks on edge intelligence models.
method EI-MTD uses differential knowledge distillation to create robust member models and a dynamic scheduling policy based on a Bayesian Stackelberg game.
result EI-MTD effectively protects edge intelligence from black-box adversarial attacks.
Hybrid classical-quantum framework optimizes portfolio rebalancing with reduced transaction costs.
problem Optimizing portfolio rebalancing with reduced transaction costs and lookahead bias.
method Combining Ledoit-Wolf shrinkage covariance estimation, hierarchical correlation clustering, entropy-regularised Genetic Algorithm, minimum-variance and equal-weight benchmarks, QUBO formulation, and QAOA for solving the combinatorial optimisation problem.
result GA + QAOA strategy outperforms classical methods with reduced rebalances and transaction costs.
Developing an Agent-Based Model to Mitigate Adverse Selection in Uniswap v3 Liquidity Providers
problem Adverse selection in Uniswap v3 liquidity providers
method Agent-Based Model incorporating blockchain microstructure and volatility dynamics
result Dynamic fee schedules improve hedged Profit and Loss for liquidity providers
GraSP-RL uses graph neural networks to improve job shop scheduling.
problem Capturing machine-unit-job sequence relationships and managing state space growth.
method Graph neural networks for feature extraction, reinforcement learning for decision-making, decentralized optimization.
result GraSP-RL outperforms existing methods in minimizing makespan for complex production environments.
GOLS-I automatically determines learning rates for various neural network training algorithms.
problem Adapting learning rates in stochastic training algorithms for neural networks.
method Gradient-Only Line Search (GOLS-I) for automatically setting learning rates.
result GOLS-I learning rate schedules are competitive with manually tuned rates across multiple algorithms, architectures, datasets, and loss functions.
New method schedules learning rate without stopping time, outperforming existing methods.
problem Learning rate schedules that require stopping time are outperformed.
method Schedule-Free approach that avoids stopping time and introduces no additional hyper-parameters.
result Exhibits state-of-the-art performance across various problems.
AdaAnn optimizes annealing for efficient probability density approximation.
problem Efficiently approximating complex probability distributions with multiple modes.
method AdaAnn is an adaptive annealing scheduler that adjusts temperature increments based on KL divergence.
result AdaAnn improves computational efficiency in variational inference and parameter estimation.
Optimizes financial auditor schedules to reduce time and costs.
problem Efficiently scheduling financial auditors with multiple constraints.
method Used Integer Linear Programming and compared two exact formulations.
result Multi-commodity network flow formulation is 24 times faster.
ScheduleFree+ improves large language model training without schedules or learning rates.
problem Scaling up Schedule-Free Learning to large language models.
method Learning-rate-free and schedule-free method for training large language models.
result ScheduleFree+ outperforms SOTA schedules by 31% at 1000 tokens per parameter.
Cosine schedule is optimal for discrete diffusion models.
problem Choosing the best discretization schedule for diffusion models.
method Optimized using Fisher-Rao geometry.
result Cosine schedule is Fisher-Rao optimal.
Optimal learning rate schedules derived for various tasks.
problem Inadequate learning rate schedules in practice compared to theory.
method Refined analysis of learning rate schedules for optimization algorithms.
result Derives new problem-adaptive learning rate schedules.
The paper optimizes interpolation schedules in generative models to improve sampling accuracy.
problem Improving sampling accuracy in generative models with fewer resources.
method Minimizing the averaged squared Lipschitzness of the drift field, using transfer formulas.
result Designed schedules yield more accurate fine-scale statistics at fixed integrator budget.
New method converts and optimizes sampling schedules for generative models.
problem Optimizing sampling schedules for generative models like flows and diffusions.
method Unified framework for stochastic interpolants, including point mass schedules.
result Demonstrated efficient generation of images with fewer steps.
This paper proposes a system-agnostic policy for dynamic scheduling.
problem Dynamic scheduling in changing systems is challenging due to system-specific optimal policies.
method Descriptive policy that learns a system-agnostic scheduling principle.
result System-agnostic meta-learning enables adaptation to unseen system characteristics.
The paper presents a multi-power law for predicting loss curves across different learning rate schedules.
problem Understanding and optimizing the relationship between model performance and hyperparameters, especially learning rates.
method Proposes a multi-power law that combines power laws based on the sum of learning rates and additional laws for loss reduction due to decay.
result The multi-power law accurately predicts loss curves for unseen learning rate schedules and finds a schedule that outperforms cosine learning rate.
Study Nash competition among dealers quoting prices to clients with unknown trading motives.
problem Adverse selection and inventory costs in dealer-client interactions.
method Analyzes one-shot Nash competition with unknown client type and inventory constraints.
result Unique symmetric Nash equilibrium exists and can be characterized by a nonlinear ODE.
Current clinical practice to monitor patients' health follows either regular or heuristic-based lab test (e.g. blood test) scheduling. Such practice not only gives rise to redundant measurements accruing cost, but may even lead to unnecessary patient discomfort. From the computational perspective, heuristic-based test …
More and more companies have deployed machine learning (ML) clusters, where deep learning (DL) models are trained for providing various AI-driven services. Efficient resource scheduling is essential for maximal utilization of expensive DL clusters. Existing cluster schedulers either are agnostic to ML workload characte…
This paper solves optimal market making for multiple goods, including bundling, under adverse selection.
problem Designing optimal market making mechanisms for multiple goods and adverse selection.
method Formulated as an optimal transport problem with geometric constraints, using differentiable economics.
result Optimal market making mechanisms can exploit bundling to improve prices and accept payments in kind.
Optimal learning rate schedules for SGD in changing data distributions.
problem Minimizing regret in online learning with changing data distributions.
method Characterized optimal schedules for linear regression, proposed schedules for general convex and non-convex losses, and defined a notion of regret for non-convex losses.
result Upper and lower bounds for regret with constants for convex losses, and an upper bound on total expected regret for non-convex losses.
Enhances multi-project scheduling with multiple priority rules.
problem Resource allocation in multi-project scheduling with limited time and resources.
method Simulation-based approach using composite priority rules.
result Increased probability of finding schedules with shortest duration.
Learning-rate schedules for large models match optimization theory closely, leading to better training.
problem Improving training of large models with optimal learning rates.
method Used a bound from non-smooth convex optimization theory to match learning-rate schedules with practical benefits.
result Extending the learning-rate schedule with optimal learning-rate and transferring it across schedules improves model training.
The study introduces anytime learning schedules for large language models without fixed horizons.
problem Training large language models without knowing the total training horizon.
method Theoretical analysis and weight averaging to create anytime learning schedules.
result Theoretical and empirical evidence shows that weight averaging with simple step sizes can achieve comparable final loss to well-tuned cosine schedules.
With online calendar services gaining popularity worldwide, calendar data has become one of the richest context sources for understanding human behavior. However, event scheduling is still time-consuming even with the development of online calendars. Although machine learning based event scheduling models have automate…
This work uses reinforcement learning to optimize task scheduling and execution in a dynamic multi-agent warehouse environment.
problem Optimizing task scheduling and execution in a dynamic multi-agent warehouse environment with limited observability.
method Deep reinforcement learning to solve both high-level scheduling and low-level multi-agent execution problems.
result Demonstrates the effectiveness of reinforcement learning in optimizing task scheduling and execution in a dynamic multi-agent environment.
AdaS adapts SGD learning rate based on knowledge gain metrics.
problem Empirical step-size selection in SGD optimization lacks consistency and insight.
method Introduces AdaS algorithm that adapts SGD learning rate based on knowledge gain metrics.
result AdaS outperforms existing adaptive learning methods in convergence and generalization.
Optimizes diffusion processes for target distributions.
problem Efficiently generating target distributions from point masses.
method Stochastic interpolant framework with conditional expectation drift.
result Optimal diffusion coefficient minimizes path-space KL divergence.
Momentum is a widely used technique for gradient-based optimizers in deep learning. In this paper, we propose a decaying momentum (\textsc{Demon}) rule. We conduct the first large-scale empirical analysis of momentum decay methods for modern neural network optimization, in addition to the most popular learning rate dec…
Schedule-free SGD is optimal for nonconvex optimization problems.
problem Nonconvex optimization in neural networks.
method Developed a general framework for online-to-nonconvex conversion, which converts schedule-free SGD into an effective nonconvex optimization algorithm.
result Schedule-free SGD achieves optimal iteration complexity for nonsmooth, nonconvex optimization problems.
The paper analyzes the InfoNCE loss under different temperature schedules using Langevin dynamics.
problem Understanding the dynamics of InfoNCE loss under fixed versus annealed temperature schedules.
method Modeling embedding evolution under Langevin dynamics on a compact Riemannian manifold, with theoretical guarantees for convergence.
result Slow logarithmic inverse-temperature schedules ensure convergence to globally optimal representations, while faster schedules risk suboptimal minima.
The scheduling of films is a major problem for the movie theatre exhibition business. The problem is two-fold: movie exhibitors ideally would like to schedule films to screens in their various locations to maximize attendance and revenue, but would also like to schedule these films such that neighbouring theatre locati…
Paper optimizes FL communication efficiency with stochastic optimization.
problem Intermittent connectivity and non-i.i.d. data in FL.
method Convergence analysis of non-convex loss functions, stochastic optimization for client selection and power allocation.
result Significant reduction in communication time compared to random participation.
MLR-SNet learns flexible LR schedules for diverse tasks.
problem Adapting LR schedules for non-convex optimization problems.
method Parameterizes LR schedules with an explicit mapping formulation.
result Meta-learned LR schedules are transferable and adaptable.
This study shows how DDPM can be represented by the OU process.
problem Designing optimal noise schedules for DDPM.
method Formal equivalence between DDPM and OU process, heuristic designs based on Fisher Information.
result Fisher-Information-motivated schedule corresponds to cosine noise schedule.