A hybrid strategy forecasts short-term loads using Warm-start Gradient Tree Boosting.
problem Lack of effective short-term load forecasting methods.
method Hybrid strategy integrating four different inference models: tree-based ensemble method Warm-start Gradient Tree Boosting (WGTB).
result Demonstrates effectiveness of hybrid strategy on real datasets.
Warm-start strategies speed up GP inference by 19x.
problem Efficient sequential inference in Gaussian processes.
method Three warm-start strategies exploiting smaller linear systems.
result Warm-starting achieves up to 19x speed-up in convergence.
New model uses pretrained biochemical language models to generate drug compounds.
problem Developing novel compounds targeting specific proteins.
method Exploits pretrained language models to initialize and fine-tune targeted molecule generation models.
result Warm-started models outperform baseline models, with one-stage strategy showing better generalization.
Accelerates optimal transport computation by 10x with spectral insights.
problem Exponential slow-down of convergence in Entropic Optimal Transport as regularization weakens.
method Spectral insights and spectral warm-start strategy to mitigate convergence issues.
result Faster convergence compared to the reference method Sinkhorn algorithm.
New algorithm speeds up HMC by generating a warm start in O(d^1/4) iterations.
problem Unclear how many iterations of HMC are needed for high-dimensional sampling.
method Developed a non-Metropolized HMC that generates a warm start in O(d^1/4) iterations, followed by Metropolized HMC.
result Final complexity of O(d^1/4) is the fastest algorithm for high-accuracy sampling under strong log-concavity assumptions.
Warm starts improve Gaussian process regression by up to 16x.
problem Optimizing hyperparameters for Gaussian processes.
method Iterative Gaussian processes with warm start optimization.
result Warm starts achieve the same results as conventional methods but significantly speed up computations.
New screening rules improve lasso model fitting efficiency.
problem Efficiently solving high-dimensional lasso problems.
method Look-ahead screening rules to discard predictors.
result Look-ahead screening rules outperform existing methods.
New method achieves optimal sample complexity without warm-start in bilevel optimization.
problem Optimizing smooth objective functions with fixed point constraints in meta-learning and equilibrium models.
method Fixed point iterations at lower-level and projected inexact gradient descent at upper-level.
result Achieves near optimal sample complexity O(ε−2) and ildeO(ε−1) samples. Reweighted ALPS improves sampling from multimodal distributions using warm start points.
problem Sampling from multimodal distributions is hard due to exponential mixing times.
method Introduces Reweighted ALPS, a modified Annealed Leap-Point Sampler that uses warm start points.
result First polynomial-time bound for Re-ALPS in a general setting, under a natural assumption.
We develop a framework for warm-starting Bayesian optimization, that reduces the solution time required to solve an optimization problem that is one in a sequence of related problems. This is useful when optimizing the output of a stochastic simulator that fails to provide derivative information, for which Bayesian opt…
WSD uses a deterministic model to accelerate diffusion-based sampling.
problem Slow refinement process in diffusion models.
method Warm-start model that predicts an informed prior conditioned on input context.
result Significantly reduces the number of diffusion steps required for realistic samples.
Warm starts improve variational quantum algorithms by avoiding barren plateaus.
problem Barren plateaus in variational quantum algorithms limit scaling.
method Exploring warm starts in iterative variational methods for quantum circuits.
result Warm starts can lead to substantial gradients in small regions, suggesting trainability.
A novel diffusion method for Bayesian posterior sampling with theoretical guarantees.
problem Efficiently sampling from complex posterior distributions in Bayesian inversion.
method Diffusion-based posterior sampling using Langevin dynamics and PnP framework.
result The method converges even for multi-modal posterior distributions with theoretical error bounds.
New bound shows variational algorithms may struggle with barren plateaus.
problem Barren plateaus in quantum loss landscapes.
method General bound on loss variance and gradient decay.
result Exponential decay of gradients in subregions of barren plateaus.
This paper speeds up iterative GP inference with warm starting.
problem Improving scalability of Gaussian process inference.
method Warm starting sequential posteriors using known solutions.
result Significant speed-ups and improved Bayesian optimisation performance.
In many real-world deployments of machine learning systems, data arrive piecemeal. These learning scenarios may be passive, where data arrive incrementally due to structural properties of the problem (e.g., daily financial data) or active, where samples are selected according to a measure of their quality (e.g., experi…
Enhances genetic programming for stock alpha discovery with warm start and structural constraints.
problem Overwhelming search space and computational burden in traditional genetic programming for alpha factor discovery.
method Proposes a new GP framework with warm start and structural constraints to enhance search performance and interpretability.
result Superior out-of-sample prediction results and higher portfolio returns compared to benchmarks.
Faster algorithm for sampling logconcave densities in high dimensions.
problem Cubic barrier in sampling logconcave densities from a cold start.
method Two key ingredients: weaker distance sampling and refined log-Sobolev inequality.
result First sub-cubic sampling algorithms for isotropic position.
Flow-VQE uses generative flows to optimize VQE efficiently.
problem Complex objective functions and expensive optimization in VQE.
method Generative normalizing flows with parameterized quantum circuits.
result Flow-VQE accelerates convergence and reduces circuit evaluations.
Improved log-concave sampling to O(d1/2) with warm starts.
problem Sampling from strongly log-concave distributions efficiently.
method Warm starts and discretized underdamped Langevin diffusion.
result Achieved O(d1/2) complexity for high-accuracy sampling. Paper presents algorithm for optimal job selection with dynamic scoring.
problem Optimal job assignment in a sequential selection process with dynamic scores.
method Developed using dynamic programming, with extensions for partial and no-information cases.
result Algorithm allows for optimal job assignment with limited information.
The paper speeds up hyperparameter optimisation in Gaussian processes.
problem Scaling hyperparameter optimisation to large datasets.
method Improvements to linear system solvers (pathwise gradient, warm starting, early stopping).
result Speed-ups of up to 72x and residual norm decreases of up to 7x.
Bayesian optimization (BO) is a model-based approach for gradient-free black-box function optimization. Typically, BO is powered by a Gaussian process (GP), whose algorithmic complexity is cubic in the number of evaluations. Hence, GP-based BO cannot leverage large amounts of past or related function evaluations, for e…
This work explores the ability of collective matrix factorization models in recommender systems to make predictions about users and items for which there is side information available but no feedback or interactions data, and proposes a new formulation with a faster cold-start prediction formula that can be used in rea…
Bayesian optimization (BO) is a successful methodology to optimize black-box functions that are expensive to evaluate. While traditional methods optimize each black-box function in isolation, there has been recent interest in speeding up BO by transferring knowledge across multiple related black-box functions. In this …
This paper analyzes EM algorithm for softmax mixture models in high dimensions.
problem Modeling heterogeneous populations choosing from multiple attributes.
method Comprehensive analysis of the EM algorithm for softmax mixture models (SMMs), proving identifiability and convergence.
result EM algorithm recovers mixture atoms at near-parametric rate under suitable initialization.
We present an affine-invariant random walk for drawing uniform random samples from a convex body K⊂Rn that uses maximum volume inscribed ellipsoids, known as John's ellipsoids, for the proposal distribution. Our algorithm makes steps using uniform sampling from the John's ellipsoid of the …
A new screening rule improves lasso solving speed.
problem Efficiently solving lasso problems with high correlation.
method Uses second-order information from the Hessian to screen predictors.
result Outperforms alternatives on simulated and real data.
We consider solving the ℓ1-regularized least-squares (ℓ1-LS) problem in the context of sparse recovery, for applications such as compressed sensing. The standard proximal gradient method, also known as iterative soft-thresholding when applied to this problem, has low computational cost per iteration but a r…
In this paper, we propose a Distributed Accumulated Newton Conjugate gradiEnt (DANCE) method in which sample size is gradually increasing to quickly obtain a solution whose empirical loss is under satisfactory statistical accuracy. Our proposed method is multistage in which the solution of a stage serves as a warm star…
Zigzag sampling algorithm efficiently samples from strongly log-concave distributions with low computational cost.
problem Sampling from strongly log-concave distributions efficiently and with low computational complexity.
method Zigzag sampling algorithm with warm start assumption, focusing on gradient evaluations.
result Achieves ε error in chi-square divergence with computational cost of O(κ²d^(1/2)(log(1/ε))^(3/2)) gradient evaluations.
This work presents an explicit-implicit procedure to compute a model predictive control (MPC) law with guarantees on recursive feasibility and asymptotic stability. The approach combines an offline-trained fully-connected neural network with an online primal active set solver. The neural network provides a control inpu…
A bilevel optimization method for reweighting data leads to suboptimal results.
problem Reweighting data for better model performance on a different distribution.
method Formalizes data reweighting as a bilevel optimization problem and analyzes the limitations of classical solvers.
result The final data weights are often very sparse, indicating suboptimal solutions.
Quantum circuit models learn better with specific initialization strategies.
problem Understanding and improving the optimization landscape of IQP-based generative models.
method Proved barren plateaus for random initialization, established lower bounds, and developed data-dependent initialization.
result Data-dependent initialization leads to faster convergence and better minimums.
In this paper, we aim to learn a low-dimensional Euclidean representation from a set of constraints of the form "item j is closer to item i than item k". Existing approaches for this "ordinal embedding" problem require expensive optimization procedures, which cannot scale to handle increasingly larger datasets. To addr…
Providing long-range forecasts is a fundamental challenge in time series modeling, which is only compounded by the challenge of having to form such forecasts when a time series has never previously been observed. The latter challenge is the time series version of the cold-start problem seen in recommender systems which…
We introduce a generic scheme for accelerating gradient-based optimization methods in the sense of Nesterov. The approach, called Catalyst, builds upon the inexact accelerated proximal point algorithm for minimizing a convex objective function, and consists of approximately solving a sequence of well-chosen auxiliary p…
We propose a general algorithmic framework for constrained matrix and tensor factorization, which is widely used in signal processing and machine learning. The new framework is a hybrid between alternating optimization (AO) and the alternating direction method of multipliers (ADMM): each matrix factor is updated in tur…
A new method combines multiple bounds and betting strategies for selective prediction, improving risk coverage in data-scarce settings.
problem Selective prediction with risk control in data-scarce domains.
method Combines concentration inequalities, multiple-testing corrections, and betting-based confidence sequences.
result Transfer-Informed Betting achieves tighter bounds and better coverage in data-scarce settings.
Adaptive SAA solves large-scale stochastic linear programs efficiently.
problem Solving large-scale two-stage stochastic linear programs.
method Iterative algorithm with adaptive sample size and warm starts.
result The algorithm converges to the true solution set with a probabilistic guarantee.
DWTS uses observational data to improve clinical trial efficiency.
problem Lack of definitive conclusions from randomized clinical trials due to insufficient patient cohorts and confounding biases.
method DWTS combines observational data with randomized clinical trials using Doubly Debiased LASSO (DDL) to identify reliable covariates.
result DWTS reduces cumulative regret in clinical trials compared to standard methods.
Study efficient power iteration for tensor models, proving convergence under specific conditions.
problem Simultaneous alternating power iteration for fixed-order asymmetric rank-one spiked tensor models.
method Finite-iteration local theory, geometrically decaying transient, fixed-order multilinear noise event, warm-start mechanism.
result Convergence to the unique informative local fixed point under specific conditions.
DiffOPF solves multi-valued OPF problems by sampling from system history.
problem Multi-valued and non-convex OPF problems due to system parameter variability.
method DiffOPF treats OPF as a conditional sampling problem, learning from historical data.
result DiffOPF enables statistically credible warm starts with favorable cost and constraint satisfaction trade-offs.
Deep reinforcement learning has been successful in a variety of tasks, such as game playing and robotic manipulation. However, attempting to learn \textit{tabula rasa} disregards the logical structure of many domains as well as the wealth of readily available knowledge from domain experts that could help "warm start" t…
Bayesian online learning algorithm for one-pass data, achieving frequentist validity and uncertainty quantification.
problem Theoretical limitations in Bayesian online learning, especially in the one-pass setting.
method Proposed a new Bayesian online learning algorithm with a warm-start phase for the one-pass regime, establishing convergence rates and valid uncertainty quantification.
result The sequentially updated posterior attains optimal convergence rates and valid uncertainty quantification without diverging mini-batch sample sizes.
Study improves Bayesian optimisation with ensemble transfer learning.
problem Improving sample efficiency in Bayesian optimisation of expensive functions.
method Empirical analysis of ensemble-based transfer learning methods and pipeline components.
result Two components (warm start initialisation and positive weight constraint) improve transfer learning Bayesian optimisation performance.
A two-stage GPR framework with automatic kernel search and subsampling improves prediction accuracy and efficiency.
problem Inaccurate predictions due to misspecified mean and kernel functions in Gaussian Process Regression.
method Two-stage GPR, automatic kernel search, subsampling for hyperparameter initialization.
result Competitive or better performance compared to full dataset training, robust on real-world datasets.
New method uses adiabatic principles to improve ground-state preparation in quantum computing.
problem Challenges in variational training of complex energy landscapes.
method Iterative Hamiltonian deformation complemented with adiabatic principles.
result Consistent convergence to target ground state through sequence of intermediate problems.