Novel method constructs covariance functions for Bayesian optimisation.
problem Bayesian optimisation with existing knowledge.
method Uses m-kernels to convert existing covariance functions to problem-specific ones.
result Constructs covariance functions matching the problem at hand.
Novel algorithms for entropic optimal transport from an optimisation perspective.
problem Solving the entropic-regularised optimal transport problem.
method Developed novel methods inspired by mirror descent, solving semi-dual problems or non-convex constrained problems over joint distributions.
result Non-asymptotic rates of convergence for the proposed methods under minimal assumptions.
Improves MCMC and VI methods by optimising simulation bias.
problem Limitations of MCMC and VI methods in practice.
method Hybrid method combining optimisation with MCMC simulation.
result Produces low-biased samples with better balance between bias and efficiency.
Deep sum-product networks learn faster than shallow models.
problem The speed of parameter optimization in sum-product networks.
method Theoretical analysis and empirical experiments on overparameterized sum-product networks.
result Gradient-based optimization in deep sum-product networks is equivalent to gradient ascent with adaptive and time-varying learning rates and additional momentum terms.
Paper proposes a new Bayesian optimisation method to handle aleatoric uncertainty.
problem Representing and minimizing aleatoric noise in Bayesian optimisation.
method Heteroscedastic Gaussian process (GP) surrogate model with AEI and ANPEI acquisition functions.
result Improved performance on toy problems and real-world datasets compared to homoscedastic methods.
Bayesian optimisation is improved by incorporating expert prior through space warping.
problem Cold start phase in expensive function optimisation.
method Prior distribution warps the search space around high probability regions of function optimum.
result Improves optimisation performance through acquisition agnostic approach.
Geometric methods solve sampling, optimisation, inference, and adaptive decision-making.
problem Efficient solutions for sampling, optimisation, inference, and adaptive decision-making.
method Derive algorithms exploiting geometric structures of Hamiltonian systems, Hilbertian subspaces, and information geometry.
result Wide range of geometric theories emerge in these fields, enabling efficient solutions.
AVATAR uses a surrogate model to quickly evaluate ML pipelines, saving time and resources.
problem Time-consuming evaluation of ML pipelines limits exploration of complex models.
method AVATAR employs a surrogate model to assess pipeline validity without execution.
result AVATAR accelerates ML pipeline evaluation, improving efficiency in complex scenarios.
Paper improves robustness of Optimisation Monte Carlo method.
problem Overconfident approximations in Optimisation Monte Carlo.
method Robust Optimisation Monte Carlo (Robust OMC) method.
result Corrects overconfident approximations by collapsing regions of similar likelihood.
FraPPE efficiently identifies Pareto optimal arms in multi-objective bandits.
problem Efficiently identifying Pareto optimal arms in multi-objective bandits with confidence.
method Deriving structural properties and using Frank-Wolfe optimisation to solve the maxmin optimisation problem.
result FraPPE achieves optimal sample complexity and identifies the exact Pareto set.
Automated framework optimizes DNN deployment on Arm CPUs.
problem Lack of globally optimised DNN deployment across software levels.
method Reinforcement Learning search for automated design space exploration.
result Up to 4x improvement in performance and 2x reduction in memory.
A new method solves general elliptical mixture models using approximate Wasserstein distance.
problem Estimating general elliptical mixture models with robust and stable optimization.
method Adaptive optimisation method on a statistical manifold defined under an approximate Wasserstein distance.
result The method provides a stable and superior optimum for EMMs, improving upon Kullback-Leibler divergence.
This study uses continuous-time analysis to understand how momentum affects the optimisation of diagonal linear networks.
problem The effect of momentum on the optimisation trajectory of gradient descent.
method Leveraging a continuous-time approach to analyze momentum gradient descent with step size γ and momentum parameter β.
result Small values of λ help recover sparse solutions in overparametrised regression settings.
Unified framework for lifted training and inversion of neural networks.
problem Challenges in gradient-based training of deep neural networks.
method Unified framework encapsulating various lifted training strategies.
result Unified framework improves training landscape and stability.
Hybrid classical-quantum framework optimizes portfolio rebalancing with reduced transaction costs.
problem Optimizing portfolio rebalancing with reduced transaction costs and lookahead bias.
method Combining Ledoit-Wolf shrinkage covariance estimation, hierarchical correlation clustering, entropy-regularised Genetic Algorithm, minimum-variance and equal-weight benchmarks, QUBO formulation, and QAOA for solving the combinatorial optimisation problem.
result GA + QAOA strategy outperforms classical methods with reduced rebalances and transaction costs.
New method accelerates Bayesian imaging using Langevin sampling.
problem Bayesian inference in imaging inverse problems with convex geometry.
method Stochastic relaxed proximal-point iteration targeting posterior distribution.
result Accelerated convergence for κ-strongly log-concave targets. In this paper we develop proximal methods for statistical learning. Proximal point algorithms are useful in statistics and machine learning for obtaining optimization solutions for composite functions. Our approach exploits closed-form solutions of proximal operators and envelope representations based on the Moreau, Fo…
This paper explores optimising acquisition functions in Bayesian optimisation.
problem Optimising acquisition functions in Bayesian optimisation is challenging due to their non-convex nature.
method The authors derive compositional forms for acquisition functions and use them to recast maximisation as a compositional optimisation problem.
result The compositional approach to maximising acquisition functions shows empirical advantages across various tasks.
New framework converts multi-objective to single-objective optimisation.
problem Solving multi-objective optimisation problems.
method Formalises scalarisation into mathematical framework, uses R2 utilities.
result R2 utilities are monotone and submodular, optimised by greedy algorithms.
Unified Bayesian Optimisation for mixed variables improves performance.
problem Efficient optimisation of problems with both categorical and continuous variables.
method Derive value proposals from the Expected Improvement criterion to optimise both categorical and continuous variables under a single acquisition metric.
result Unified approach significantly outperforms existing methods across mixed-variable tasks.
A new method learns priors for Bayesian optimisation to improve performance.
problem Bayesian optimisation tasks often assume strong similarity, which is violated in many cases.
method Replace strong similarity assumption with shape similarity, learn priors for hyperparameters.
result PLeBO and prior transfer find good inputs in fewer evaluations.
Bayesian optimisation for expensive experiments with shape prior.
problem Expensive experiments with time-varying control variables.
method Developed a novel Bayesian optimisation framework using Bernstein polynomial basis and dynamic polynomial degree adjustment.
result Demonstrated effectiveness on polymer fibre design and learning rate optimisation.
Develops a fast Bayesian optimisation method that reduces computational overhead.
problem Computational inefficiency and restrictive kernel choices in information-theoretic Bayesian optimisation.
method FITBO method that avoids sampling the global minimizer and allows for more flexible kernel choices.
result Demonstrates that FITBO inherits performance from information-theoretic Bayesian optimisation but is faster.
Study improves Bayesian optimisation with ensemble transfer learning.
problem Improving sample efficiency in Bayesian optimisation of expensive functions.
method Empirical analysis of ensemble-based transfer learning methods and pipeline components.
result Two components (warm start initialisation and positive weight constraint) improve transfer learning Bayesian optimisation performance.
AdamZ optimiser improves neural network training efficiency.
problem Challenges in optimisation like overshooting and stagnation.
method Dynamic learning rate adjustment based on overshoot and stagnation factors.
result Consistently minimises loss function, improving model performance.
Wind farm layout optimisation tackles space constraints with Bayesian multi-objective approach.
problem Optimizing wind farm layout due to limited space and conflicting objectives.
method Set-based multi-objective Bayesian optimisation using Gaussian process.
result Demonstrates potential of set-based Bayesian multi-objective optimisation for wind farm layout.
Efficient Gauss-Newton optimisation for neural networks.
problem Optimizing deep learning models efficiently and effectively.
method Block-diagonal approximation of Gauss-Newton matrix for neural networks.
result The resulting algorithm outperforms state-of-the-art first-order optimisation methods.
Book covers tools for zeroth-order convex optimisation.
problem Zeroth-order convex optimisation.
method Cutting plane methods, interior point methods, continuous exponential weights, gradient descent, online Newton step.
result Improved existing bounds and algorithms.
A new method optimises problems with both continuous and categorical inputs.
problem Optimising black-box problems with mixed continuous and categorical inputs.
method Continuous and Categorical Bayesian Optimisation (CoCaBO) combining multi-armed bandits and Bayesian optimisation.
result CoCaBO outperforms existing methods on synthetic and real-world tasks.
Bayesian optimisation tackles high-dimensional categorical and mixed search spaces.
problem Bayesian optimisation on high-dimensional categorical and mixed search spaces is challenging.
method Combining local optimisation with a tailored kernel design.
result Empirically outperforms current baselines in performance and computational costs.
O3 optimizes samples from generative models without extra training.
problem Optimizing specific criteria within rich data distributions.
method Surrogate latent spaces for efficient black-box optimisation.
result Surrogate-space optimisation finds higher-scoring samples.
A new optimisation method efficiently scales Hessian-vector products for neural networks.
problem Challenges in applying second-order quasi-Newton methods due to large Hessian and non-convexity.
method Proposes an optimisation algorithm that asymptotically uses the exact inverse Hessian with modified eigenvalues.
result Demonstrates scalability and comparable performance to other optimisation methods in neural networks.
Bayesian optimisation framework for graph functions.
problem Optimizing functions on graph structures efficiently.
method Learning suitable kernels and local modelling.
result Demonstrated effectiveness on synthetic and real-world graphs.
Bayesian optimisation tackles expensive black-box functions with constraints.
problem Optimizing constrained black-box functions in machine learning and simulation.
method Proposes a new Knowledge Gradient acquisition function for constrained Bayesian optimisation.
result Demonstrates superior performance over four state-of-the-art constrained Bayesian optimisation algorithms.
MTL2L learns to adapt optimisation rules for unseen data.
problem Learners need to adapt to unseen data domains.
method Introduces MTL2L, a context-aware neural optimiser.
result MTL2L can adapt optimisation rules for unseen data.
Bayesian optimisation algorithm for unknown search spaces with sub-linear regret.
problem Efficient optimisation of expensive black-box functions in unknown search spaces.
method Expands search space over iterations based on a hyperharmonic series, scales to high dimensions.
result Sub-linear regret growth for both algorithms.
DO uses DNNs to solve combinatorial optimisation problems.
problem Solving complex combinatorial optimisation problems.
method Combines evolutionary search with DNNs to learn and exploit problem structure.
result DO can solve problems like HTOP and MCparity that other algorithms cannot.
New methods improve global optimisation for expensive functions using lookahead strategies.
problem Optimising expensive functions without gradient info in high dimensions.
method Nonmyopic acquisition strategies based on approximate dynamic programming.
result Nonmyopic methods outperform myopic approaches in various applications.
One-pass optimisation for high-dimensional hyperparameters.
problem Efficient optimisation of hyperparameters in machine learning models.
method Approximate hypergradient-based optimisation for any continuous hyperparameter, requiring only one training episode.
result Competitive performance on various datasets without hyperparameter restarts.
PBO methods improve RNN performance in learning long-term dependencies.
problem Training RNNs to learn long-term dependencies is challenging.
method Population-based global optimisation (PBO) techniques, including evolution strategies and particle swarm optimisation.
result PBO methods lead to performance improvements in RNNs for volatility forecasting.
A novel optimisation framework through quadratic nonlinear projection is introduced for credit portfolio when the portfolio risk is measured by Conditional Value-at-Risk (CVaR). The whole optimisation procedure to search toward the optimal portfolio state is conducted by a series of single-step optimisations under the …
New approach separates VAE and GP for better molecular optimisation.
problem Optimizing complex structured domains like molecular spaces using VAEs.
method Decouples VAE for structure generation and GP for predictive modelling, combining them with a Bayesian update rule.
result Improves identification of high-potential candidates in molecular optimisation.
Optimizes data power control in cell-free networks for better spectral efficiency.
problem Maximizing overall spectral efficiency in cell-free networks with multi-objective optimisation.
method Applied scalable multi-objective Bayesian optimisation to solve convergence-time limitations.
result Improved radio resource management in cell-free networks.
Dragonfly optimizes complex functions without needing grad students.
problem Challenging real-world optimisation tasks where conventional BO methods fail.
method Scalable and robust Bayesian Optimisation with multiple advanced methods.
result Significant improvements in BO performance when integrating new methods.
Study analyzes a new algorithm for complex optimization problems.
problem Stochastic bilevel optimisation problems in continuous-time models.
method Continuous-time, two-timescale stochastic approximation algorithm.
result Obtained weak convergence rate using central limit theorem.
New adaptive importance samplers improve stability and accuracy.
problem Improving the stability and accuracy of importance sampling estimators.
method Introducing AdaOAIS, a new adaptive importance sampler using adaptive optimisers to address the instability of OAIS.
result AdaOAIS leads to stable importance sampling estimators in practice.
Convex optimisation solves inverse kinematics problems more reliably.
problem Finding the best parameters of a kinematic skeleton from observed joint locations.
method Convex optimisation using semidefinite programming.
result The proposed method significantly outperforms local optimisation methods.
A simplified model for fixed income portfolio optimisation.
problem Modeling interest rates and credit risk in fixed income portfolios.
method Proposes a two-factor model for the time evolution of the efficient frontier.
result The efficient frontier is mainly controlled by linear constraints, with standard deviation less important.