Study estimates and predicts dynamic traffic OD flows for improved DTA models.
problem Estimating and predicting time-varying OD trip tables for dynamic traffic assignment.
method Bi-level optimisation for OD flow estimation and time series prediction for OD demand.
result High capability of proposed OD demand estimation method to reduce DTA model error.
New algorithm stabilizes bi-level hyperparameter optimization.
problem Stability issues in bi-level hyperparameter optimization.
method Uses Moreau-Yosida regularization to stabilize convergence.
result Significant improvement in loss values with fixed computation budget.
New framework tackles bi-level optimization without LLS condition.
problem Bi-level optimization problems without LLS condition.
method Bi-level Descent Aggregation (BDA) framework, proving convergence without LLS condition.
result Proves convergence of BDA without LLS condition.
Paper explores generalization of AID-based bi-level optimization methods.
problem Uncertainty in generalization properties of AID-based bi-level optimization methods.
method Uniform stability analysis and convergence study of AID-based methods.
result AID-based methods can achieve similar generalization as single-level nonconvex problems.
A new algorithm improves Wasserstein discriminant analysis for better data classification.
problem Improving data classification in machine learning.
method Bi-level nonlinear eigenvector algorithm (WDA-nepv) for optimal transport and trace ratio optimizations.
result WDA-nepv enhances classification accuracy and scalability.
A method for fair representation learning through bi-level optimization and implicit differentiation.
problem Ensuring fair predictors invariant across sub-groups.
method Bi-level optimization with inner-loop for invariant predictors, implicit path alignment for efficiency.
result Consistently better trade-off in prediction performance and fairness measurement.
Bi-GNN models drug interactions using a bi-level graph approach.
problem Predicting drug-drug interactions using machine learning.
method Bi-level graph neural networks that consider both interaction graph and representation graphs of drugs.
result Bi-GNN model improves DDI prediction accuracy compared to existing methods.
New method uses bi-level optimization to learn useful representations for imitation learning.
problem Learning useful representations for multiple tasks in imitation learning settings.
method Formulates representation learning as a bi-level optimization problem.
result Bi-level optimization framework provides sample complexity benefits for imitation learning.
This paper proves IRM minimizes o.o.d. risk under certain conditions.
problem Deep networks can fail to generalize to new domains with different distributions.
method Proves IRM minimizes o.o.d. risk through a bi-level optimization problem.
result IRM minimizes o.o.d. risk under specific conditions.
BiDVL improves EBLVMs for visual tasks by optimizing two variational distributions.
problem Training EBLVMs is challenging due to intractable distributions.
method Bi-level doubly variational learning with two tractable distributions.
result BiDVL achieves impressive image generation and reconstruction performance.
New algorithm tackles nested bi-level optimization problems for robust feature learning.
problem Nested compositional bi-level optimization problems in machine learning.
method Stochastic approximation algorithms for solving nested compositional bi-level optimization problems without matrix inversions.
result Achieves an ε-stationary solution with an oracle complexity of approximately O_T(1/ε^2).
SHINE uses forward pass quasi-Newton matrices to approximate Jacobian inverses for faster bi-level optimization.
problem Efficiently solving bi-level optimization problems with large Jacobian matrices.
method Proposes using quasi-Newton matrices from the forward pass to approximate the inverse Jacobian matrix.
result Empirically shows SHINE reduces computational cost of the backward pass for various problems.
A new stochastic method tackles bi-level optimization problems in deep learning.
problem Bi-level optimization problems in deep learning, including hyperparameter optimization and meta learning.
method Turning a BLO problem into a stochastic optimization, using SGLD MCMC and a recurrent algorithm to compute MC-estimated hypergradient.
result Our method is more robust to suboptimal inner optimization and non-unique inner minima, leading to more reliable solutions.
DGKIP extends KIP for dataset distillation without bi-level optimization.
problem Efficiently distill datasets for various loss functions.
method Leverages duality theory to avoid bi-level optimization.
result DGKIP supports a wider range of loss functions.
Paper tackles NAS problem by modeling it as a sparse supernet.
problem Neural Architecture Search (NAS) problem, particularly Mixed-Path Search.
method Model NAS as a sparse supernet with sparsity constraints. Use hierarchical accelerated proximal gradient algorithm for optimization.
result Proposed method finds compact, general, and powerful neural architectures.
This paper improves HNNs by learning optimal curvature for better generalization.
problem Inappropriate curvatures in HNNs lead to suboptimal performance.
method Sharpness-aware curvature learning method to smooth loss landscape.
result Proposed method improves HNNs' generalization across various settings.
Develops a generic two-layer framework for adaptive ABMs.
problem Bi-level adaptation problem in ABMs: agents adapt to environment, and environment adapts to agents.
method Formalizes bi-level problem as a Stackelberg game with conditional policies, solving coupled non-linear equations.
result Unified framework for adaptive ABMs, addressing traditional ABM limitations.
Proposes efficient, modular method for implicit differentiation.
problem Implicit differentiation of optimization problems.
method Automatic implicit differentiation using autodiff and implicit function theorem.
result Automatic differentiation of optimization problems is made easier and more modular.
This paper explores optimising acquisition functions in Bayesian optimisation.
problem Optimising acquisition functions in Bayesian optimisation is challenging due to their non-convex nature.
method The authors derive compositional forms for acquisition functions and use them to recast maximisation as a compositional optimisation problem.
result The compositional approach to maximising acquisition functions shows empirical advantages across various tasks.
pFedMe uses Moreau envelopes to improve personalized FL performance.
problem Statistical diversity among clients limits FL performance.
method pFedMe employs Moreau envelopes as clients' regularized loss functions in a bi-level optimization problem.
result pFedMe achieves state-of-the-art convergence rates and superior empirical performance.
A new method estimates corporate bond defaults in financial networks efficiently.
problem Challenges in valuing corporate bonds in interconnected financial systems.
method Bi-Level Importance Sampling with Splitting
result The method efficiently estimates rare default events in financial networks.
HRT uses bi-level reinforcement learning to optimize stock selection and execution in multi-asset equity markets.
problem Optimizing automated equity trading decisions under risk, turnover, and transaction costs.
method Hierarchical Reinforced Trader (HRT) framework that separates selection and execution decisions.
result HRT outperforms other methods in learning-based return-risk-cost trade-offs, improving Sharpe ratio and reducing turnover.
New framework converts multi-objective to single-objective optimisation.
problem Solving multi-objective optimisation problems.
method Formalises scalarisation into mathematical framework, uses R2 utilities.
result R2 utilities are monotone and submodular, optimised by greedy algorithms.
Unified Bayesian Optimisation for mixed variables improves performance.
problem Efficient optimisation of problems with both categorical and continuous variables.
method Derive value proposals from the Expected Improvement criterion to optimise both categorical and continuous variables under a single acquisition metric.
result Unified approach significantly outperforms existing methods across mixed-variable tasks.
New algorithm improves dataset distillation with 108% improvement on ImageNet.
problem Improving dataset distillation performance.
method Reparameterization and convexification of implicit gradients (RCIG).
result Establishes new state-of-the-art on various dataset distillation tasks.
A new method learns priors for Bayesian optimisation to improve performance.
problem Bayesian optimisation tasks often assume strong similarity, which is violated in many cases.
method Replace strong similarity assumption with shape similarity, learn priors for hyperparameters.
result PLeBO and prior transfer find good inputs in fewer evaluations.
Paper solves inverse optimal transport problem with convex optimization and neural network.
problem Learning the cost function for optimal transport from observed data.
method Unconstrained convex optimization, Sinkhorn-Knopp algorithm, and deep neural network parameterization.
result Novel framework avoids repeated OT solving, demonstrating efficiency and accuracy.
DARTS fails to generalize well; adding regularization improves robustness.
problem DARTS fails to find architectures that generalize well across different tasks.
method Identified failure modes, added regularization, proposed variations.
result Regularization robustifies DARTS to find better generalizing architectures.
Study improves Bayesian optimisation with ensemble transfer learning.
problem Improving sample efficiency in Bayesian optimisation of expensive functions.
method Empirical analysis of ensemble-based transfer learning methods and pipeline components.
result Two components (warm start initialisation and positive weight constraint) improve transfer learning Bayesian optimisation performance.
AdamZ optimiser improves neural network training efficiency.
problem Challenges in optimisation like overshooting and stagnation.
method Dynamic learning rate adjustment based on overshoot and stagnation factors.
result Consistently minimises loss function, improving model performance.
Wind farm layout optimisation tackles space constraints with Bayesian multi-objective approach.
problem Optimizing wind farm layout due to limited space and conflicting objectives.
method Set-based multi-objective Bayesian optimisation using Gaussian process.
result Demonstrates potential of set-based Bayesian multi-objective optimisation for wind farm layout.
Study resolves conjecture on overparameterized linear models' generalization.
problem Asymptotic generalization of multiclass classification with overparameterized models.
method Gaussian covariates bi-level model, Hanson-Wright inequality variant.
result Min-norm interpolating classifier can be suboptimal compared to noninterpolating classifiers.
Book covers tools for zeroth-order convex optimisation.
problem Zeroth-order convex optimisation.
method Cutting plane methods, interior point methods, continuous exponential weights, gradient descent, online Newton step.
result Improved existing bounds and algorithms.
A new method optimises problems with both continuous and categorical inputs.
problem Optimising black-box problems with mixed continuous and categorical inputs.
method Continuous and Categorical Bayesian Optimisation (CoCaBO) combining multi-armed bandits and Bayesian optimisation.
result CoCaBO outperforms existing methods on synthetic and real-world tasks.
Bayesian optimisation tackles high-dimensional categorical and mixed search spaces.
problem Bayesian optimisation on high-dimensional categorical and mixed search spaces is challenging.
method Combining local optimisation with a tailored kernel design.
result Empirically outperforms current baselines in performance and computational costs.
O3 optimizes samples from generative models without extra training.
problem Optimizing specific criteria within rich data distributions.
method Surrogate latent spaces for efficient black-box optimisation.
result Surrogate-space optimisation finds higher-scoring samples.
A new optimisation method efficiently scales Hessian-vector products for neural networks.
problem Challenges in applying second-order quasi-Newton methods due to large Hessian and non-convexity.
method Proposes an optimisation algorithm that asymptotically uses the exact inverse Hessian with modified eigenvalues.
result Demonstrates scalability and comparable performance to other optimisation methods in neural networks.
Bayesian optimisation framework for graph functions.
problem Optimizing functions on graph structures efficiently.
method Learning suitable kernels and local modelling.
result Demonstrated effectiveness on synthetic and real-world graphs.
Bayesian optimisation tackles expensive black-box functions with constraints.
problem Optimizing constrained black-box functions in machine learning and simulation.
method Proposes a new Knowledge Gradient acquisition function for constrained Bayesian optimisation.
result Demonstrates superior performance over four state-of-the-art constrained Bayesian optimisation algorithms.
MTL2L learns to adapt optimisation rules for unseen data.
problem Learners need to adapt to unseen data domains.
method Introduces MTL2L, a context-aware neural optimiser.
result MTL2L can adapt optimisation rules for unseen data.
The paper examines how to protect LASSO-based feature selection from adversarial attacks.
problem Adversarial attacks on LASSO-based feature selection.
method Formulated as a bi-level optimization problem, reformulated LASSO with linear inequality constraints, solved using interior-point method, and modified using projected gradient descent.
result Demonstrated the effectiveness of the proposed method in protecting LASSO-based feature selection from adversarial attacks.
Real world experiments are expensive, and thus it is important to reach a target in minimum number of experiments. Experimental processes often involve control variables that changes over time. Such problems can be formulated as a functional optimisation problem. We develop a novel Bayesian optimisation framework for s…
Information-theoretic Bayesian optimisation techniques have demonstrated state-of-the-art performance in tackling important global optimisation problems. However, current information-theoretic approaches require many approximations in implementation, introduce often-prohibitive computational overhead and limit the choi…
New method calibrates deep models' uncertainty predictions.
problem Quantifying deep models' uncertainties for trustworthiness.
method Separate prediction and interval estimation with uncertainty matching.
result Significant improvements in model fidelity and calibration error.
Bayesian optimisation algorithm for unknown search spaces with sub-linear regret.
problem Efficient optimisation of expensive black-box functions in unknown search spaces.
method Expands search space over iterations based on a hyperharmonic series, scales to high dimensions.
result Sub-linear regret growth for both algorithms.
New methods improve global optimisation for expensive functions using lookahead strategies.
problem Optimising expensive functions without gradient info in high dimensions.
method Nonmyopic acquisition strategies based on approximate dynamic programming.
result Nonmyopic methods outperform myopic approaches in various applications.
Paper learns dictionaries for sparse signal recovery using automatic differentiation.
problem Learning dictionaries for sparse signal recovery from noisy data.
method Approximates reconstructions using FB algorithm and learns dictionaries with projected gradient descent.
result Successfully learns 1D TV dictionary from piecewise constant signals.
Deep Optimisation (DO) combines evolutionary search with Deep Neural Networks (DNNs) in a novel way - not for optimising a learning algorithm, but for finding a solution to an optimisation problem. Deep learning has been successfully applied to classification, regression, decision and generative tasks and in this paper…