arXiv research

New algorithm optimizes PAC-Bayes bound without surrogate loss.

Develops a new algorithm for estimating model parameters using interacting particle systems.

Adaptive importance samplers are adaptive Monte Carlo algorithms to estimate expectations with respect to some target distribution which \textit{adapt} themselves to obtain better estimators over a sequence of iterations. Although it is straightforward to show that they have the same $\mathcal{O}(1/\sqrt{N})$ convergen…

Gradient descent performs well on weakly convex losses, offering generalization guarantees.

We describe a novel algorithm for noisy global optimisation and continuum-armed bandits, with good convergence properties over any continuous reward function having finitely many polynomial maxima. Over such functions, our algorithm achieves square-root regret in bandits, and inverse-square-root error in optimisation, …

Optimal best arm identification for multi-objective bandits with fixed error probability.

Bayesian optimisation (BO) has been a successful approach to optimise functions which are expensive to evaluate and whose observations are noisy. Classical BO algorithms, however, do not account for errors about the location where observations are taken, which is a common issue in problems with physical components. In …

New method optimises learning via surrogate PAC-Bayes bounds.

Improved Bayesian optimisation method using randomised Gaussian process UCB.

Continuous-time PCD for MLE with explicit error bounds.

New PAC-Bayes bounds use Wasserstein distances to improve generalization.

Improved regret bound for adversarial bandit convex optimisation.

Random forests are among the most popular classification and regression methods used in industrial applications. To be effective, the parameters of random forests must be carefully tuned. This is usually done by choosing values that minimize the prediction error on a held out dataset. We argue that error reduction is o…

Bayesian optimisation has gained great popularity as a tool for optimising the parameters of machine learning algorithms and models. Somewhat ironically, setting up the hyper-parameters of Bayesian optimisation methods is notoriously hard. While reasonable practical solutions have been advanced, they can often fail to …

Book covers tools for zeroth-order convex optimisation.

New adaptive importance samplers improve stability and accuracy.

Despite the improved accuracy of deep neural networks, the discovery of adversarial examples has raised serious safety concerns. In this paper, we study two variants of pointwise robustness, the maximum safe radius problem, which for a given input sample computes the minimum distance to an adversarial example, and the …

Novel approach finds implicit regularisation in two-player games using BEA.

Unified high-probability regret bounds for online convex optimisation with randomised gradient estimators.

Bayesian optimisation (BO) is a well-known efficient algorithm for finding the global optimum of expensive, black-box functions. The current practical BO algorithms have regret bounds ranging from $\mathcal{O}(\frac{logN}{\sqrt{N}})$ to $\mathcal O(e^{-\sqrt{N}})$, where $N$ is the number of evaluations. This paper exp…

Bayesian optimisation algorithm for unknown search spaces with sub-linear regret.

The paper develops methods for novelty detection on path space using signature-based statistics.

Bayesian optimisation improves with fully-Bayesian treatment of hyperparameters.

Bayesian optimisation's mean function choice affects convergence speed.

Investigates gradient descent dynamics and introduces new regularisation methods.

Method solves optimisation problems on non-Riemannian surfaces with bilateral curvature bounds.

Deep Neural Network (DNN) acoustic models often use discriminative sequence training that optimises an objective function that better approximates the word error rate (WER) than frame-based training. Sequence training is normally implemented using Stochastic Gradient Descent (SGD) or Hessian Free (HF) training. This pa…

Bayesian optimization guided by experimenter intuition and beliefs.

Techniques known as Nonlinear Set Membership prediction, Kinky Inference or Lipschitz Interpolation are fast and numerically robust approaches to nonparametric machine learning that have been proposed to be utilised in the context of system identification and learning-based control. They utilise presupposed Lipschitz p…

Automated method creates compact chemical models from detailed ones, reducing complexity and improving accuracy.

New approach shapes error distribution in long-term forecasting.

New bounds for model generalization under deterministic gradient descent.

We propose a new framework to improve the calibration of neural networks.

FP uses random projections to train networks without feedback, achieving comparable performance to backpropagation.

Recent DNN pruning algorithms have succeeded in reducing the number of parameters in fully connected layers, often with little or no drop in classification accuracy. However, most of the existing pruning schemes either have to be applied during training or require a costly retraining procedure after pruning to regain c…

This paper introduces the variational Rényi bound (VR) that extends traditional variational inference to Rényi's alpha-divergences. This new family of variational methods unifies a number of existing approaches, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is co…

This paper presents a new optimisation approach to train Deep Neural Networks (DNNs) with discriminative sequence criteria. At each iteration, the method combines information from the Natural Gradient (NG) direction with local curvature information of the error surface that enables better paths on the parameter manifol…

Optimal switching regret for all segmentations in online convex optimisation.

Investigates portfolio optimization with and without gearing constraints.

Pure exploration (aka active testing) is the fundamental task of sequentially gathering information to answer a query about a stochastic environment. Good algorithms make few mistakes and take few samples. Lower bounds (for multi-armed bandit models with arms in an exponential family) reveal that the sample complexity …

Optimizes task allocation for financial analysts to balance work efficiency and well-being.

Theory for RLHF generalization under reward shift and clipped KL.

Paper develops robust SGLD for solving non-convex DRO problems.

Proposes PE-GP-UCB for time-varying Bayesian optimisation.

FraPPE efficiently identifies Pareto optimal arms in multi-objective bandits.

In this paper, several modifications are introduced to the functional approximation method iterLap to reduce the approximation error, including stopping rule adjustment, proposal of new residual function, starting point selection for numerical optimisation, scaling of Hessian matrix. Illustrative examples are also prov…

We propose an algorithm for the adaptation of the learning rate for stochastic gradient descent (SGD) that avoids the need for validation set use. The idea for the adaptiveness comes from the technique of extrapolation: to get an estimate for the error against the gradient flow which underlies SGD, we compare the resul…

This paper explores optimising acquisition functions in Bayesian optimisation.