A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Improves meta-learning efficiency with mixed-mode differentiation.
Despite recent innovations in network architectures and loss functions, training RNNs to learn long-term dependencies remains difficult due to challenges with gradient-based optimisation methods. Inspired by the success of Deep Neuroevolution in reinforcement learning (Such et al. 2017), we explore the use of gradient-…
It seems to be a pearl of conventional wisdom that parameter learning in deep sum-product networks is surprisingly fast compared to shallow mixture models. This paper examines the effects of overparameterization in sum-product networks on the speed of parameter optimisation. Using theoretical analysis and empirical exp…
Unified framework for lifted training and inversion of neural networks.
This paper considers extractive summarisation in a comparative setting: given two or more document groups (e.g., separated by publication time), the goal is to select a small number of documents that are representative of each group, and also maximally distinguishable from other groups. We formulate a set of new object…
We introduce a gradient-based learning method to automatically adapt Markov chain Monte Carlo (MCMC) proposal distributions to intractable targets. We define a maximum entropy regularised objective function, referred to as generalised speed measure, which can be robustly optimised over the parameters of the proposal di…
One-pass optimisation for high-dimensional hyperparameters.
Optimising discrete data for a desired characteristic using gradient-based methods involves projecting the data into a continuous latent space and carrying out optimisation in this space. Carrying out global optimisation is difficult as optimisers are likely to follow gradients into regions of the latent space that the…
Training generative adversarial networks requires balancing of delicate adversarial dynamics. Even with careful tuning, training may diverge or end up in a bad equilibrium with dropped modes. In this work, we improve CS-GAN with natural gradient-based latent optimisation and show that it improves adversarial dynamics b…
Statistical inference methods are fundamentally important in machine learning. Most state-of-the-art inference algorithms are variants of Markov chain Monte Carlo (MCMC) or variational inference (VI). However, both methods struggle with limitations in practice: MCMC methods can be computationally demanding; VI methods …
MPLP learns neural network weights by treating operations as message-passing agents.
Investigates gradient descent dynamics and introduces new regularisation methods.
Adaptive model learns from time series data with changing distributions.
In many contexts the modal properties of a structure change, either due to the impact of a changing environment, fatigue, or due to the presence of structural damage. For example during flight, an aircraft's modal properties are known to change with both altitude and velocity. It is thus important to quantify these cha…
PDNAS optimizes GNN architectures for diverse datasets.
Saliency maps are often used in computer vision to provide intuitive interpretations of what input regions a model has used to produce a specific prediction. A number of approaches to saliency map generation are available, but most require access to model parameters. This work proposes an approach for saliency map gene…
Novel approach finds implicit regularisation in two-player games using BEA.
During recent years there has been an increased interest in stochastic adaptations of limited memory quasi-Newton methods, which compared to pure gradient-based routines can improve the convergence by incorporating second order information. In this work we propose a direct least-squares approach conceptually similar to…
Implicit models, which allow for the generation of samples but not for point-wise evaluation of probabilities, are omnipresent in real-world problems tackled by machine learning and a hot topic of current research. Some examples include data simulators that are widely used in engineering and scientific research, genera…
Optimizes Euclidean functions on Riemannian manifolds with warped metrics.
Develops a kernel-based framework for dynamic trading strategies.
The goal of reinforcement learning algorithms is to estimate and/or optimise the value function. However, unlike supervised learning, no teacher or oracle is available to provide the true value function. Instead, the majority of reinforcement learning algorithms estimate and/or optimise a proxy for the value function. …
The Extreme Deconvolution method fits a probability density to a dataset where each observation has Gaussian noise added with a known sample-specific covariance, originally intended for use with astronomical datasets. The existing fitting method is batch EM, which would not normally be applied to large datasets such as…
Gradient-based methods for optimisation of objectives in stochastic settings with unknown or intractable dynamics require estimators of derivatives. We derive an objective that, under automatic differentiation, produces low-variance unbiased estimators of derivatives at any order. Our objective is compatible with arbit…
AdamQLR optimizes Adam with K-FAC heuristics, achieving comparable performance to tuned benchmarks.
Wide neural networks with asymmetrical node scaling converge globally and learn features.
Improved SSD for faster and more accurate goodness-of-fit tests and model learning.
This paper uses Nested Sampling to improve Gaussian Process uncertainty quantification.
This paper explores optimising acquisition functions in Bayesian optimisation.
A new estimator improves training of probabilistic models with latent Gaussian variables.
New framework converts multi-objective to single-objective optimisation.
Unified Bayesian Optimisation for mixed variables improves performance.
We solve 6-DoF localisation and 3D reconstruction using deep state-space models.
A new method learns priors for Bayesian optimisation to improve performance.
Gaussian processes (GPs) are Bayesian nonparametric generative models that provide interpretability of hyperparameters, admit closed-form expressions for training and inference, and are able to accurately represent uncertainty. To model general non-Gaussian data with complex correlation structure, GPs can be paired wit…
Enhanced feature learning using neural networks and kernel methods with improved robustness.
Study improves Bayesian optimisation with ensemble transfer learning.
We present a probabilistic framework for both (i) determining the initial settings of kernel adaptive filters (KAFs) and (ii) constructing fully-adaptive KAFs whereby in addition to weights and dictionaries, kernel parameters are learnt sequentially. This is achieved by formulating the estimator as a probabilistic mode…
AdamZ optimiser improves neural network training efficiency.
Wind farm layout optimisation tackles space constraints with Bayesian multi-objective approach.
Book covers tools for zeroth-order convex optimisation.
Bayesian optimisation tackles high-dimensional categorical and mixed search spaces.
O3 optimizes samples from generative models without extra training.
A new optimisation method efficiently scales Hessian-vector products for neural networks.
Bayesian optimisation framework for graph functions.
Gradient-based algorithms are effective for many machine learning tasks, but despite ample recent effort and some progress, it often remains unclear why they work in practice in optimising high-dimensional non-convex functions and why they find good minima instead of being trapped in spurious ones. Here we present a qu…
Bayesian optimisation tackles expensive black-box functions with constraints.
MTL2L learns to adapt optimisation rules for unseen data.