arXiv research

This work optimizes RL algorithms using entropy regularisation for continuous-time LQ problems.

Study uses actor-critic method for continuous-time mean-field control with entropy regularisation.

ProSelfLC improves robustness of deep neural networks by automatically deciding trust in predictions.

In this report, we review the calculation of entropy-regularised Wasserstein loss introduced by Cuturi and document a practical implementation in PyTorch. Code is available at https://github.com/t-vi/pytorch-tvmisc/blob/master/wasserstein-distance/Pytorch_Wasserstein.ipynb

Grogan et al [11,12] have recently proposed a solution to colour transfer by minimising the Euclidean distance L2 between two probability density functions capturing the colour distributions of two images (palette and target). It was shown to be very competitive to alternative solutions based on Optimal Transport for c…

Improved control approach for correlated bandits with better performance.

We develop a new method for regularising neural networks. We learn a probability distribution over the activations of all layers of the model and then insert imputed values into the network during training. We obtain a posterior for an arbitrary subset of activations conditioned on the remainder. This is a generalisati…

A new measure helps compute suboptimality in entropy-regularized methods.

This paper examines the assumptions of the derived equivalence between dropout noise injection and $L_2$ regularisation for logistic regression with negative log loss. We show that the approximation method is based on a divergent Taylor expansion, making, subsequent work using this approximation to compare the dropout …

Stability result for a popular algorithm in optimal transport.

Framework expands particle filtering to estimate states beyond prior boundaries.

The study analyzes multi-class teacher-student perceptron performance and generalization errors.

Study compares synthetic and distributional Ricci curvature bounds.

We introduce multiplicative LSTM (mLSTM), a recurrent neural network architecture for sequence modelling that combines the long short-term memory (LSTM) and multiplicative recurrent neural network architectures. mLSTM is characterised by its ability to have different recurrent transition functions for each possible inp…

An important property for lifelong-learning agents is the ability to combine existing skills to solve unseen tasks. In general, however, it is unclear how to compose skills in a principled way. We provide a "recipe" for optimal value function composition in entropy-regularised reinforcement learning (RL) and then exten…

This paper improves inverse problem solving with weakly convex regularisers and proves convergence.

GNIs induce a regulariser that penalizes high-frequency components in neural network activations.

A Python package solves source duplication in single channel LVMs using spectral regularisation.

We consider a general regularised interpolation problem for learning a parameter vector from data. The well known representer theorem says that under certain conditions on the regulariser there exists a solution in the linear span of the data points. This is the core of kernel methods in machine learning as it makes th…

We introduce a class of regularisable infinite dimensional principal fibre bundles which includes fibre bundles arising in gauge field theories like Yang-Mills and string theory and which generalise finite dimensional Riemannian principal fibre bundles induced by an isometric action. We show that the orbits of regulari…

New framework monitors neural network training and reveals regularisation mechanisms.

This work uncovers algorithm-dependent regularisation in diffusion models.

Despite recent advances in regularisation theory, the issue of parameter selection still remains a challenge for most applications. In a recent work the framework of statistical learning was used to approximate the optimal Tikhonov regularisation parameter from noisy data. In this work, we improve their results and ext…

The problem of adversarial examples has highlighted the need for a theory of regularisation that is general enough to apply to exotic function classes, such as universal approximators. In response, we give a very general equality result regarding the relationship between distributional robustness and regularisation, as…

New method prevents deep learning forgetting past by remembering key examples.

Novel approach finds implicit regularisation in two-player games using BEA.

Regularization preserves topological data structure in autoencoders.

Regularisation improves ML classifier stability against poisoning attacks.

Investigates gradient descent dynamics and introduces new regularisation methods.

New method generates synthetic time series paths with more flexibility.

Researchers compare different gradient methods for ridge regression, finding conjugate gradients have similar performance.

Study on GD and SGD over diagonal networks, focusing on stepsizes and regularisation.

We propose graph-dependent implicit regularisation strategies for distributed stochastic subgradient descent (Distributed SGD) for convex problems in multi-agent learning. Under the standard assumptions of convexity, Lipschitz continuity, and smoothness, we establish statistical learning rates that retain, up to logari…

This work shows how penalising bias terms in norm regularisation leads to sparse solutions.

Regularizes ML algorithms for robust multivariate analysis against distribution shifts.

Study on how noise and variation-norm regularisation help shallow ReLU networks use fewer neurons.

Paper studies particle method for LSV model calibration, proving convergence and error bounds.

Effective regularisation of neural networks is essential to combat overfitting due to the large number of parameters involved. We present an empirical analogue to the Lipschitz constant of a feed-forward neural network, which we refer to as the maximum gain. We hypothesise that constraining the gain of a network will h…

In large scale systems, approximate nearest neighbour search is a crucial algorithm to enable efficient data retrievals. Recently, deep learning-based hashing algorithms have been proposed as a promising paradigm to enable data dependent schemes. Often their efficacy is only demonstrated on data sets with fixed, limite…

New insights into how neural networks learn features, especially when they are very wide.

Study on neural networks with regularisation and its impact on training dynamics.

Recent work has established the equivalence between deep neural networks and Gaussian processes (GPs), resulting in so-called neural network Gaussian processes (NNGPs). The behaviour of these models depends on the initialisation of the corresponding network. In this work, we consider the impact of noise regularisation …

We introduce a gradient-based learning method to automatically adapt Markov chain Monte Carlo (MCMC) proposal distributions to intractable targets. We define a maximum entropy regularised objective function, referred to as generalised speed measure, which can be robustly optimised over the parameters of the proposal di…

We show that many standard results of Lorentzian causality theory remain valid if the regularity of the metric is reduced to $C^{1,1}$. Our approach is based on regularisations of the metric adapted to the causal structure.

New algorithm computes Schrödinger Bridge for unpaired data translation.

Regularising for invariance to data augmentation improves machine learning models.

The transfer of knowledge from one policy to another is an important tool in Deep Reinforcement Learning. This process, referred to as distillation, has been used to great success, for example, by enhancing the optimisation of agents, leading to stronger performance faster, on harder domains [26, 32, 5, 8]. Despite the…

Unified understanding of three continual learning regularisation methods.