arXiv research

GNIs induce a regulariser that penalizes high-frequency components in neural network activations.

Noise regularisation in deep nets makes them behave like Gaussian processes.

New framework monitors neural network training and reveals regularisation mechanisms.

Noise-driven neural networks emerge modular structures, improving robustness and generalization.

Study on how noise and variation-norm regularisation help shallow ReLU networks use fewer neurons.

Stochastic regularisation is an important weapon in the arsenal of a deep learning practitioner. However, despite recent theoretical advances, our understanding of how noise influences signal propagation in deep neural networks remains limited. By extending recent work based on mean field theory, we develop a new frame…

Graph poly-Laplacian method improves regression accuracy.

This paper examines the assumptions of the derived equivalence between dropout noise injection and $L_2$ regularisation for logistic regression with negative log loss. We show that the approximation method is based on a divergent Taylor expansion, making, subsequent work using this approximation to compare the dropout …

Denoising autoencoders (DAEs) have proven useful for unsupervised representation learning, but a thorough theoretical understanding is still lacking of how the input noise influences learning. Here we develop theory for how noise influences learning in DAEs. By focusing on linear DAEs, we are able to derive analytic ex…

Differential privacy of Gaussian process posterior sampling

A method improves Cryo-EM 3D map refinement by regularizing rotation estimation.

This work optimizes RL algorithms using entropy regularisation for continuous-time LQ problems.

Detect changes in noisy dynamical systems using empirical approximations and finite-sample bounds.

New insights into how neural networks learn features, especially when they are very wide.

Improved deep kernel machine achieves 94.5% test accuracy on CIFAR-10.

This research improves multimodal systems by adding a second objective and regularisation methods.

Paper advances sparse regularisation theory for measures with new kernel insights.

A new method for manifold learning using sparse regularised optimal transport.

ProSelfLC improves robustness of deep neural networks by automatically deciding trust in predictions.

A trade-off exists between reconstruction quality and the prior regularisation in the Evidence Lower Bound (ELBO) loss that Variational Autoencoder (VAE) models use for learning. There are few satisfactory approaches to deal with a balance between the prior and reconstruction objective, with most methods dealing with t…

EarlyStopping package helps prevent overfitting in iterative learning procedures.

Improved graph-based connectivity estimation using heat modelling.

This paper improves inverse problem solving with weakly convex regularisers and proves convergence.

Generalised regularisation equals robustness for exotic function classes.

Geometry-aware noise improves model generalization on complex manifolds.

A Python package solves source duplication in single channel LVMs using spectral regularisation.

This paper addresses the issue of feature importance landscapes in complex images and proposes a regularisation technique to improve network performance.

Proposes a new noise injection method for neural networks that improves accuracy and representation clarity.

We consider a general regularised interpolation problem for learning a parameter vector from data. The well known representer theorem says that under certain conditions on the regulariser there exists a solution in the linear span of the data points. This is the core of kernel methods in machine learning as it makes th…

We introduce a class of regularisable infinite dimensional principal fibre bundles which includes fibre bundles arising in gauge field theories like Yang-Mills and string theory and which generalise finite dimensional Riemannian principal fibre bundles induced by an isometric action. We show that the orbits of regulari…

Single-cell RNA sequencing (scRNA-seq) is a fast growing approach to measure the genome-wide transcriptome of many individual cells in parallel, but results in noisy data with many dropout events. Existing methods to learn molecular signatures from bulk transcriptomic data may therefore not be adapted to scRNA-seq data…

This work uncovers algorithm-dependent regularisation in diffusion models.

Study examines robust regression in high dimensions with heavy-tailed data.

Despite recent advances in regularisation theory, the issue of parameter selection still remains a challenge for most applications. In a recent work the framework of statistical learning was used to approximate the optimal Tikhonov regularisation parameter from noisy data. In this work, we improve their results and ext…

GNIs induce asymmetric heavy-tailed noise in SGD, affecting network performance.

New method prevents deep learning forgetting past by remembering key examples.

Novel approach finds implicit regularisation in two-player games using BEA.

Regularization preserves topological data structure in autoencoders.

Regularisation improves ML classifier stability against poisoning attacks.

Investigates gradient descent dynamics and introduces new regularisation methods.

Researchers compare different gradient methods for ridge regression, finding conjugate gradients have similar performance.

Marginalising out uncertain quantities within the internal representations or parameters of neural networks is of central importance for a wide range of learning techniques, such as empirical, variational or full Bayesian methods. We set out to generalise fast dropout (Wang & Manning, 2013) to cover a wider variety of …

Study on GD and SGD over diagonal networks, focusing on stepsizes and regularisation.

We propose graph-dependent implicit regularisation strategies for distributed stochastic subgradient descent (Distributed SGD) for convex problems in multi-agent learning. Under the standard assumptions of convexity, Lipschitz continuity, and smoothness, we establish statistical learning rates that retain, up to logari…

This work shows how penalising bias terms in norm regularisation leads to sparse solutions.

Deep models can fit noisy labels, but robustness and reliability are still issues.

Regularizes ML algorithms for robust multivariate analysis against distribution shifts.

Paper studies particle method for LSV model calibration, proving convergence and error bounds.