This work improves generalisation bounds using chaining and information theory.
problem Improving generalisation bounds for supervised learning algorithms.
method Developed a theoretical framework linking generalisation bounds to their chained counterparts, derived new bounds using Wasserstein distance.
result Chained generalisation bounds can be tighter than standard bounds, especially for concentrated hypothesis distributions.
Study shows bounds on volumes of weakly generalised alternating knots.
problem Volume bounds for weakly generalised alternating knots.
method Analysis of weakly generalised alternating knots in 3-manifolds.
result Upper volume bound does not hold for weakly generalised alternating knots.
New PAC-Bayes bounds use Wasserstein distances to improve generalization.
problem Lack of geometric properties in existing PAC-Bayes bounds.
method Developed new PAC-Bayes bounds with Wasserstein distances.
result Optimization guarantees translate to good generalization abilities.
New method optimises learning via surrogate PAC-Bayes bounds.
problem Computational intractability of optimising generalisation bounds.
method Iteratively optimising surrogate training objectives derived from PAC-Bayes bounds.
result Iteratively optimising surrogates implies optimising original generalisation bounds.
The study examines sequences of 3D generalised monopoles with uniform bounds.
problem Analyzing sequences of 3D generalised monopoles with uniform L2-norm bounds.
method Focus on Swann bundles and uniform lower bounds on hyperKahler potential.
result Convergent subsequences of generalised monopoles over compact subsets of Y' are found.
Paper simplifies link classification in 3-sphere using braids and templates.
problem Classify links in the 3-sphere.
method Simplified braid description, generalised T-links, bunch algorithm.
result Established upper volume bound for 3-manifolds.
Novel strategy for federated learning with privacy-preserving predictors and nonvacuous generalization bounds.
problem Privacy-preserving federated learning with nonvacuous generalization bounds.
method Randomized predictors, PAC-Bayesian generalization bound, synchronous and heterogeneous/homogenous cases.
result Achieves comparable predictive performance to batch approach while preserving privacy.
The paper improves generalization bounds using interpolation between various divergences.
problem Improving generalization bounds in machine learning.
method Derives new PAC-Bayes generalization bounds based on ( f , Γ ) (f, Γ) ( f , Γ ) -divergence and interpolates between various divergences. result Connects derived bounds to earlier statistical learning results and provides practical training objectives.
PAC-Bayes bound requires prior to place mass on high-performing predictors.
problem Explaining generalization in machine learning.
method Analyzing necessary conditions for PAC-Bayes bounds to provide meaningful generalization guarantees.
result Achieving a target generalisation level requires the prior to place sufficient mass on high-performing predictors.
Develops PAC-Bayesian framework for physics-informed machine learning.
problem Lack of statistical generalisation understanding for PIML models.
method PAC-Bayesian framework with multi-task perspective, incorporating physical structure.
result High-probability generalisation guarantees with unbounded losses.
Study generalization of voting classifiers using margin-based bounds.
problem Understanding the generalization of ensemble classifiers like voting.
method Proved margin-based generalization bounds using PAC-Bayes theory and Dirichlet posteriors.
result Provided state-of-the-art guarantees on classification tasks.
New algorithm reduces regret in delayed feedback generalised linear bandits.
problem Regret in delayed feedback generalised linear bandits.
method Adaptation of optimistic algorithm to delayed feedback.
result Achieves a regret bound independent of the horizon's delay penalty.
New bounds for shallow neural networks with deterministic parameters.
problem Developing generalisation bounds for shallow neural networks.
method PAC-Bayesian theory applied to shallow neural networks with deterministic parameters.
result Empirical non-vacuous bounds for shallow neural networks trained with vanilla SGD.
New bounds link flat minima to good generalisation in overparameterized models.
problem Understanding the relationship between flat minima and generalisation in overparameterized machine learning models.
method Combining PAC-Bayes, Poincaré, and Log-Sobolev inequalities to derive generalisation bounds involving gradient terms.
result Flat minima positively influence generalisation performance, highlighting the benefits of the optimisation phase.
Quantum reservoirs risk bounds are analyzed using Rademacher complexity.
problem Bounding generalization errors of quantum reservoirs.
method Using Rademacher complexity, specific bounds are derived for quantum reservoir classes.
result Risk bounds converge with increasing training samples and qubits.
New stability bounds for GD in overparameterised shallow nets without NTK assumptions.
problem Generalisation and excess risk bounds for shallow neural networks.
method Oracle inequalities and stability analysis of GD without kernelisation.
result Oracle type bounds reveal GD's generalisation is controlled by an interpolating network with shortest GD path.
This work uncovers algorithm-dependent regularisation in diffusion models.
problem Understanding and improving generalisation in high-dimensional diffusion models.
method Algorithmic stability and score stability analysis.
result Identifies multiple sources of implicit regularisation unique to diffusion models.
Contrastive unsupervised representation learning (CURL) is the state-of-the-art technique to learn representations (as a set of features) from unlabelled data. While CURL has collected several empirical successes recently, theoretical understanding of its performance was still missing. In a recent work, Arora et al. (2…
Mixability of a loss is known to characterise when constant regret bounds are achievable in games of prediction with expert advice through the use of Vovk's aggregating algorithm. We provide a new interpretation of mixability via convex analysis that highlights the role of the Kullback-Leibler divergence in its definit…
New PAC-Bayes bounds for unbounded loss functions.
problem Generalization bounds for learning problems with unbounded loss functions.
method Introducing HYPE, a new notion for loss range, and deriving a novel PAC-Bayesian generalization bound.
result PAC-Bayes framework extended to unbounded loss functions.
We provide bounds on control learning error in stochastic systems.
problem Learning optimal controls in stochastic environments with uncontrolled parts.
method Dynamic programming and mean-field interpretation of neural networks.
result Non-asymptotic bounds on generalization error for stable overparametrised settings.
Investigates Lipschitz continuity in neural networks across various settings.
problem Understanding the Lipschitz behavior of neural networks.
method Empirical investigation of Lipschitz bounds in different neural network architectures and datasets.
result Remarkable fidelity of the lower Lipschitz bound and a Double Descent trend in both upper and lower bounds.
In this paper, we establish a generalised Blaschke-Santalò inequality for convex bodies in R n + 1 \mathbb R^{n+1} R n + 1 . This inequality gives an upper bound estimate for the product of dual quermassintegrals of convex body and its polar set. Our argument is based on induction on dimensions.
New bounds improve generalization in machine learning with high probability.
problem Erratic behavior of KL divergence limits practical applications.
method Replaced KL divergence with Wasserstein distance for better bounds.
result Proved high probability generalization bounds for i.i.d. and non-i.i.d. data.
Graph neural networks generalize well under certain conditions, explained by learning theory.
problem Understanding why graph neural networks generalize well in transductive inference.
method Analysis of transductive Rademacher complexity to explain generalization properties of graph convolutional networks.
result Transductive Rademacher complexity can explain the generalization of graph convolutional networks for node classification in stochastic block models.
Improved PAC-Bayesian bounds by considering example difficulty.
problem Improving generalization bounds in machine learning.
method Introducing a modified excess risk that leverages example difficulty to reduce variance and tighten PAC-Bayesian bounds.
result Tighter PAC-Bayesian generalization bounds for machine learning models.
PAC-Bayes bound for stable RNNs in time-series data.
problem Bounding generalization gap for stable RNNs in time-series data.
method Derived a PAC-Bayes bound with stability constraints for discrete-time non-linear dynamical systems, including stable RNNs.
result The bound converges to zero as dataset size increases, and does not grow with RNN steps.
New PAC-Bayes bounds for heavy-tailed losses using supermartingales.
problem Extending PAC-Bayes bounds to heavy-tailed losses.
method Using supermartingales and bounded variance assumption.
result PAC-Bayes generalization bounds for heavy-tailed losses.
We consider links that are alternating on surfaces embedded in a compact 3-manifold. We show that under mild restrictions, the complement of the link decomposes into simpler pieces, generalising the polyhedral decomposition of alternating links of Menasco. We use this to prove various facts about the hyperbolic geometr…
Most machine learning theory and practice is concerned with learning a single task. In this thesis it is argued that in general there is insufficient information in a single task for a learner to generalise well and that what is required for good generalisation is information about many similar learning tasks. Similar …
New examples show no upper bounds on link volumes on incompressible surfaces.
problem Finding upper bounds on volumes of links on incompressible surfaces.
method Examined weakly generalised alternating and fully augmented links on incompressible surfaces.
result Found infinite families of links on incompressible surfaces with no upper bounds on volume.
Improved similarity search in embeddings using InfoNCE loss.
problem Improving similarity search in embedding models trained by contrastive learning.
method Introduced a new continuity bound for InfoNCE loss via Gâteaux differentiation, preserving the averaging effect of negative samples.
result Demonstrated that the averaging effect of k k k negative samples in InfoNCE loss carries over to stabilisation of generalisation error as k k k grows. New method reveals why GNNs perform well on certain datasets.
problem Understanding why GNNs perform differently on similar datasets.
method Deriving exact generalization error for various GNN architectures.
result Benchmark datasets favor architectures that rely on graph structure.
New bounds for model generalization under deterministic gradient descent.
problem Establishing generalization bounds for models trained with gradient descent methods.
method PAC-Bayesian bounds for deterministic optimisation algorithms.
result Fully computable bounds that depend on initial distribution and Hessian.
OGD proves robustness to Catastrophic Forgetting in Continual Learning.
problem Catastrophic Forgetting in Continual Learning with deep neural networks.
method Theoretical framework based on Neural Tangent Kernel for OGD.
result First generalization bound for SGD and OGD in Continual Learning.
We show that finiteness of the Lorentzian distance is equivalent to the existence of generalised time functions with gradient uniformly bounded away from light cones. To derive this result we introduce new techniques to construct and manipulate achronal sets. As a consequence of these techniques we obtain a functional …
Investigates tight PAC-Bayes bounds for small datasets.
problem Tightening PAC-Bayes bounds for small data.
method Generic PAC-Bayes theorem, meta-learning, synthetic tasks.
result PAC-Bayes bounds are competitive with Chernoff bounds but not as tight.
A new method constrains deep networks during fine-tuning to improve generalization.
problem Improving generalization of fine-tuned deep networks.
method A neural network generalisation bound based on distance from initial weights constrains the hypothesis class to a small sphere.
result Empirical evaluation shows superior generalization performance compared to existing methods.
The present paper provides a new generic strategy leading to non-asymptotic theoretical guarantees on the Leave-one-Out procedure applied to a broad class of learning algorithms. This strategy relies on two main ingredients: the new notion of L q L^q L q stability, and the strong use of moment inequalities. L q L^q L q stability e…
The paper improves transformer generalization bounds using rank-dependent covering number bounds.
problem Improving generalization bounds for transformers.
method Introducing rank-dependent covering number bounds for linear function classes and applying them to transformers.
result Generalization error bounds for transformers decay as O ( 1 / n ) O(1/\sqrt{n}) O ( 1/ n ) and O ( log r w ) O(\log r_w) O ( log r w ) , improving existing bounds. Paper uses SLT to improve model selection for SHM.
problem Model selection for SHM using data-based systems.
method Utilizes Statistical Learning Theory to rigorously estimate generalisation.
result Incorporating domain knowledge improves model generalisation.
Sharp lower bound found for integral varifolds' mean curvature.
problem Finding a sharp lower bound for the mean curvature integral of integral varifolds.
method Developed a new approach using integral varifolds and mean curvature.
result A sharp lower bound on the mean curvature integral with critical power for integral varifolds.
We prove a splitting theorem for Riemannian n-manifolds with scalar curvature bounded below by a negative constant and containing certain area-minimising hypersurfaces (Theorem 3). Thus we generalise [25,Theorem 3] by Nunes. This splitting result follows from an area comparison theorem for hypersurfaces with non-positi…
In this paper we present a self-contained combinatorial proof of the lower bound theorem for normal pseudomanifolds, including a treatment of the cases of equality in this theorem. We also discuss McMullen and Walkup's generalised lower bound conjecture for triangulated spheres in the context of the lower bound theorem…
Study introduces indecomposability for varifolds, leading to geometric consequences.
problem Understanding the structure of varifolds and their connectedness properties.
method Introducing indecomposability and related concepts for varifolds.
result Substantial geometric consequences derived from the connectedness properties of varifolds.
The notion of i-bounded geometry generalises simultaneously bounded geometry and the geometry of punctured torus Kleinian groups. We show that the limit set of a surface Kleinian group of i-bounded geometry is locally connected by constructing a natural Cannon-Thurston map. This is an exposition of a special case of th…
Study timelike Ricci curvature bounds via optimal transport with Orlicz-type costs.
problem Characterize timelike Ricci curvature bounds.
method Optimal transport with Orlicz-type costs, convexity of relative entropy.
result Characterize timelike Ricci curvature lower bounds via convexity of relative entropy.
Study on approximability and generalization in machine learning.
problem Understanding how approximation affects learning and generalization in machine learning.
method Introducing a notion of sensitivity to analyze the impact of approximation operators on predictors and proving upper bounds on generalization.
result Proven that approximable target concepts are learnable with fewer labelled samples and sufficient unlabelled data.