A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
We present a supervised-learning algorithm from graph data (a set of graphs) for arbitrary twice-differentiable loss functions and sparse linear models over all possible subgraph features. To date, it has been shown that under all possible subgraph features, several types of sparse learning, such as Adaboost, LPBoost, …
This paper shows how deep neural networks can learn rich, independent features that significantly deviate from initialization.
problem Understanding how deep neural networks achieve meaningful feature learning and global convergence.
method Investigation of infinitely wide, L-layer neural networks using the tensor program framework under Maximal Update parametrization.
result SGD enables these networks to learn linearly independent features that substantially deviate from their initial values, capturing relevant data information.
We give elementary constructions of manifold with corner structures and associative gluing maps on compactifications of spaces of infinite, half infinite, and finite Morse flow lines.
Existence of an infinite sequence of harmonic maps between spheres of certain dimensions was proven by Bizon and Chmaj. This sequence shares many features of the Bartnik-McKinnon sequence of solutions to the Einstein-Yang-Mills equations as well as sequences of solutions that have arisen in other physical models. We ap…
Infinite mixture models are commonly used for clustering. One can sample from the posterior of mixture assignments by Monte Carlo methods or find its maximum a posteriori solution by optimization. However, in some problems the posterior is diffuse and it is hard to interpret the sampled partitionings. In this paper, we…
Indian Buffet Process based models are an elegant way for discovering underlying features within a data set, but inference in such models can be slow. Inferring underlying features using Markov chain Monte Carlo either relies on an uncollapsed representation, which leads to poor mixing, or on a collapsed representation…
More features and data lead to better model performance in random feature regression.
problem Improving model performance in random feature regression.
method Theoretical analysis of random feature regression, demonstrating the benefits of overparameterization, overfitting, and more data.
result Infinite width RF architectures are preferable to those of any finite width, and training to near-zero training loss is obligatory for near-optimal performance.
An infinitely wide model is a weighted integration ∫φ(x,v)dμ(v) of feature maps. This model excels at handling an infinite number of features, and thus it has been adopted to the theoretical study of deep learning. Kernel quadrature is a kernel-based numerical integration scheme developed for fast approxi…
Scientific explanation often requires inferring maximally predictive features from a given data set. Unfortunately, the collection of minimal maximally predictive features for most stochastic processes is uncountably infinite. In such cases, one compromises and instead seeks nearly maximally predictive features. Here, …
We propose a probabilistic model to infer supervised latent variables in the Hamming space from observed data. Our model allows simultaneous inference of the number of binary latent variables, and their values. The latent variables preserve neighbourhood structure of the data in a sense that objects in the same semanti…
We consider Thompson's groups from the perspective of mapping class groups of surfaces of infinite type. This point of view leads us to the braided Thompson groups, which are extensions of Thompson's groups by infinite (spherical) braid groups. We will outline the main features of these groups and some applications to …
Given any closed Kaehler manifold we define, following an idea by Eugenio Calabi, a Riemannian metric on the space of Kaehler metrics regarded as an infinite dimensional manifold. We prove several geometrical features of the resulting space, some of which we think were already known to Calabi. In particular, the space …
We prove that many features of Thurston's Dehn surgery theory for hyperbolic 3-manifolds generalize to Einstein metrics in any dimension. In particular, this gives large, infinite families of new Einstein metrics on compact manifolds.
Analysis of deep neural networks under various learning rules reveals dynamics of feature and prediction learning.
problem Understanding how different learning rules affect feature and prediction dynamics in deep neural networks.
method Analysis of infinite-width deep networks trained with gradient descent and various learning rules.
result The evolution of the output function is governed by an effective neural tangent kernel (eNTK), which varies depending on the learning rule and training regime.
Matrix factorisation methods decompose multivariate observations as linear combinations of latent feature vectors. The Indian Buffet Process (IBP) provides a way to model the number of latent features required for a good approximation in terms of regularised reconstruction error. Previous work has focussed on latent fe…
Distributions over exchangeable matrices with infinitely many columns, such as the Indian buffet process, are useful in constructing nonparametric latent variable models. However, the distribution implied by such models over the number of features exhibited by each data point may be poorly- suited for many modeling tas…
In this paper we consider a problem of searching a space of predictive models for a given training data set. We propose an iterative procedure for deriving a sequence of improving models and a corresponding sequence of sets of non-linear features on the original input space. After a finite number of iterations N, the n…