A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New complexity measure ADL connects to classical complexity measures.
New method improves bivariate causal discovery by accurately estimating cause variable complexity.
The Fisher information approximation (FIA) is an implementation of the minimum description length principle for model selection. Unlike information criteria such as AIC or BIC, it has the advantage of taking the functional form of a model into account. Unfortunately, FIA can be misleading in finite samples, resulting i…
Minimum Description Length prevents overfitting in noisy data.
We tackle the problem of penalty selection of regularization on the basis of the minimum description length (MDL) principle. In particular, we consider that the design space of the penalty function is high-dimensional. In this situation, the luckiness-normalized-maximum-likelihood(LNML)-minimization approach is favorab…
New architectures improve KANs, making them more interpretable and accurate.
Study improves neural network performance in sequential learning for image classification.
The Minimum Description Length (MDL) principle states that the optimal model for a given data set is that which compresses it best. Due to practial limitations the model can be restricted to a class such as linear regression models, which we address in this study. As in other formulations such as the LASSO and forward …
This paper introduces a new method for model selection and more generally hyperparameter selection in machine learning. Minimum description length (MDL) is an established method for model selection, which is however not directly aimed at minimizing generalization error, which is often the primary goal in machine learni…
Study shows LLC correlates with neural network compressibility.
Two trees in the boundary of outer space are said to be \emph{primitive-equivalent} whenever their translation length functions are equal in restriction to the set of primitive elements of . We give an explicit description of this equivalence relation, showing in particular that it is nontrivial. This question is …
We present an asymptotic criterion to determine the optimal number of clusters in k-means. We consider k-means as data compression, and propose to adopt the number of clusters that minimizes the estimated description length after compression. Here we report two types of compression ratio based on two ways to quantify t…
The study quantifies the information needed for causal queries at different levels of Pearl's hierarchy.
New methods evaluate data representations by complexity of low-loss predictor learning.
Method estimates dataset utility via minimal program length proxy.
We introduce a deep, generative autoencoder capable of learning hierarchies of distributed representations from data. Successive deep stochastic hidden layers are equipped with autoregressive connections, which enable the model to be sampled from quickly and exactly via ancestral sampling. We derive an efficient approx…
Critical trajectories in a sphere are found for a specific bending functional.
The minimum description length (MDL) principle in supervised learning is studied. One of the most important theories for the MDL principle is Barron and Cover's theory (BC theory), which gives a mathematical justification of the MDL principle. The original BC theory, however, can be applied to supervised learning only …
We analyze MDL for binary classification, quantifying overfitting and underfitting.
Reformulated Markov's conjecture in combinatorial terms.
Time-invariant linear dynamical system arises in many real-world applications,and its usefulness is widely acknowledged. A practical limitation with this model is that its latent dimension that has a large impact on the model capability needs to be manually specified. It can be demonstrated that a lower-order model cla…
We investigate the sample complexity of networks with bounds on the magnitude of its weights. In particular, we consider the class \[ H=\left\{W_t\circρ\circ \ldots\circρ\circ W_{1} :W_1,\ldots,W_{t-1}\in M_{d, d}, W_t\in M_{1,d}\right\} \] where the spectral norm of each is bounded by , the Frobenius norm …
Study on stable translation lengths of surface homeomorphisms and their approximations.
Neural networks generalize on simple data generated by a programming language.
DL/FBF improves GPSR solutions by selecting compact, generalising expressions.
PCA (Principal Component Analysis) and its variants areubiquitous techniques for matrix dimension reduction and reduced-dimensionlatent-factor extraction. One significant challenge in using PCA, is thechoice of the number of principal components. The information-theoreticMDL (Minimum Description Length) principle gives…
This paper studies spectral properties of spheres with one equator.
Length metrics can be closely approximated by conformally flat metrics.
CDL index improves clustering validation for non-convex data.
Let be the Teichmüller space of marked genus , punctured Riemann surfaces with its bordification $\Tbar$ the {\em augmented Teichmüller space} of marked Riemann surfaces with nodes, \cite{Abdegn, Bersdeg}. Provided with the WP metric $\Tbar$ is a complete CAT(0) metric space, \cite{DW2, Wlcomp, Yam2…
Study on reducing forgetting in neural networks using compression theory.
Study approximate marked length spectrum rigidity in non-positively curved groups.
Paper establishes generalization bounds for representation learning using Minimum Description Length.
Kernel networks' stability edge linked to Fisher Information singularity.
A new method avoids overfitting in network reconstruction by using the minimum description length principle.
We introduce length dilatation structures on metric spaces, tempered dilatation structures and coherent projections and explore the relations between these objects and the Radon-Nikodym property and Gamma-convergence of length functionals. Then we show that the main properties of sub-riemannian spaces can be obtained f…
ACNML method improves uncertainty estimation for deep networks.
Given a surface of infinite topological type, there are several Teichmüller spaces associated with it, depending on the basepoint and on the point of view that one uses to compare different complex structures. This paper is about the comparison between the quasiconformal Teichmüller space and the length-spectrum Teichm…
Multivariate Poisson approximation of the length spectrum of random surfaces is studied by means of the Chen-Stein method. This approach delivers simple and explicit error bounds in Poisson limit theorems. They are used to prove that Poisson approximation applies to curves of length up to order with …
The parametric complexity is the key quantity in the minimum description length (MDL) approach to statistical model selection. Rissanen and others have shown that the parametric complexity of a statistical model approaches a simple function of the Fisher information volume of the model as the sample size goes to in…
We present a method for the reconstruction of networks, based on the order of nodes visited by a stochastic branching process. Our algorithm reconstructs a network of minimal size that ensures consistency with the data. Crucially, we show that global consistency with the data can be achieved through purely local consid…
Proving that next-token prediction makes language models generate coherent long documents.
Gaussian multiplicative noise is commonly used as a stochastic regularisation technique in training of deterministic neural networks. A recent paper reinterpreted the technique as a specific algorithm for approximate inference in Bayesian neural networks; several extensions ensued. We show that the log-uniform prior us…
New method uses short geodesics to approximate marked length spectrum.
APD method decomposes neural network parameters into simple, faithful components.
Human decision-making deviates from the optimal solution, that maximizes cumulative rewards, in many situations. Here we approach this discrepancy from the perspective of bounded rationality and our goal is to provide a justification for such seemingly sub-optimal strategies. More specifically we investigate the hypoth…
Fast, fully-automated histograms for large data sets.
We propose a simple, tractable lower bound on the mutual information contained in the joint generative density of any latent variable generative model: the GILBO (Generative Information Lower BOund). It offers a data-independent measure of the complexity of the learned latent variable description, giving the log of the…