arXiv research

New methods for Markov Blanket discovery using MML outperform existing approaches.

The minimum message length principle is an information theoretic criterion that links data compression with statistical inference. This paper studies the strict minimum message length (SMML) estimator for $d$-dimensional exponential families with continuous sufficient statistics, for all $d \ge 1$. The partition of an …

We analyze differences between two information-theoretically motivated approaches to statistical inference and model selection: the Minimum Description Length (MDL) principle, and the Minimum Message Length (MML) principle. Based on this analysis, we present two revised versions of MML: a pointwise estimator which give…

The K-Mean and EM algorithms are popular in clustering and mixture modeling, due to their simplicity and ease of implementation. However, they have several significant limitations. Both coverage to a local optimum of their respective objective functions (ignoring the uncertainty in the model space), require the apriori…

New statistical model improves protein alignment accuracy.

We study the misclassification error for community detection in general heterogeneous stochastic block models (SBM) with noisy or partial label information. We establish a connection between the misclassification rate and the notion of minimum energy on the local neighborhood of the SBM. We develop an optimally weighte…

Mixture modelling involves explaining some observed evidence using a combination of probability distributions. The crux of the problem is the inference of an optimal number of mixture components and their corresponding parameters. This paper discusses unsupervised learning of mixture models using the Bayesian Minimum M…

Minimum Description Length prevents overfitting in noisy data.

A method is given for calculating the strict minimum message length (SMML) estimator for 1-dimensional exponential families with continuous sufficient statistics. A set of $n$ equations are found that the $n$ cut-points of the SMML estimator must satisfy. These equations can be solved using Newton's method and this app…

A major problem for the learning of Bayesian networks (BNs) is the exponential number of parameters needed for conditional probability tables. Recent research reduces this complexity by modeling local structure in the probability tables. We examine the use of log-linear local models. While log-linear models in this con…

Motivated by the observation that overexposure to unwanted marketing activities leads to customer dissatisfaction, we consider a setting where a platform offers a sequence of messages to its users and is penalized when users abandon the platform due to marketing fatigue. We propose a novel sequential choice model to ca…

Strict Minimum Message Length (SMML) is an information-theoretic statistical inference method widely cited (but only with informal arguments) as providing estimations that are consistent for general estimation problems. It is, however, almost invariably intractable to compute, for which reason only approximations of it…

Knots are commonly found in molecular chains such as DNA and proteins, and they have been considered to be useful models for structural analysis of these molecules. One interested quantity is the minimum number of monomers necessary to realize a molecular knot. The minimum lattice length $\mbox{Len}(K)$ of a knot $K$ i…

We define a new class of Bayesian point estimators, which we refer to as risk averse. Using this definition, we formulate axioms that provide natural requirements for inference, e.g. in a scientific setting, and show that for well-behaved estimation problems the axioms uniquely characterise an estimator. Namely, for es…

We show that the minimum of asymptotic translation lengths of all point-pushing pseudo-Anosov maps on any one punctured Riemann surface is one.

Study shows LLC correlates with neural network compressibility.

Let $\mbox{Len}(K)$ be the minimum length of a knot on the cubic lattice (namely the minimum length necessary to construct the knot in the cubic lattice). This paper provides upper bounds for $\mbox{Len}(K)$ of a nontrivial knot $K$ in terms of its crossing number $c(K)$ as follows: $\mbox{Len}(K) \leq \min \left\{ \fr…

Study on folded ribbon knots and their minimum length.

Study finds minimum lengths of curves on a one-holed torus.

The modelling of empirically observed data is commonly done using mixtures of probability distributions. In order to model angular data, directional probability distributions such as the bivariate von Mises (BVM) is typically used. The critical task involved in mixture modelling is to determine the optimal number of co…

New approach finds minima of geodesic lengths for non-uniform fillings.

Study minimum ribbonlength of immersed flat knots and links.

Data clustering has received a lot of attention and numerous methods, algorithms and software packages are available. Among these techniques, parametric finite-mixture models play a central role due to their interesting mathematical properties and to the existence of maximum-likelihood estimators based on expectation-m…

This paper proves a conjecture about trisections with a specific length.

A $triangulation$ is an embedding of a graph on surfaces where every face has length three. In this article, we show the existence of contractible Hamiltonian cycle in triangulated maps of which minimum degree is four.

We consider the relations between different measures of complexity for free homotopy classes of curves on a surface $Σ$, including the minimum number of self-intersections, the minimum length of the words representing them in a geometric presentation of $π_1(Σ)$, and the minimum degree of the coverings of $Σ$ to which …

Time-invariant linear dynamical system arises in many real-world applications,and its usefulness is widely acknowledged. A practical limitation with this model is that its latent dimension that has a large impact on the model capability needs to be manually specified. It can be demonstrated that a lower-order model cla…

We show that the volume of any Riemannian metric on a three sphere is bounded below by the length of the shortest closed curve that links its antipodal image. In particular, the volume is bounded below by the minimum of the length of the shortest closed geodesic and the minimal distance between antipodal points.

Minimum algebraic intersection found in hyperbolic surfaces, growing with genus.

HGNet improves GNNs' ability to handle long-range interactions in graphs.

An associative memory is a framework of content-addressable memory that stores a collection of message vectors (or a dataset) over a neural network while enabling a neurally feasible mechanism to recover any message in the dataset from its noisy version. Designing an associative memory requires addressing two main task…

Method estimates dataset utility via minimal program length proxy.

Matsumoto conjectured that for any Finsler manifold $(M, F)$ for which the restriction of the fundamental tensor to the indicatrix of $F$ is positive definite, the absolute length $F(X)$ of any tangent vector $X \in T_xM$ is the global minimum for the relative length $|X|_y$ as $y$ varies along the indicatrix $I_x \sub…

PCA (Principal Component Analysis) and its variants areubiquitous techniques for matrix dimension reduction and reduced-dimensionlatent-factor extraction. One significant challenge in using PCA, is thechoice of the number of principal components. The information-theoreticMDL (Minimum Description Length) principle gives…

The modelling of data on a spherical surface requires the consideration of directional probability distributions. To model asymmetrically distributed data on a three-dimensional sphere, Kent distributions are often used. The moment estimates of the parameters are typically used in modelling tasks involving Kent distrib…

The Fisher information approximation (FIA) is an implementation of the minimum description length principle for model selection. Unlike information criteria such as AIC or BIC, it has the advantage of taking the functional form of a model into account. Unfortunately, FIA can be misleading in finite samples, resulting i…

Knots have been considered to be useful models for simulating molecular chains such as DNA and proteins. One quantity that we are interested on molecular knots is the minimum number of monomers necessary to realize a knot. In this paper we consider every knot in the cubic lattice. Especially the minimal length of a kno…

Novel graph neural network combines random walks with local message passing.

Approximations of loopy belief propagation, including expectation propagation and approximate message passing, have attracted considerable attention for probabilistic inference problems. This paper proposes and analyzes a generalization of Opper and Winther's expectation consistent (EC) approximate inference method. Th…

We tackle the problem of penalty selection of regularization on the basis of the minimum description length (MDL) principle. In particular, we consider that the design space of the penalty function is high-dimensional. In this situation, the luckiness-normalized-maximum-likelihood(LNML)-minimization approach is favorab…

We investigate the time series of the degree of minimum spanning trees obtained by using a correlation based clustering procedure which is starting from (i) asset return and (ii) volatility time series. The minimum spanning tree is obtained at different times by computing correlation among time series over a time windo…

While the channel capacity reflects a theoretical upper bound on the achievable information transmission rate in the limit of infinitely many bits, it does not characterise the information transfer of a given encoding routine with finitely many bits. In this note, we characterise the quality of a code (i. e. a given en…

Active sampling algorithm improves accuracy of inferred scores from pairwise comparisons.

Paper establishes generalization bounds for representation learning using Minimum Description Length.

Neural networks generalize on simple data generated by a programming language.

Kernel networks' stability edge linked to Fisher Information singularity.

A new method avoids overfitting in network reconstruction by using the minimum description length principle.

Study on the minimum length of curves on once-punctured hyperbolic surfaces.