arXiv research

Low-entropy surfaces can be flowed into spheres and cylinders.

Generic low-entropy hypersurfaces in 4-6D flow with only generic singularities.

Low entropy hypersurfaces in 4D are isotopic to a sphere.

Gradient flow in softmax models tends to produce low-entropy outputs.

Understanding the inductive bias of neural networks is critical to explaining their ability to generalise. Here, for one of the simplest neural networks -- a single-layer perceptron with n input neurons, one output neuron, and no threshold bias term -- we prove that upon random initialisation of weights, the a priori p…

We show that if $Σ\subset \mathbb R^4$ is a closed, connected hypersurface with entropy $λ(Σ)\leq λ(\mathbb{S}^2\times \mathbb R)$, then the level set flow of $Σ$ never disconnects. We also obtain a sharp version of the forward clearing out lemma for non-fattening flows in $\mathbb R^4$ of low entropy.

The entropy of a hypersurface is given by the supremum over all F-functionals with varying centers and scales, and is invariant under rigid motions and dilations. As a consequence of Huisken's monotonicity formula, entropy is non-increasing under mean curvature flow. We show here that a compact mean convex hypersurface…

We prove the asymptotic roundness under normalized Gauss curvature flow provided entropy is initially small enough.

Backwards uniqueness proved for flows with asymptotically conical singularities.

New framework detects near vs. far out-of-distribution samples for AI safety.

ProSelfLC improves robustness of deep neural networks by automatically deciding trust in predictions.

Research shows surfaces close to planes in Hausdorff distance.

In this article, we prove the mean convex neighborhood conjecture for the mean curvature flow of surfaces in $\mathbb{R}^3$. Namely, if the flow has a spherical or cylindrical singularity at a space-time point $X=(x,t)$, then there exists a positive $\varepsilon=\varepsilon(X)>0$ such that the flow is mean convex in a …

This paper proposes an improved active learning method using classification trees.

In this article, we extend the mean curvature flow with surgery to mean convex hypersurfaces with entropy less than $Λ_{n-2}$. In particular, 2-convexity is not assumed. Next we show the surgery flow with just the initial convexity assumption $H - \frac{\langle x, ν\rangle}{2} > 0$ is possible and as an application we …

New theorem finds new minimal hypersurfaces in hyperbolic space.

Curve shortening flow converges to a point with entropy bound.

The study proves compactness and existence of entropy minimizers for self-shrinking surfaces.

Smooth flows with surgery approximate weak mean curvature flows with spherical and neck-pinch singularities.

The paper proves smoothness of mean curvature flow for generic initial data in 3D and 4D.

We show compactness in the locally smooth topology for certain natural families of asymptotically conical self-expanding solutions of mean curvature flow. Specifically, we show such compactness for the set of all two-dimensional self-expanders of a fixed topological type and, in all dimensions, for the set of self-expa…

We present a new method of generating mixture models for data with categorical attributes. The keys to this approach are an entropy-based density metric in categorical space and annealing of high-entropy/low-density components from an initial state with many components. Pruning of low-density components using the entro…

In this paper, we generalize White's regularity and structure theory for mean-convex mean curvature flow to the setting with free boundary. A major new challenge in the free boundary setting is to derive an a priori bound for the ratio between the norm of the second fundamental form and the mean curvature. We establish…

Study curve shortening flow in high dimensions with boundary constraints.

New estimator reduces variance in discrete random variables.

Density destructors simplify complex PDFs to maximize entropy, linking to information theory.

In this manuscript we propose two objective terms for neural image compression: a compression objective and a cycle loss. These terms are applied on the encoder output of an autoencoder and are used in combination with reconstruction losses. The compression objective encourages sparsity and low entropy in the activatio…

In this paper we apply a compressibility loss that enables learning highly compressible neural network weights. The loss was previously proposed as a measure of negated sparsity of a signal, yet in this paper we show that minimizing this loss also enforces the non-zero parts of the signal to have very low entropy, thus…

Semi-supervised learning has proven to be a powerful paradigm for leveraging unlabeled data to mitigate the reliance on large labeled datasets. In this work, we unify the current dominant approaches for semi-supervised learning to produce a new algorithm, MixMatch, that works by guessing low-entropy labels for data-aug…

The Weyl curvature hypothesis of Penrose attempts to explain the high homogeneity and isotropy, and the very low entropy of the early universe, by conjecturing the vanishing of the Weyl tensor at the Big-Bang singularity. In previous papers it has been proposed an equivalent form of Einstein's equation, which extends i…

We investigate the relative information efficiency of financial markets by measuring the entropy of the time series of high frequency data. Our tool to measure efficiency is the Shannon entropy, applied to 2-symbol and 3-symbol discretisations of the data. Analysing 1-minute and 5-minute price time series of 55 Exchang…

Generative networks are analyzed using spline operators to understand their properties and limitations.

High-frequency trading models fail due to overfitting and survivor bias.

In many applications the process of generating label information is expensive and time consuming. We present a new method that combines active and semi-supervised deep learning to achieve high generalization performance from a deep convolutional neural network with as few known labels as possible. In a setting where a …

Deep learning models are known to be overconfident in their predictions on out of distribution inputs. There have been several pieces of work to address this issue, including a number of approaches for building Bayesian neural networks, as well as closely related work on detection of out of distribution samples. Recent…

We study the problem of discovering the simplest latent variable that can make two observed discrete variables conditionally independent. The minimum entropy required for such a latent is known as common entropy in information theory. We extend this notion to Renyi common entropy by minimizing the Renyi entropy of the …

Hyperfitting improves LLM generation quality by enhancing diversity, contrary to simple temperature scaling.

Self-attention networks localize when eigenspectrum variance is small.

We study a flow of $G_2$ structures which induce the same Riemannian metric which is the negative gradient flow of an energy functional. We prove Shi-type estimates for the torsion tensor along the flow. We show that at a finite-time singularity the torsion must blow-up, so the flow exists as long as the torsion remain…

Two new metrics assess LLM faithfulness and entropy, improving model reliability.

Entropy-based decoding improves DLM sampling efficiency.

Higher conservative training increases reward-hacking in reasoning models.

A new loss function improves neural networks' out-of-distribution detection without side effects.

At the core of any inference procedure in deep neural networks are dot product operations, which are the component that require the highest computational resources. A common approach to reduce the cost of inference is to reduce its memory complexity by lowering the entropy of the weight matrices of the neural network, …

In the Minority, Majority and Dollar Games (MG, MAJG, $G), synthetic agents compete for rewards, at each time-step acting in accord with the previously best-performing of their limited sets of strategies. Different components and/or aspects of real-world financial markets are modelled by these games. In the MG, agents …

Active learning selects most informative unlabeled samples for labeling.

A toy model shows how locality can emerge in the universe's Hamiltonian and initial state.