arXiv research

Proposes RN for unsupervised attention in neural networks.

We introduce new definitions of universal and superuniversal computable codes, which are based on a code's ability to approximate Kolmogorov complexity within the prescribed margin for all individual sequences from a given set. Such sets of sequences may be singled out almost surely with respect to certain probability …

Develops a universal waveform selection scheme for radar tracking.

Paper proves method for calculating NML code length works for continuous models.

New technique for flow models achieves theoretical compression lengths.

The normalized maximized likelihood (NML) provides the minimax regret solution in universal data compression, gambling, and prediction, and it plays an essential role in the minimum description length (MDL) method of statistical modeling and estimation. Here we show that the normalized maximum likelihood has a Bayes-li…

New coding theorem shows achievable rate matches theoretical limit.

Each element of the commutator subgroup of a group can be represented as a product of commutators. The minimal number of factors in such a product is called the commutator length of the element. The commutator length of a group is defined as the supremum of commutator lengths of elements of its commutator subgroup. We …

Study of manifolds with prime cyclic group actions and curvature properties.

FibQuant improves KV-cache compression for long-context inference.

New quantum codes improve error correction with local tests.

A basic question in the theory of fault-tolerant quantum computation is to understand the fundamental resource costs for performing a universal logical set of gates on encoded qubits to arbitrary accuracy. Here we consider qubits encoded with constant space overhead (i.e. finite encoding rate) in the limit of arbitrari…

Transformer pretraining yields strong EB performance without explicit adaptation.

Universal perturbations misclassify text with high accuracy.

For reliable transmission across a noisy communication channel, classical results from information theory show that it is asymptotically optimal to separate out the source and channel coding processes. However, this decomposition can fall short in the finite bit-length regime, as it requires non-trivial tuning of hand-…

RATQ is a new quantizer for optimizing noisy gradients in machine learning.

In the Friedmann Model of the universe, cosmologists assume that spacelike slices of the universe are Riemannian manifolds of constant sectional curvature. This assumption is justified via Schur's Theorem by stating that the spacelike universe is locally isotropic. Here we define a Riemannian manifold as almost locally…

In this survey article we will consider universal lower bounds on the volume of a Riemannian manifold, given in terms of the volume of lower dimensional objects (primarily the lengths of geodesics). By `universal' we mean without curvature assumptions. The restriction to results with no (or only minimal) curvature assu…

ICQ improves high-dimensional similarity search without sacrificing precision.

Paper proposes a method to improve semantic segmentation for fisheye urban driving images.

Sparse data models, where data is assumed to be well represented as a linear combination of a few elements from a dictionary, have gained considerable attention in recent years, and their use has led to state-of-the-art results in many signal and image processing tasks. It is now well understood that the choice of the …

Paper trains a Transformer to add numbers of any length.

Study proves deep narrow RNNs can approximate any function, with minimum width independent of data length.

We discuss algorithms for estimating the Shannon entropy h of finite symbol sequences with long range correlations. In particular, we consider algorithms which estimate h from the code lengths produced by some compression algorithm. Our interest is in describing their convergence with sequence length, assuming no limit…

The Minimum Description Length (MDL) principle states that the optimal model for a given data set is that which compresses it best. Due to practial limitations the model can be restricted to a class such as linear regression models, which we address in this study. As in other formulations such as the LASSO and forward …

Constrained sequence codes have been widely used in modern communication and data storage systems. Sequences encoded with constrained sequence codes satisfy constraints imposed by the physical channel, hence enabling efficient and reliable transmission of coded symbols. Traditional encoding and decoding of constrained …

This paper gives a quantitative version of Thurston's hyperbolic Dehn surgery theorem. Applications include the first universal bounds on the number of non-hyperbolic Dehn fillings on a cusped hyperbolic 3-manifold, and estimates on the changes in volume and core geodesic length during hyperbolic Dehn filling. The proo…

DeepJSCC-f uses feedback to improve image transmission quality.

Based on empirical financial time-series, we show that the "silence-breaking" probability follows a super-universal power law: the probability of observing a large movement is inversely proportional to the length of the on-going low-variability period. Such a scaling law has been previously predicted theoretically [R. …

We prove that on a compact $n$-dimensional spin manifold admitting a non-trivial harmonic 1-form of constant length, every eigenvalue $λ$ of the Dirac operator satisfies the inequality $λ^2 \geq \frac{n-1}{4(n-2)}\inf_M Scal$. In the limiting case the universal cover of the manifold is isometric to $R\times N$ where $N…

Framework for universal graph function approximators outperforms existing methods.

For curves of prescribed length embedded into the unit disc in two dimensions, we obtain scaling results for the minimal elastic energy as the length just exceeds $2π$ and in the large length limit. In the small excess length case, we prove convergence to a fourth order obstacle type problem with integral constraint on…

Time series constitute a challenging data type for machine learning algorithms, due to their highly variable lengths and sparse labeling in practice. In this paper, we tackle this challenge by proposing an unsupervised method to learn universal embeddings of time series. Unlike previous works, it is scalable with respe…

Paper reinterprets majorizing measure theorem in terms of coding theory.

The design of codes for communicating reliably over a statistically well defined channel is an important endeavor involving deep mathematical research and wide-ranging practical applications. In this work, we present the first family of codes obtained via deep learning, which significantly beats state-of-the-art codes …

Extended systolic inequality for 2-complexes to improve group systolic area bounds.

Paper creates universal adversarial attacks.

Sumformer simplifies Transformers to handle long sequences efficiently.

This paper addresses the nearest neighbor search problem under inner product similarity and introduces a compact code-based approach. The idea is to approximate a vector using the composition of several elements selected from a source dictionary and to represent this vector by a short code composed of the indices of th…

Non-parametric estimators improve quickest changepoint detection under irregular sequence lengths.

Suppose $C$ is a compact, $n$-edged two-cell of the centered dual decomposition of a locally finite set in the hyperbolic plane, a coarsening of the Delaunay tessellation which was introduced in the author's prior work. We describe an effectively computable lower bound on the area of $C$, given an $n$-tuple of positive…

In 1992, Reid asked whether hyperbolic 3-manifolds with the same geodesic length spectra are necessarily commensurable. While this is known to be true for arithmetic hyperbolic 3-manifolds, the non-arithmetic case is still open. Building towards a negative answer to this question, Futer and Millichap recently construct…

Reasoning models generate differently based on problem difficulty, not just length.

We study various covering spectra for complete noncompact length spaces with universal covers (including Riemannian manifolds and the pointed Gromov Hausdorff limits of Riemannian manifolds with lower bounds on their Ricci curvature). We relate the covering spectrum to the (marked) shift spectrum of such a space. We de…

This paper has two parts, on Baumslag-Solitar groups and on general G-trees. In the first part we establish bounds for stable commutator length (scl) in Baumslag-Solitar groups. For a certain class of elements, we further show that scl is computable and takes rational values. We also determine exactly which of these el…

PieClam autoencodes graphs into communities, improving graph anomaly detection.

Study on reducing forgetting in neural networks using compression theory.

In section 1 we reformulate a theorem of Blichfeldt in the framework of manifolds of nonpositive curvature. As a result we obtain a lower bound on the number of homotopically distinct geodesic loops emanating from a common point q whose length is smaller than a fixed constant. This bound depends only on the volume grow…