arXiv research

In this study, we prove that an intrinsic low dimensionality of covariates is the main factor that determines the performance of deep neural networks (DNNs). DNNs generally provide outstanding empirical performance. Hence, numerous studies have actively investigated the theoretical properties of DNNs to understand thei…

Deep multi-task learning benefits from low intrinsic dimensionality, leading to better generalization.

Paper examines global Covid-19 data complexity and finds low intrinsic dimensions.

Low-dimensional structure in images helps deep learning models generalize better.

New research shows DDPM can adapt to data's intrinsic low dimensionality efficiently.

Deep networks can adapt to intrinsic dimensionality beyond domain constraints.

Diffusion models learn multi-modal distributions with optimal efficiency.

New algorithm estimates intrinsic dimension of discrete datasets.

Novelty search in low-dimensional space improves sample efficiency in exploration tasks.

W2S FT often outperforms weak teachers due to low intrinsic dimensionality.

Wasserstein Autoencoders improve model efficiency and interpretability for low-dimensional data.

The input data features set for many data driven tasks is high-dimensional while the intrinsic dimension of the data is low. Data analysis methods aim to uncover the underlying low dimensional structure imposed by the low dimensional hidden parameters by utilizing distance metrics that consider the set of attributes as…

Estimates intrinsic dimension of data for GANs.

The paper refutes the manifold hypothesis for image data and proposes the union of manifolds hypothesis.

Enhances Gaussian process regression with multi-fidelity models and active subspaces for high-dimensional problems.

Generative models learn complex data from low-dimensional manifolds.

Study shows how diffusion models learn on low-dimensional manifolds.

Contrastive learning adapts to data intrinsic dimensions, learning low-dimensional representations.

GeLoRA optimizes LoRA fine-tuning by dynamically adjusting ranks based on intrinsic dimensionality.

Cryo-em images are found to be low-dimensional.

ConvResNets approximate Besov functions and classify on low-dimensional manifolds.

Paper adapts DDPM to low-dimensional structures in image distributions.

Modern large-scale datasets are frequently said to be high-dimensional. However, their data point clouds frequently possess structures, significantly decreasing their intrinsic dimensionality (ID) due to the presence of clusters, points being located close to low-dimensional varieties or fine-grained lumping. We test a…

Deep networks can approximate high-dimensional distributions from low-dimensional ones.

TOFU-POV tackles partially observed linear bandits, achieving sublinear regret with low-dimensional action vectors.

The paper proposes a least squares method for binary compressive sampling with low intrinsic dimension signals.

We consider the ability of deep neural networks to represent data that lies near a low-dimensional manifold in a high-dimensional space. We show that deep networks can efficiently extract the intrinsic, low-dimensional coordinates of such data. We first show that the first two layers of a deep network can exactly embed…

Isometry regularizer improves autoencoder performance on manifold learning.

Maximizes coding rate difference for robust, discriminative features.

Paper explores how Rectified Flow adapts to low-dimensional data.

Shallow nonlinear networks can separate classes linearly with polynomially scaling width.

New proof shows diffusion models implicitly estimate intrinsic dimensionality.

The study explains transformer scaling laws using statistical and approximation theories.

In this paper we propose and explore the k-Nearest Neighbour UCB algorithm for multi-armed bandits with covariates. We focus on a setting where the covariates are supported on a metric space of low intrinsic dimension, such as a manifold embedded within a high dimensional ambient feature space. The algorithm is concept…

Chart autoencoders learn latent features preserving manifold topology and geometry, with robust denoising capabilities.

A new method for SVGD reduces variance in high dimensions.

The paper tackles transfer learning for growing matrix representations, improving estimation accuracy.

The success of modern Artificial Intelligence (AI) technologies depends critically on the ability to learn non-linear functional dependencies from large, high dimensional data sets. Despite recent high-profile successes, empirical evidence indicates that the high predictive performance is often paired with low robustne…

New theory shows deep networks adapt to data's intrinsic dimensionality even when data isn't on a low-dimensional manifold.

We briefly recall a fundamental exterior differential system introduced by the author and then apply it to the case of three dimensions. Here we find new global tensors and intrinsic invariants of oriented Riemaniann 3-manifolds. The system leads to a remarkable Weingarten type equation for surfaces on hyperbolic 3-spa…

This paper analyzes deep federated learning for low-dimensional data, revealing intrinsic dimensionality's role in convergence rates.

Paper analyzes dataset distillation for efficient encoding of task-relevant information.

The paper improves GANs' theoretical guarantees for low-dimensional data.

Data living on manifolds commonly appear in many applications. Often this results from an inherently latent low-dimensional system being observed through higher dimensional measurements. We show that under certain conditions, it is possible to construct an intrinsic and isometric data representation, which respects an …

Real world data often exhibit low-dimensional geometric structures, and can be viewed as samples near a low-dimensional manifold. This paper studies nonparametric regression of Hölder functions on low-dimensional manifolds using deep ReLU networks. Suppose $n$ training data are sampled from a Hölder function in $\mathc…

In this paper, we build an organization of high-dimensional datasets that cannot be cleanly embedded into a low-dimensional representation due to missing entries and a subset of the features being irrelevant to modeling functions of interest. Our algorithm begins by defining coarse neighborhoods of the points and defin…

Scattering networks maximize separation on low-dimensional data.

Datasets such as images, text, or movies are embedded in high-dimensional spaces. However, in important cases such as images of objects, the statistical structure in the data constrains samples to a manifold of dramatically lower dimensionality. Learning to identify and extract task-relevant variables from this embedde…