Estimates intrinsic dimension of data for GANs.
problem Estimating intrinsic dimension of high-dimensional data.
method Uses Wasserstein distances for estimation.
result Provides sample complexity bounds for GANs.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “intrinsic estimation”
The intrinsic entropy model accurately estimates stock market volatility.
problem Accurately estimating historical volatility of stock market indices.
method Incorporates traded volumes alongside OHLC prices in daily data.
result Intrinsic entropy model delivers reliable estimates with lower coefficient of variation.
Geometry-Aware Optimal Transport: Fast Intrinsic Dimension and Wasserstein Distance Estimationstat.ML
New estimators for intrinsic dimension and Wasserstein distance improve OT accuracy.
problem Intrinsic dimension estimation and Wasserstein distance estimation in large-scale OT.
method Introduces novel estimators for intrinsic dimension and Wasserstein distance.
result Simple, tuning-free estimator of OT and fast intrinsic dimension estimator.
The paper corrects biases in estimating intrinsic dimension and differential entropy.
problem Systematic bias in estimating intrinsic dimension and differential entropy.
method A bias-corrected estimator for both measures is proposed, highlighting shared steps and useful consequences.
result Simultaneous estimation of differential entropy and intrinsic dimension provides complementary perspectives on underlying manifolds.
New method estimates intrinsic dimensionality using angles, not distances.
problem Estimating local intrinsic dimensionality accurately.
method Introduces a new estimator using the distribution of angles between neighbor points.
result New estimator behaves similarly but complementarily to existing measures of intrinsic dimensionality.
eDCF estimates intrinsic dimension using local connectivity.
problem Challenges in estimating intrinsic dimension due to scale dependence.
method eDCF: a novel, scalable, and parallelizable method based on Connectivity Factor (CF).
result eDCF consistently matches leading estimators with comparable MAE and higher exact intrinsic dimension match rates.
LIDL estimates local intrinsic dimension in high dimensions.
problem Estimating local intrinsic dimension in high-dimensional data.
method Approximate likelihood using parametric neural density estimation.
result LIDL scales to thousands of dimensions and yields competitive results.
Estimates scalar curvature of point clouds without embedding.
problem Estimating scalar curvature of data sets without embedding.
method Intrinsic estimator based on metric structure, consistent and stable.
result Estimator converges to scalar curvature as sample size increases.
Python package for estimating intrinsic dimensionality of datasets.
problem Estimating intrinsic dimensionality for machine learning applications.
method Implementation of various intrinsic dimension estimators in Python.
result Benchmarking of ID estimation methods on real and synthetic data.
We propose a new method for estimating the intrinsic dimension of a dataset by applying the principle of regularized maximum likelihood to the distances between close neighbors. We propose a regularization scheme which is motivated by divergence minimization principles. We derive the estimator by a Poisson process appr…
Study provides bounds for estimating intrinsic dimension using Gaussian kernels.
problem Estimating intrinsic dimension from data.
method Finite-sample concentration and anti-concentration bounds for Gaussian kernel sums.
result Explicit dependence on sample size, bandwidth, and geometric parameters.
Efficient method estimates intrinsic dimension for big data.
problem Estimating intrinsic dimension for large datasets is costly and complex.
method Proposes a matrix-vector product-based approach for efficient intrinsic dimension estimation.
result Demonstrates superior performance compared to state-of-the-art methods.
We consider non-parametric estimation and inference of conditional moment models in high dimensions. We show that even when the dimension of the conditioning variable is larger than the sample size , estimation and inference is feasible as long as the distribution of the conditioning variable has small intrinsic…
Posterior Contraction for Sparse Neural Networks in Besov Spaces with Intrinsic Dimensionalitystat.ML
Bayesian neural networks achieve optimal posterior contraction rates in Besov spaces with intrinsic dimensionality.
problem High-dimensional structured estimation problems with unknown smoothness levels.
method Sparse Bayesian neural networks with either sparse or continuous shrinkage priors.
result Optimal posterior contraction rates are achieved, adapting to the unknown smoothness level of the true function.
The paper improves ranking by integrating covariates and sparse intrinsic scores.
problem Ranking items with incomplete preference scores explained by covariates.
method Extends BTL model with covariate information and sparse intrinsic scores, using penalized MLE.
result Developed debiased estimator for penalized MLE with distributional properties.
Robust estimation of the intrinsic dimension of data sets with quantum cognition machine learningstat.ML
Estimates intrinsic dimension of data sets robustly to noise.
problem Estimating intrinsic dimension of noisy data sets.
method Quantum Cognition Machine Learning for data representation and spectral gap detection.
result Robust estimation of intrinsic dimension in the presence of Gaussian noise.
The paper examines Wiener process for LID estimation methods.
problem Estimating local intrinsic dimension in high-dimensional datasets.
method Investigates recent LID estimation methods from a Wiener process perspective.
result Explains how methods behave under non-ideal conditions.
Rdimtools simplifies DR and IDE for high-dimensional data analysis.
problem Discovering patterns in complex high-dimensional data.
method Provides an R package with 133 DR and 17 IDE algorithms.
result Facilitates geometric understanding of high-dimensional data.
Paper introduces CSIE for estimating stock market volatility.
problem Temporal uncertainty in stock market volatility.
method Cross-sectional intrinsic entropy model based on OHLC prices.
result CSIE is 10 times more sensitive to market changes.
We study the stability of the Positive Mass Theorem using the Intrinsic Flat Distance. In particular we consider the class of complete asymptotically flat rotationally symmetric Riemannian manifolds with nonnegative scalar curvature and no interior closed minimal surfaces whose boundaries are either outermost minimal h…
Improved estimation of concentration using half-spaces for adversarial vulnerability.
problem Understanding the concentration of measure phenomenon and its impact on adversarial vulnerability.
method Extending Gaussian Isoperimetric Inequality to non-spherical Gaussian measures and arbitrary ℓ_p-norms, using half-spaces to estimate concentration.
result Proposed method finds tighter intrinsic robustness bounds, providing evidence against concentration as a cause of adversarial vulnerability.
New algorithm estimates intrinsic dimension of discrete datasets.
problem Inaccuracies in using continuous methods for discrete datasets.
method Introduced an algorithm to infer intrinsic dimension of discrete spaces.
result Demonstrated accuracy on benchmark datasets and found a small intrinsic dimension in a metagenomic dataset.
Sharp estimates for Finsler metrics in convex domains.
problem Estimating distances in Finsler metrics near convex points.
method Sharp estimates for intrinsic distances of Finsler metrics.
result Characterization of -quasi hyperbolic metric in convex geometry.
New score matching method estimates local intrinsic dimension efficiently.
problem Quantifying the local intrinsic dimension of complex data.
method Denoising score matching loss and equivalent implicit score matching loss.
result Denoising score matching loss is a highly competitive and scalable LID estimator.
Estimating the effective dimension of large biological datasets using Fisher separability analysiscs.LG
Modern large-scale datasets are frequently said to be high-dimensional. However, their data point clouds frequently possess structures, significantly decreasing their intrinsic dimensionality (ID) due to the presence of clusters, points being located close to low-dimensional varieties or fine-grained lumping. We test a…
We give new estimates for the eigenvalues of the hypersurface Dirac operator in terms of the intrinsic energy-momentum tensor, the mean curvature and the scalar curvature. We also discuss their limiting cases as well as the limiting cases of the estimates obtained by X. Zhang and O. Hijazi in [13] and [10]. We compare …
The existing approaches to intrinsic dimension estimation usually are not reliable when the data are nonlinearly embedded in the high dimensional space. In this work, we show that the explicit accounting to geometric properties of unknown support leads to the polynomial correction to the standard maximum likelihood est…
Minimal surfaces' area bounds proven equivalent, extending known results.
problem Equivalence of area bounds for minimal surfaces.
method Combining recent breakthroughs, extending known results.
result Equivalence of intrinsic and extrinsic area density bounds for minimal immersions.
Low-dimensional structure in images helps deep learning models generalize better.
problem Understanding the intrinsic dimensionality of images for better model performance.
method Applied dimension estimation tools to popular image datasets and used GANs to manipulate intrinsic dimensionality.
result Natural image datasets have very low intrinsic dimensionality, which aids neural networks in learning and generalizing.
Estimates submanifold diameters in curved spaces.
problem Estimating the intrinsic diameter of submanifolds in curved spaces.
method Using mean curvature field integrals and boundary lengths.
result Diameter estimates for submanifolds in curved spaces.
Paper extends Bayesian Cramér-Rao bound with geometric considerations.
problem Estimation of covariance matrices with geometric structures.
method Intrinsic Bayesian Cramér-Rao bound with Riemannian geometry.
result Performance bounds for covariance matrix estimation.
New method tightens sub-Gaussian concentration inequalities.
problem Estimating variance-type parameters of sub-Gaussian distributions.
method Using sub-Gaussian intrinsic moment norm to maximize normalized moments.
result Provides tighter sub-Gaussian concentration inequalities.
New research shows common ID estimators in neural representations are inaccurate.
problem Inaccurate estimation of intrinsic dimensions in neural representations.
method Theoretical and empirical investigation of ID estimators in neural representations.
result Common ID estimators do not accurately reflect the true underlying ID of neural representations.
Scale-adaptive and robust intrinsic dimension estimation via optimal neighbourhood identificationstat.ML
This work introduces a protocol to automatically select the correct range of scales for meaningful Intrinsic Dimension estimation.
problem The Intrinsic Dimension (ID) varies with scale in real-world datasets, leading to erroneous results.
method The protocol selects the correct range of scales by ensuring constant density of data points.
result The method provides a robust and scale-adaptive approach to estimating meaningful Intrinsic Dimension.
This paper addresses the following questions pertaining to the intrinsic dimensionality of any given image representation: (i) estimate its intrinsic dimensionality, (ii) develop a deep neural network based non-linear mapping, dubbed DeepMDS, that transforms the ambient representation to the minimal intrinsic space, an…
We give lower bounds for the eigenvalues of the submanifold Dirac operator in terms of intrinsic and extrinsic curvature expressions. We also show that the limiting cases give rise to a class generalizing that of Killing spinors. We conclude by translating these results in terms of intrinsic twisted Dirac operators.
A new method uses diffusion models to efficiently estimate local intrinsic dimensionality of data.
problem Estimating the local intrinsic dimensionality of high-dimensional data.
method Developed a method using the Fokker-Planck equation associated with diffusion models to estimate local intrinsic dimensionality.
result Diffusion models can effectively estimate local intrinsic dimensionality, outperforming existing methods in accuracy and speed.
New proof shows diffusion models implicitly estimate intrinsic dimensionality.
problem Estimating intrinsic dimensionality of data from diffusion models.
method Formal proof of FLIPD under realistic assumptions.
result FLIPD's correctness proven under realistic conditions.
The paper analyzes the intrinsic exploration terms in policy-gradient algorithms.
problem Exploration in policy-gradient algorithms and its impact on policy optimization.
method Numerical optimization criteria and stochastic gradient analysis.
result Exploration techniques improve policy optimization by smoothing the learning objective and modifying gradient estimates.
Efficient exploration remains a challenging problem in reinforcement learning, especially for those tasks where rewards from environments are sparse. A commonly used approach for exploring such environments is to introduce some "intrinsic" reward. In this work, we focus on model uncertainty estimation as an intrinsic r…
In this paper we prove an extrinsic one-sided curvature estimate for disks embedded in with constant mean curvature which is independent of the value of the constant mean curvature. We apply this extrinsic one-sided curvature estimate in [24] to prove to prove a weak chord arc type result for these disks…
In the last decades the estimation of the intrinsic dimensionality of a dataset has gained considerable importance. Despite the great deal of research work devoted to this task, most of the proposed solutions prove to be unreliable when the intrinsic dimensionality of the input dataset is high and the manifold where th…
Intrinsic dimensionality (ID) is one of the most fundamental characteristics of multi-dimensional data point clouds. Knowing ID is crucial to choose the appropriate machine learning approach as well as to understand its behavior and validate it. ID can be computed globally for the whole data point distribution, or comp…
Optimally estimate distances on surfaces using reconstructed meshes.
problem Estimating intrinsic distances on smooth submanifolds.
method Reconstruction of the surface using a tangential Delaunay complex, and Isomap variant.
result Minimax optimality achieved for distance estimation.
Improved manifold-adaptive dimension estimator for better data complexity assessment.
problem Estimating intrinsic dimensionality of complex data.
method Revised and improved Farahmand-Szepesvári-Audibert (FSA) estimator, incorporating probability density function and median.
result Median-FSA estimator outperforms existing methods in accuracy and robustness.
Information about intrinsic dimension is crucial to perform dimensionality reduction, compress information, design efficient algorithms, and do statistical adaptation. In this paper we propose an estimator for the intrinsic dimension of a data set. The estimator is based on binary neighbourhood information about the ob…
Novel framework improves GNN uncertainty estimates under distribution shifts.
problem Improving reliability of GNN uncertainty estimates under distribution shifts.
method Adapting stochastic data centering to graph data through novel graph anchoring strategies.
result G-UQ leads to better calibrated GNNs for node and graph classification.
Develops new oracle inequalities for Gaussian ranking estimators.
problem Lack of rigorous theoretical support for Gaussian ranking estimators.
method Novel oracle inequalities for regularized pairwise ranking.
result Derives fast learning rates under general dimension assumptions.