The Bonnet theorem is proven for statistical manifolds.
problem Locally embeddable statistical manifolds in flat spaces.
method Using statistical embedding and the Gauss--Codazzi--Ricci equations.
result Statistical manifolds with specific tensor properties are locally embeddable to flat statistical manifolds.
In this note we prove certain necessary and sufficient conditions for the existence of an embedding of statistical manifolds. In particular, we prove that any compact smooth (C1 resp.) statistical manifold can be embedded into the space of probability measures on a finite set. As a result, we get an answer to the La…
This work analyzes label embedding for large multiclass classification problems.
problem Label embedding for large multiclass classification problems.
method Analysis of label embedding in extreme multiclass classification, presenting an excess risk bound and showing a trade-off between computational and statistical efficiency.
result The statistical penalty for label embedding vanishes with sufficiently low coherence under the Massart noise condition.
This paper develops embeddings that preserve likelihood-based statistical inference.
problem Modern machine learning embeddings destroy the geometric structure required for likelihood-based inference.
method Developed a rigorous theory of likelihood-preserving embeddings and introduced the Likelihood-Ratio Distortion metric.
result Controlling the distortion Δn is necessary and sufficient for preserving inference. New test for conditional independence using kernel embeddings.
problem Testing conditional independence in high-dimensional settings.
method Analytic kernel embeddings, asymptotic distribution.
result New test outperforms existing methods in high-dimensional settings.
This paper connects functional data analysis with machine learning techniques.
problem Lack of theoretical analysis for functional depths.
method Viewing functional depths as kernel mean embeddings in machine learning.
result Facilitates answers to open questions about functional depths.
We propose a novel node embedding of directed graphs to statistical manifolds, which is based on a global minimization of pairwise relative entropy and graph geodesics in a non-linear way. Each node is encoded with a probability density function over a measurable space. Furthermore, we analyze the connection between th…
Transformers encode latent distributions in text, improving performance in out-of-distribution cases.
problem What should embeddings from language models represent?
method Connecting autoregressive prediction to sufficient statistics, identifying three settings.
result Transformers encode latent generating distributions, improving performance.
This paper provides a dictionary of closed-form kernel mean embeddings.
problem Challenges in deriving closed-form kernel mean embeddings.
method Comprehensive dictionary and practical tools for deriving new embeddings.
result Provides a Python library with minimal implementations of embeddings.
In this paper, we aim to learn a low-dimensional Euclidean representation from a set of constraints of the form "item j is closer to item i than item k". Existing approaches for this "ordinal embedding" problem require expensive optimization procedures, which cannot scale to handle increasingly larger datasets. To addr…
The paper provides concentration bounds for embeddings of generative models.
problem Establishing accurate statistical analysis of generative models.
method Data Kernel Perspective Space embedding method.
result Required number of sample responses for accurate approximation.
Efficient RL in large POMDPs with latent determinism and embeddings.
problem Efficient reinforcement learning in large-scale POMDPs with latent states and observations.
method Conditional Hilbert space embeddings, linear optimal Q-function, deterministic latent transitions, gap assumption. result Computationally and statistically efficient algorithm for exact optimal policy.
Novel approach to OT using kernel mean embeddings controls overfitting and achieves dimension-free sample complexity.
problem Consistently estimate optimal transport plan from samples.
method Pose OT as learning kernel mean embedding, employ MMD regularization.
result ε-optimal recovery of transport plan and map with dimension-free sample complexity.
The paper establishes concentration bounds for embeddings of generative models.
problem Analyzing statistical properties of generative models.
method High probability concentration bounds on sample vector embeddings using Data Kernel Perspective Space.
result Determines the number of samples needed for accurate approximation of generative model embeddings.
Modern LLMs fail at authorship attribution without fine-tuning, but topic embeddings outperform them.
problem Authorship attribution of the Federalist Papers using modern LLMs.
method Examined popular LLMs, compared word/phrase embeddings, and used Bayesian analysis with topic embeddings.
result Topic embeddings trained on 'function words' outperform default LLM embeddings in authorship attribution.
Paper develops a unified framework for measuring differences between conditional distributions.
problem Comparing conditional distributions in a unified and theoretically sound manner.
method Kernel embeddings and conditional maximum mean discrepancy (CMMD) framework.
result Established a coherent framework for measuring divergence between conditional distributions.
Study how knots occupy space using topological methods.
problem Understanding how knots occupy a volume of space.
method Statistical analysis of persistent homology features from Vietoris-Rips complexes.
result Existence of correlations between geometric and topological features of knots.
Diffusion means converge to extrinsic means for long times on spheres.
problem Understanding the long-time behavior of diffusion means on manifolds.
method Introduced diffusion means as a parameterized family of location statistics on manifolds, and analyzed their convergence to extrinsic means for long times.
result For real projective spaces and connected compact symmetric spaces, the long-time limit of diffusion means is conjectured to be the extrinsic mean in the isometric embedding.
Paper shows graphs can be embedded in lower dimensions than expected.
problem Choosing the right embedding dimension for graph analysis.
method Utilizes hidden manifold structure to predict lower-dimensional embedding.
result Graphs can be embedded in much lower dimensions than previously thought.
A statistical test of independence may be constructed using the Hilbert-Schmidt Independence Criterion (HSIC) as a test statistic. The HSIC is defined as the distance between the embedding of the joint distribution, and the embedding of the product of the marginals, in a Reproducing Kernel Hilbert Space (RKHS). It has …
Paper establishes identifiability conditions for a model with two latent vectors and auxiliary data.
problem Identifying conditions for a statistical model with two latent vectors and auxiliary data.
method Proposes a statistical model with two latent vectors and auxiliary data, establishing various identifiability conditions.
result Identifiability conditions reveal a dimensionality relation and link model indeterminacies to maximum link weights.
Deep-embedding methods aim to discover representations of a domain that make explicit the domain's class structure and thereby support few-shot learning. Disentangling methods aim to make explicit compositional or factorial structure. We combine these two active but independent lines of research and propose a new parad…
CPME embeds counterfactual outcomes in RKHS for flexible policy evaluation.
problem Estimating counterfactual policy outcomes for decision-making.
method Counterfactual Policy Mean Embedding (CPME) framework in RKHS, plug-in and doubly robust estimators, kernel test statistic.
result Doubly robust estimator improves convergence rates and asymptotic normality.
A tractable pseudo-metric for non-parametric distributions via SPD geometry.
problem Computing distances between non-parametric probability distributions is intractable.
method Two-stage framework: projection onto parametric family, embedding into SPD matrices.
result Closed-form pseudo-metric for two-sample hypothesis testing.
Proposes QQE for transforming and embedding data distributions.
problem Transforming and embedding data distributions for better representation or visualization.
method Quantile-Quantile Embedding (QQE) using quantile-quantile plot concept.
result QQE allows for better discrimination of classes in some cases.
MMbeddings reduces categorical embeddings by treating them as latent effects, significantly decreasing parameters and mitigating overfitting.
problem Large cardinalities in categorical embeddings lead to high parameter counts and overfitting.
method MMbeddings treats embeddings as latent random effects in a variational autoencoder framework, reducing parameter count and mitigating overfitting.
result MMbeddings consistently outperforms traditional embeddings across various tasks, demonstrating its potential in machine learning applications.
A novel kernel-based test detects equality versus singularity of two probability measures.
problem Detecting equality versus singularity of two probability distributions.
method Combines kernel mean and kernel covariance embeddings to construct a likelihood ratio test statistic.
result The test statistic satisfies a '0/\infty' law, vanishing under the null and diverging under the alternative.
We show how to learn low-dimensional representations (embeddings) of patient visits from the corresponding electronic health record (EHR) where International Classification of Diseases (ICD) diagnosis codes are removed. We expect that these embeddings will be useful for the construction of predictive statistical models…
Method embeds numeric tabular datasets into a shared vector space for similarity and retrieval.
problem Lack of meaningful representation for numeric tabular datasets in large language models.
method Structured exploratory data analysis descriptors, sentence transformer embedding, CCA for cross-dataset alignment.
result Total P@1 score of 0.9 across 15 datasets, robust nearest-neighbor retrieval and cluster structure.
Study optimizes learning rates for conditional mean embedding estimates.
problem Consistency of kernel ridge regression for conditional mean embedding.
method Adaptive statistical learning rate derived for misspecified setting.
result Upper bound matches optimal O(logn/n) rates without assuming finite dimensionality. Semantic word embeddings represent the meaning of a word via a vector, and are created by diverse methods. Many use nonlinear operations on co-occurrence statistics, and have hand-tuned hyperparameters and reweighting methods. This paper proposes a new generative model, a dynamic version of the log-linear topic model o…
TSFMs embed non-stationary time series data, revealing specific types of changes.
problem Understanding non-stationarity in TSFMs' embedding spaces.
method Examined mean shifts, variance changes, linear trends, and persistence in TSFMs.
result Different TSFMs exhibit distinct failure modes in detecting non-stationarity.
We lay theoretical foundations for new database release mechanisms that allow third-parties to construct consistent estimators of population statistics, while ensuring that the privacy of each individual contributing to the database is protected. The proposed framework rests on two main ideas. First, releasing (an esti…
New tests for binary classification regression functions without distribution assumptions.
problem Testing regression functions in binary classification without distributional assumptions.
method Conditional kernel mean embeddings and resampling-based framework.
result Distribution-free hypothesis tests with exact type I error control.
News embeddings improve volatility forecasts.
problem Improving volatility forecasting accuracy.
method Transformed news text into embeddings, evaluated standalone and combined with benchmarks.
result News contains useful predictive information, especially for stock-related content.
The paper introduces Relative Bias to quantify LLM bias systematically.
problem Quantifying bias in LLMs is challenging due to ambiguity and rapid model emergence.
method Relative Bias framework using Embedding Transformation and LLM-as-a-Judge methodologies.
result The two scoring methods show strong alignment, providing a systematic approach.
Complicated generative models often result in a situation where computing the likelihood of observed data is intractable, while simulating from the conditional density given a parameter value is relatively easy. Approximate Bayesian Computation (ABC) is a paradigm that enables simulation-based posterior inference in su…
DKMD is a fast signed statistic for comparing univariate distributions.
problem Comparing univariate distributions, especially preserving directionality.
method DKMD integrates kernel mean embeddings against an odd weighting function.
result DKMD preserves directionality and is robust to outliers.
In statistical relational learning, the link prediction problem is key to automatically understand the structure of large knowledge bases. As in previous studies, we propose to solve this problem through latent factorization. However, here we make use of complex valued embeddings. The composition of complex embeddings …
Survey of embedding methods for high-dimensional and network data.
problem Embedding high-dimensional and nonlinear data structures in a lower-dimensional space.
method Survey of various embedding methods including principal curves, multidimensional scaling, graph-based methods, and topological embeddings.
result Discussion of the pros and cons of algorithmic machine learning and statistical modeling approaches.
This paper proposes a new method for embedding sequences using Wasserstein distances.
problem Embedding sequences in a metric space for better pattern recognition.
method Develops a deep learning model that embeds sequences as distributions and uses Wasserstein distances for comparison.
result Distributional embeddings using Wasserstein distances outperform traditional vector embeddings.
Perfect clustering achieved in hypergraphs with enough interactions.
problem Complexity and lack of tractable models for analyzing hypergraphs.
method Introduced an interaction hypergraph model for analyzing hypergraphs, defined latent embeddings, and analyzed spectral estimators.
result A spectral estimate of interaction latent positions can achieve perfect clustering with enough interactions.
Proposes statistical inference for dependency knowledge graphs from EHR data.
problem Statistical uncertainty in linking entities in EHR data.
method Dynamic log-linear topic model with singular value decomposition.
result Established asymptotic normality for sparse graph edge recovery.
The paper tackles data misappropriation in LLMs by embedding watermarks and testing for their presence.
problem Detecting data misappropriation in LLMs trained on copyrighted data.
method Embedding watermarks, formulating as hypothesis testing, developing statistical framework, constructing test statistics, determining optimal thresholds, controlling errors, establishing asymptotic optimality.
result The proposed statistical testing framework effectively detects data misappropriation in LLMs.
In machine learning or statistics, it is often desirable to reduce the dimensionality of a sample of data points in a high dimensional space Rd. This paper introduces a dimensionality reduction method where the embedding coordinates are the eigenvectors of a positive semi-definite kernel obtained as the sol…
Detects data drift in deep learning models using neural embeddings.
problem Detecting changes in data distribution in deep learning models.
method Formulates drift detection in a sequential decision framework and introduces a loss function to balance false alarms and quick detection.
result Demonstrates improved ability to balance false alarms and quick detection in change detection.
Validates network bootstraps for uncertainty quantification in network visualisation.
problem Quantifying uncertainty in network embeddings when only a single observation is available.
method Statistical indistinguishable embeddings using k-nearest neighbour smoothing, validated by an exchangeable network test.
result Proposes a principled, distribution-free network bootstrap that passes the exchangeable network test.
We give a fast oblivious L2-embedding of A∈Rnxd to B∈Rrxd satisfying (1−ε)∥Ax∥22≤∥Bx∥22<=(1+ε)∥Ax∥22. Our embedding dimension r equals d, a constant independent of the distortion ε. We use as a black-box any L2-embedding $Π…