Proposes QQE for transforming and embedding data distributions.
problem Transforming and embedding data distributions for better representation or visualization.
method Quantile-Quantile Embedding (QQE) using quantile-quantile plot concept.
result QQE allows for better discrimination of classes in some cases.
New method integrates topological knowledge into data embeddings.
problem Lack of general tools to incorporate prior topological knowledge into embeddings.
method Introduces new topological losses to topologically regularize data embeddings.
result Natural representation of simple models like clusters and flares.
Embeddings leak sensitive information about input data, which can be recovered or inferred.
problem Information leakage in embedding models.
method Developed three classes of attacks to study information leakage.
result Embeddings leak sensitive information about input data, which can be partially recovered or inferred.
Phase2vec learns embeddings of dynamical systems without supervision.
problem Predicting the classes of 2D dynamical systems from data.
method Physics-informed convolutional network that extracts geometric features and minimizes reconstruction loss.
result Learned embeddings respect the semantics of physical systems and outperform blackbox classifiers.
Models use embeddings and attention for better claim severity prediction.
problem Improving predictive modeling of claim severity with categorical data.
method Developed neural networks and attention-based architectures with embeddings.
result Attention-based models enhance predictive performance with contextual augmentation.
This paper finds a linear relationship between t-SNE perplexity and data set size.
problem Choosing the right perplexity for t-SNE embeddings.
method Analyzed the relationship between perplexity and data set size.
result Embeddings remain structurally consistent when perplexity is adjusted accordingly.
A novel method extracts topological features from word embeddings for text classification.
problem High dimensional and noisy text representations in natural language processing.
method Persistent homology for topological data analysis on word embeddings.
result Topological features outperform conventional text mining features on long textual documents.
The study shows how to accurately estimate embedding vectors in high dimensions.
problem How to accurately estimate embedding vectors in high-dimensional spaces.
method A simple probability model and a variant of low-rank approximate message passing (AMP) method.
result The AMP approach enables precise predictions of the accuracy of the estimation in certain high-dimensional limits.
Word embeddings are a powerful approach for capturing semantic similarity among terms in a vocabulary. In this paper, we develop exponential family embeddings, a class of methods that extends the idea of word embeddings to other types of high-dimensional data. As examples, we studied neural data with real-valued observ…
Flexible embedding framework for diverse data types.
problem Limited applicability of existing embedding learning methods.
method A flexible framework using entity-relation-matrices and sampling mechanism.
result Framework outperforms state-of-the-art approaches in various tasks.
Two methods factor out prior knowledge from low-dimensional embeddings.
problem Visualizing data without considering background knowledge.
method JEDI for tSNE and CONFETTI for any embedding.
result Embeddings reveal meaningful structure hidden by prior knowledge.
The paper explores theories behind graph and relational data vector embeddings.
problem Understanding the foundations of vector embeddings for graphs and relational structures.
method Proposes two theoretical approaches to understand vector embeddings.
result Draws connections between various embedding techniques and suggests future research directions.
Online shopping caters to the needs of millions of users daily. Search, recommendations, personalization have become essential building blocks for serving customer needs. Efficacy of such systems is dependent on a thorough understanding of products and their representation. Multiple information sources and data types p…
Representation learning has become an invaluable approach for learning from symbolic data such as text and graphs. However, while complex symbolic datasets often exhibit a latent hierarchical structure, state-of-the-art methods typically learn embeddings in Euclidean vector spaces, which do not account for this propert…
The paper establishes concentration bounds for embeddings of generative models.
problem Analyzing statistical properties of generative models.
method High probability concentration bounds on sample vector embeddings using Data Kernel Perspective Space.
result Determines the number of samples needed for accurate approximation of generative model embeddings.
Improves visualization of high-dimensional data by correcting misleading artifacts in neighbor embedding methods.
problem Misleading visual artifacts in t-SNE and UMAP due to lack of data-independent manifold learning interpretations.
method LOO-map framework that extends embedding maps to the entire input space, identifying and correcting map discontinuities.
result Developed point-wise diagnostic scores to detect unreliable embedding points and improve hyperparameter selection.
Optimizes embedding accuracy for data variance and error.
problem Efficiently embedding data while minimizing distortion.
method Uses Johnson-Lindenstrauss embeddings with orthogonal matrices and singular-value latent variables.
result Achieves best accuracy in variance, mean-squared error, and length distortion.
Trains word embeddings from music and text data to link music contexts.
problem Varying vocabulary size and musical relevance in word embeddings.
method Combines general text and music-specific data to train word embeddings.
result Trained embeddings better associate music contexts with compositions.
New tensors capture intrinsic embedding data of conformal hypersurfaces.
problem Classifying hypersurface invariants in conformal manifolds.
method Constructing curvatures and conformal fundamental forms.
result Finite family of tensors captures extrinsic embedding data.
Transformers encode latent distributions in text, improving performance in out-of-distribution cases.
problem What should embeddings from language models represent?
method Connecting autoregressive prediction to sufficient statistics, identifying three settings.
result Transformers encode latent generating distributions, improving performance.
Author2Vec generates user embeddings from social media data.
problem Generating useful user embeddings from noisy social media data.
method End-to-end neural network with BERT sentence representations and unsupervised pre-training.
result Author2Vec outperforms traditional methods in user classification tasks.
We solve the vector embedding problem by minimizing total distortion under constraints.
problem Assigning representative vectors to items with similarity and dissimilarity constraints.
method Projected quasi-Newton method for MDE problems, scalable to large data sets.
result Our method provides principled ways to validate embeddings and scales to millions of items.
MCE reduces embedding instability in nonlinear dimensionality reduction.
problem Embedding instability caused by random initialization.
method Median of multiple embeddings (MCE) based on large deviation theory.
result MCE achieves consistency at an exponential rate and effectively mitigates instability.
QENDy learns quadratic dynamics from nonlinear systems data.
problem Identifying governing equations of highly nonlinear dynamical systems.
method QENDy embeds nonlinear dynamics into a quadratic feature space, requiring trajectory data and preselected basis functions.
result QENDy accurately identifies quadratic dynamics and outperforms SINDy and deep learning methods.
Unified framework for hyperbolic embeddings from mixed data types.
problem Computing hyperbolic embeddings from noisy metric and non-metric data.
method Semidefinite programming and spectral factorization methods.
result Efficient computation of hyperbolic embeddings from arbitrary data.
Natural language processing has improved tremendously after the success of word embedding techniques such as word2vec. Recently, the same idea has been applied on source code with encouraging results. In this survey, we aim to collect and discuss the usage of word embedding techniques on programs and source code. The a…
Kernel embeddings help estimate causal effects from observational data.
problem Estimating causal effects from observational data with confounding variables.
method Kernel embeddings in reproducing kernel Hilbert spaces (RKHS).
result Robust nonparametric framework for causal inference.
The success of machine learning methods heavily relies on having an appropriate representation for data at hand. Traditionally, machine learning approaches relied on user-defined heuristics to extract features encoding structural information about data. However, recently there has been a surge in approaches that learn …
FREDE efficiently embeds graphs using linear space and guarantees quality.
problem Efficiently embedding graphs with quality guarantees and linear space complexity.
method FREDE combines matrix sketching with a nonlinear transform of PageRank similarities to achieve linear space and quality guarantees.
result FREDE provides column-covariance approximation guarantees that are nearly as good as SVD, even with limited node similarities.
Contrastive embeddings improve neural architecture search performance.
problem Improving performance of neural architecture search algorithms.
method Contrastive learning to identify networks based on data Jacobians and produce embeddings.
result Traditional black-box optimization algorithms can reach state-of-the-art performance with contrastive embeddings.
A framework converts spatial data into embeddings for insurance risk modelling.
problem Improving underwriting precision and risk management in insurance with spatial data.
method Multi-view contrastive learning framework for generating spatial embeddings.
result Spatial embeddings consistently improve predictive accuracy across various models.
The recent proliferation of publicly available graph-structured data has sparked an interest in machine learning algorithms for graph data. Since most traditional machine learning algorithms assume data to be tabular, embedding algorithms for mapping graph data to real-valued vector spaces has become an active area of …
Paper explores embedding methods for detecting pseudo-cliques in random graphs, showing limitations and potential.
problem Detecting planted pseudo-cliques in random dot product graphs.
method Adjacency Spectral Embedding (ASE) and Graph Encoder Embedding (GEE).
result These methods can localize pseudo-cliques with additional clean network data, but not without it.
A new method combines multiple node embeddings using tensor decomposition.
problem Generating accurate node embeddings for complex networks.
method TenSemble2Vec: combines multiple embeddings via tensor decomposition.
result Improves node embeddings by leveraging complementary information from different methods.
Embeddings are one of the fundamental building blocks for data analysis tasks. Embeddings are already essential tools for large language models and image analysis, and their use is being extended to many other research domains. The generation of these distributed representations is often a data- and computation-expensi…
The paper addresses data uncertainty in graph embedding by modeling data points as Gaussian distributions.
problem Data uncertainty in machine learning pipelines leads to misleading embeddings and lower accuracy.
method The paper proposes modeling data uncertainty using Gaussian distributions and reformulates graph embedding techniques.
result The proposed methods improve the accuracy of graph embedding by accounting for data uncertainty.
Survey of Locally Linear Embedding and its variants.
problem Representing high-dimensional data in a lower-dimensional space while preserving local structure.
method Explains various LLE and variant methods, including kernel LLE, inverse LLE, feature fusion, out-of-sample embedding, incremental LLE, landmark LLE, supervised LLE, robust LLE, fusion with other methods, and weighted LLE.
result Comprehensive overview of LLE and its variants.
This paper introduces an acceleration structure for hyperbolic embeddings.
problem Efficiently embedding and visualizing high-dimensional data in hyperbolic spaces.
method Building upon a polar quadtree, the paper introduces a new acceleration structure for hyperbolic embeddings.
result The new method computes embeddings in significantly less time compared to existing methods.
Metric learning methods for dimensionality reduction in combination with k-Nearest Neighbors (kNN) have been extensively deployed in many classification, data embedding, and information retrieval applications. However, most of these approaches involve pairwise training data comparisons, and thus have quadratic computat…
Graph embedding is an effective method to represent graph data in a low dimensional space for graph analytics. Most existing embedding algorithms typically focus on preserving the topological structure or minimizing the reconstruction errors of graph data, but they have mostly ignored the data distribution of the laten…
This paper tightens the generalization error bound for graph embedding in non-Euclidean spaces.
problem High generalization error in non-Euclidean graph embedding, preventing practical applications.
method Novel upper bound of graph embedding's generalization error using local Rademacher complexity.
result The new bound is tighter and faster, allowing better performance in non-Euclidean spaces.
Graph embedding provides an efficient solution for graph analysis by converting the graph into a low-dimensional space which preserves the structure information. In contrast to the graph structure data, the i.i.d. node embedding can be processed efficiently in terms of both time and space. Current semi-supervised graph…
Euclidean embeddings of data are fundamentally limited in their ability to capture latent semantic structures, which need not conform to Euclidean spatial assumptions. Here we consider an alternative, which embeds data as discrete probability distributions in a Wasserstein space, endowed with an optimal transport metri…
The Whitney embedding theorem gives an upper bound on the smallest embedding dimension of a manifold. If a data set lies on a manifold, a random projection into this reduced dimension will retain the manifold structure. Here we present an algorithm to find a projection that distorts the data as little as possible.
Robustly computes intrinsic coordinates on point clouds using resampling and averaging.
problem Computing intrinsic coordinates on noisy or outlier-prone point clouds.
method Subsample data, vary hyperparameters, cluster candidate embeddings, identify representative embeddings, and average them using Procrustes analysis.
result Robust to noise and outliers, validated on synthetic and real data.
Parametric UMAP learns a mapping from data to embeddings.
problem Representing and learning from structured data.
method Parametric optimization over neural network weights for UMAP.
result Parametric UMAP performs comparably to non-parametric UMAP with faster online embeddings.
This paper proves the manifold hypothesis for lower embedding dimensions using osculating hyperspheres.
problem The dataset lies on a low-dimensional submanifold in high-dimensional space.
method Constructing osculating hyperspheres and applying surgery theory to embed the hypersurface.
result The manifold hypothesis holds for embedding dimensionalities up to d−1. New method detects and analyzes correlation in multiple network data.
problem Detecting and analyzing correlation in multiple network data.
method Generalized omnibus embedding methodology.
result Induced correlation can significantly extend the reach of spectral inference procedures.