Proposes QQE for transforming and embedding data distributions.
problem Transforming and embedding data distributions for better representation or visualization.
method Quantile-Quantile Embedding (QQE) using quantile-quantile plot concept.
result QQE allows for better discrimination of classes in some cases.
This paper proposes a new method for embedding sequences using Wasserstein distances.
problem Embedding sequences in a metric space for better pattern recognition.
method Develops a deep learning model that embeds sequences as distributions and uses Wasserstein distances for comparison.
result Distributional embeddings using Wasserstein distances outperform traditional vector embeddings.
Unified framework for word embedding models using noise examples.
problem Improving word embedding models with negative sampling.
method Formulated a Word-Context Classification (WCC) framework that generalizes SkipGram word embedding models.
result The best noise distribution is the data distribution, improving both performance and training speed.
Transformers encode latent distributions in text, improving performance in out-of-distribution cases.
problem What should embeddings from language models represent?
method Connecting autoregressive prediction to sufficient statistics, identifying three settings.
result Transformers encode latent generating distributions, improving performance.
Maps sets to probability distributions to minimize information loss.
problem Learning to map sets to probability distributions to preserve information.
method Relates set operations to probability distribution interpolations and demonstrates a preliminary solution.
result Experimental results show the effectiveness of the set embedding approach.
A new method for graph node embeddings by discriminating similarity distributions.
problem Unsupervised learning of node embeddings in graphs.
method Maximizing the earth mover distance between distributions of similarities of similar and dissimilar nodes.
result Generates embeddings with state-of-the-art performance in link prediction.
This note optimizes distributions using kernel mean embeddings with a new parameterization.
problem Optimizing distributions using kernel mean embeddings is challenging due to the difficulty of characterizing probability distribution vectors.
method Proposes a new parameterization of positive functions using kernel sums-of-squares to fit distributions in the MMD geometry.
result Distributions with kernel sum-of-squares densities are dense in the MMD geometry, allowing optimization in the finite-sample setting.
This research embeds data as discrete probability distributions in Wasserstein spaces, capturing semantic structures more effectively.
problem Limitations of Euclidean embeddings in capturing latent semantic structures.
method Learning embeddings into entropic Wasserstein spaces, capturing semantic information in Wasserstein distance.
result Wasserstein embeddings can embed a wider variety of metric structures with smaller distortion than Euclidean embeddings.
A new method estimates multi-dimensional value distributions using Hilbert space embeddings.
problem Estimating value distributions in complex, multi-dimensional reinforcement learning settings.
method Hilbert space mappings and kernel mean embeddings to estimate the kernel mean embedding of multi-dimensional value distributions.
result Uniform convergence guarantees and robust off-policy evaluation demonstrated in simulations.
Survey of word embedding techniques for NLP.
problem Creating effective word representations for natural language processing.
method Describes recent strategies for fixed-length, dense word embeddings.
result Word embeddings encode syntactic and semantic information and improve NLP tasks.
New algorithm for RL using mean embeddings of return distributions.
problem Improving reinforcement learning algorithms for dynamic programming.
method Mean embeddings of return distributions, novel algorithms for RL.
result Asymptotic convergence and improved performance in deep RL.
We embed objects as elliptical distributions using the Wasserstein metric.
problem Embedding complex objects as vectors in low dimensional spaces.
method Embedding objects as elliptical probability distributions with the 2-Wasserstein metric.
result Wasserstein elliptical embeddings provide more intuitive and numerically stable tools than Gaussian embeddings.
A new method embeds distributions in a common space for optimal transport comparison.
problem Comparing distributions in different metric spaces.
method Sub-embedding robust Wasserstein (SERW) distance.
result SERW mimics GW distance properties and provides a cost relation.
Explains SNE, t-SNE, and their variants for manifold learning.
problem Dimensionality reduction and manifold learning.
method Probabilistic approach using Gaussian and Student-t distributions.
result Out-of-sample extension and acceleration methods for t-SNE.
Paper introduces a new method for learning with distributions using dissimilarity measures.
problem Learning with probability distributions using dissimilarity measures.
method Introduces embeddings based on dissimilarity of distributions to templates, extending similarity theory to population distributions.
result Proves that dissimilarity theory holds for empirical distributions and shows better performance of Wasserstein distance embedding.
This paper tackles UDA by learning domain-invariant embeddings using distribution alignment and pseudo-labels.
problem Unsupervised domain adaptation between two visual domains.
method Shared deep encoder, Sliced-Wasserstein Distance, deep classifier, pseudo-labels for class alignment.
result Effective solution for training deep classification networks on source domain to generalize to target domain.
Prototypical Networks improve multi-label classification accuracy.
problem Multi-label classification with nonlinear label dependencies.
method Formulate multi-label learning as class distribution in a non-linear embedding space. For each label, positive and negative embeddings are compactly distributed. Labels are inferred by measuring the distance to prototype positive or negative embeddings.
result Extensive experiments show improved accuracy compared to state-of-the-art algorithms.
Paper explores vector embeddings, distributional hypothesis, and PIP loss for natural language processing.
problem Understanding the effect of dimensionality on vector embeddings and their functionality.
method Formulates a theoretical framework, proposes PIP loss, and reveals bias-variance trade-off.
result Discovers robustness and forward stability of vector embeddings, answers dimensionality selection problem.
A new method speeds up SoftMax normalization for embedding learning.
problem Efficiently learning distributed representations with SoftMax normalization.
method Proposes a linear-time heuristic approximation for mSoftMax(XYT), optimizing cross entropy. result Achieves higher or comparable accuracy to existing methods with lower computational time.
Generative Distribution Embeddings learn multiscale representations of distributions.
problem Learning representations of entire distributions for multiscale reasoning.
method Introducing GDE framework that lifts autoencoders to the space of distributions, using conditional generative models and distributional invariance.
result GDEs learn predictive sufficient statistics embedded in Wasserstein space, recovering distances and trajectories for Gaussian and Gaussian mixture distributions.
Smooth distributions on subcartesian spaces can be globally finitely generated.
problem Understanding smooth distributions on subcartesian spaces.
method Embedding in Euclidean space, Whitney Embedding Theorem, and distribution theory.
result Smooth generalized distributions and subbundles on connected subcartesian spaces are globally finitely generated.
Enhances ordinal embedding with less data by focusing on margin distribution.
problem Insufficient labeled data for ordinal embedding.
method Proposes Distributional Margin based Ordinal Embedding (DMOE) to improve generalization with less data.
result Demonstrates improved generalization performance with less labeled data.
New word distributions capture multiple meanings and outperform existing methods.
problem Capturing semantic information for words with multiple meanings.
method Gaussian mixtures with an energy-based max-margin objective.
result Multimodal word distributions outperform word2vec and Gaussian embeddings.
This paper presents a new framework for distributed training of embeddings using graph analytics.
problem Training embeddings is computationally intensive and challenging to distribute.
method Formulated Any2Vec training algorithm as a graph application, leveraging D-Galois for dynamic graph generation and repartitioning, and incorporating novel communication optimizations.
result GraphAny2Vec matches state-of-the-art shared-memory implementations in accuracy and provides significant speedups.
Improved learning of probabilistic box embeddings by modeling parameters with Gumbel distributions.
problem Local identifiability issues in geometric embeddings.
method Modeling box parameters with min and max Gumbel distributions, calculating expected intersection volume.
result Improves the ability of probabilistic box embeddings to learn.
Parallelizes graph embedding for large graphs.
problem Large graphs make existing graph embedding techniques inefficient.
method Distributed parallel computation framework using a cluster of compute nodes.
result Parallel computation scales well and maintains embedding quality.
Paper proposes a framework to detect distribution shifts using embedding space geometry.
problem Detecting distribution shifts in candidate datasets to improve model generalizability.
method Non-parametric framework using embedding space geometry for two tests: robustness boundary and in-distribution/out-of-distribution classification.
result Both tests successfully detect distribution shifts in various scenarios for both synthetic and real-world datasets.
Paper develops a unified framework for measuring differences between conditional distributions.
problem Comparing conditional distributions in a unified and theoretically sound manner.
method Kernel embeddings and conditional maximum mean discrepancy (CMMD) framework.
result Established a coherent framework for measuring divergence between conditional distributions.
The paper clarifies that embeddings can either reduce or maintain the dimensionality of sparse feature spaces.
problem Misconception about embedding dimensionality in sparse feature spaces.
method Analysis of information entropy and upper bounds for embedding dimensions.
result Embeddings can either reduce or maintain the dimensionality of sparse feature spaces, providing a meaningful representation.
Develops a rigorous theory for conditional mean embeddings.
problem Efficient conditioning of probability distributions in RKHSs.
method Mathematical theory for both centred and uncentred covariance operators.
result Significantly weakens conditions for applicability of CMEs.
The paper addresses data uncertainty in graph embedding by modeling data points as Gaussian distributions.
problem Data uncertainty in machine learning pipelines leads to misleading embeddings and lower accuracy.
method The paper proposes modeling data uncertainty using Gaussian distributions and reformulates graph embedding techniques.
result The proposed methods improve the accuracy of graph embedding by accounting for data uncertainty.
New test for conditional independence using kernel embeddings.
problem Testing conditional independence in high-dimensional settings.
method Analytic kernel embeddings, asymptotic distribution.
result New test outperforms existing methods in high-dimensional settings.
Kernel embeddings help estimate causal effects from observational data.
problem Estimating causal effects from observational data with confounding variables.
method Kernel embeddings in reproducing kernel Hilbert spaces (RKHS).
result Robust nonparametric framework for causal inference.
MCE reduces embedding instability in nonlinear dimensionality reduction.
problem Embedding instability caused by random initialization.
method Median of multiple embeddings (MCE) based on large deviation theory.
result MCE achieves consistency at an exponential rate and effectively mitigates instability.
Kernel embeddings separate distinct probability distributions, simplifying testing.
problem Testing equality of non-atomic probability distributions.
method Kernel covariance embeddings and Gaussian measures in reproducing kernel Hilbert spaces.
result Testing for singularity between Gaussian measures is equivalent to testing for equality of non-atomic probability distributions.
A Hilbert space embedding of a distribution---in short, a kernel mean embedding---has recently emerged as a powerful tool for machine learning and inference. The basic idea behind this framework is to map distributions into a reproducing kernel Hilbert space (RKHS) in which the whole arsenal of kernel methods can be ex…
A probabilistic method for deep embedding that improves classification accuracy and interpretability.
problem Improving classification accuracy and interpretability in deep learning.
method A probabilistic approach that treats embeddings as random variables, using a product distribution over labeled instances and marginalizing prototype proximity.
result Superior large- and open-set classification accuracy compared to state-of-the-art methods.
PyTorch-BigGraph scales graph embeddings to large graphs.
problem Large graphs with billions of nodes and trillions of edges.
method Graph partitioning, multi-relation embedding system, distributed training.
result Comparable performance on benchmarks, scalable to large graphs.
A novel method learns word embeddings and topics using Wasserstein distance.
problem Learning word embeddings and topics from text data.
method Distilled Wasserstein learning framework for joint word embedding and topic modeling.
result Superior performance on disease network construction, mortality prediction, and procedure recommendation.
This paper proposes an efficient method to train word embeddings for large corpora without synchronization.
problem Training word embeddings for large text corpora is computationally expensive and requires synchronization.
method Partition the input space instead of the vocabulary size, using asynchronous training without parameter synchronization.
result Comparable and up to 45% performance improvement in NLP benchmarks with 1/10 the training time.
We unify subsampling methods for network embeddings and prove their asymptotic distribution.
problem Understanding and improving the performance of network embeddings learned via subsampling.
method Unified framework for node2vec-like methods, proving asymptotic distribution under exchangeable graph assumption.
result Asymptotic distribution of learned embedding vectors decouples and provides rates of convergence.
A scalable graph embedding system for large networks.
problem Training graph embeddings on large datasets efficiently.
method Distributed infrastructure avoiding graph partitioning, dynamic graph creation, efficient indexing.
result System scales skip-gram algorithm to Friendster and internal networks.
Paper proposes a new method to learn distribution kernels via entropy maximization.
problem Challenges in applying kernel methods to distribution regression tasks.
method Proposes a novel objective for unsupervised learning of data-dependent distribution kernels based on entropy maximization.
result Demonstrates the effectiveness of the learned kernel across different modalities.
The paper characterizes Pfaffian embeddings from 2,3,5-manifolds to 7-dimensional isotropic spaces.
problem Characterizing Pfaffian embeddings from (2,3,5)-into flat (4,7)-geometries.
method Analyzing Pfaffian embeddings with specific geometric constraints.
result A generic (2,3,5)-manifold does not embed, with the first obstruction being a double root in the Cartan quartic.
New framework uses geometry of embeddings to predict robustness.
problem Monitoring robustness in models without OOD labels.
method Constructs graphs from embeddings, measures spectral complexity and curvature.
result Representation geometry predicts robustness reliably.
DMSE embeds multiple species and environmental covariates for better species distribution modeling.
problem Single species models ignore species interactions, DMSE addresses this.
method Deep neural network for joint embedding of multiple species and environmental covariates.
result DMSE outperforms single-species and multi-label models in predicting bird species distributions.
Study horizontal discs in fat distributions, proving their existence.
problem Existence of embedded horizontal discs in fat distributions.
method Analyzing nonlinear PDEs and proving local invertibility.
result Existence of germs of embedded horizontal discs.
Trains word embeddings from music and text data to link music contexts.
problem Varying vocabulary size and musical relevance in word embeddings.
method Combines general text and music-specific data to train word embeddings.
result Trained embeddings better associate music contexts with compositions.