Proposes QQE for transforming and embedding data distributions.
problem Transforming and embedding data distributions for better representation or visualization.
method Quantile-Quantile Embedding (QQE) using quantile-quantile plot concept.
result QQE allows for better discrimination of classes in some cases.
This paper proposes a new method for embedding sequences using Wasserstein distances.
problem Embedding sequences in a metric space for better pattern recognition.
method Develops a deep learning model that embeds sequences as distributions and uses Wasserstein distances for comparison.
result Distributional embeddings using Wasserstein distances outperform traditional vector embeddings.
Unified framework for word embedding models using noise examples.
problem Improving word embedding models with negative sampling.
method Formulated a Word-Context Classification (WCC) framework that generalizes SkipGram word embedding models.
result The best noise distribution is the data distribution, improving both performance and training speed.
Transformers encode latent distributions in text, improving performance in out-of-distribution cases.
problem What should embeddings from language models represent?
method Connecting autoregressive prediction to sufficient statistics, identifying three settings.
result Transformers encode latent generating distributions, improving performance.
This note optimizes distributions using kernel mean embeddings with a new parameterization.
problem Optimizing distributions using kernel mean embeddings is challenging due to the difficulty of characterizing probability distribution vectors.
method Proposes a new parameterization of positive functions using kernel sums-of-squares to fit distributions in the MMD geometry.
result Distributions with kernel sum-of-squares densities are dense in the MMD geometry, allowing optimization in the finite-sample setting.
A new method estimates multi-dimensional value distributions using Hilbert space embeddings.
problem Estimating value distributions in complex, multi-dimensional reinforcement learning settings.
method Hilbert space mappings and kernel mean embeddings to estimate the kernel mean embedding of multi-dimensional value distributions.
result Uniform convergence guarantees and robust off-policy evaluation demonstrated in simulations.
New algorithm for RL using mean embeddings of return distributions.
problem Improving reinforcement learning algorithms for dynamic programming.
method Mean embeddings of return distributions, novel algorithms for RL.
result Asymptotic convergence and improved performance in deep RL.
This letter introduces an abstract learning problem called the "set embedding": The objective is to map sets into probability distributions so as to lose less information. We relate set union and intersection operations with corresponding interpolations of probability distributions. We also demonstrate a preliminary so…
A new method embeds distributions in a common space for optimal transport comparison.
problem Comparing distributions in different metric spaces.
method Sub-embedding robust Wasserstein (SERW) distance.
result SERW mimics GW distance properties and provides a cost relation.
Explains SNE, t-SNE, and their variants for manifold learning.
problem Dimensionality reduction and manifold learning.
method Probabilistic approach using Gaussian and Student-t distributions.
result Out-of-sample extension and acceleration methods for t-SNE.
Euclidean embeddings of data are fundamentally limited in their ability to capture latent semantic structures, which need not conform to Euclidean spatial assumptions. Here we consider an alternative, which embeds data as discrete probability distributions in a Wasserstein space, endowed with an optimal transport metri…
This paper presents a distance-based discriminative framework for learning with probability distributions. Instead of using kernel mean embeddings or generalized radial basis kernels, we introduce embeddings based on dissimilarity of distributions to some reference distributions denoted as templates. Our framework exte…
The problem of unsupervised learning node embeddings in graphs is one of the important directions in modern network science. In this work we propose a novel framework, which is aimed to find embeddings by \textit{discriminating distributions of similarities (DDoS)} between nodes in the graph. The general idea is implem…
A new method speeds up SoftMax normalization for embedding learning.
problem Efficiently learning distributed representations with SoftMax normalization.
method Proposes a linear-time heuristic approximation for mSoftMax(XYT), optimizing cross entropy. result Achieves higher or comparable accuracy to existing methods with lower computational time.
Smooth distributions on subcartesian spaces can be globally finitely generated.
problem Understanding smooth distributions on subcartesian spaces.
method Embedding in Euclidean space, Whitney Embedding Theorem, and distribution theory.
result Smooth generalized distributions and subbundles on connected subcartesian spaces are globally finitely generated.
Generative Distribution Embeddings learn multiscale representations of distributions.
problem Learning representations of entire distributions for multiscale reasoning.
method Introducing GDE framework that lifts autoencoders to the space of distributions, using conditional generative models and distributional invariance.
result GDEs learn predictive sufficient statistics embedded in Wasserstein space, recovering distances and trajectories for Gaussian and Gaussian mixture distributions.
Improved learning of probabilistic box embeddings by modeling parameters with Gumbel distributions.
problem Local identifiability issues in geometric embeddings.
method Modeling box parameters with min and max Gumbel distributions, calculating expected intersection volume.
result Improves the ability of probabilistic box embeddings to learn.
Paper proposes a framework to detect distribution shifts using embedding space geometry.
problem Detecting distribution shifts in candidate datasets to improve model generalizability.
method Non-parametric framework using embedding space geometry for two tests: robustness boundary and in-distribution/out-of-distribution classification.
result Both tests successfully detect distribution shifts in various scenarios for both synthetic and real-world datasets.
Paper develops a unified framework for measuring differences between conditional distributions.
problem Comparing conditional distributions in a unified and theoretically sound manner.
method Kernel embeddings and conditional maximum mean discrepancy (CMMD) framework.
result Established a coherent framework for measuring divergence between conditional distributions.
We propose to formulate multi-label learning as a estimation of class distribution in a non-linear embedding space, where for each label, its positive data embeddings and negative data embeddings distribute compactly to form a positive component and negative component respectively, while the positive component and nega…
The paper addresses data uncertainty in graph embedding by modeling data points as Gaussian distributions.
problem Data uncertainty in machine learning pipelines leads to misleading embeddings and lower accuracy.
method The paper proposes modeling data uncertainty using Gaussian distributions and reformulates graph embedding techniques.
result The proposed methods improve the accuracy of graph embedding by accounting for data uncertainty.
New test for conditional independence using kernel embeddings.
problem Testing conditional independence in high-dimensional settings.
method Analytic kernel embeddings, asymptotic distribution.
result New test outperforms existing methods in high-dimensional settings.
Kernel embeddings help estimate causal effects from observational data.
problem Estimating causal effects from observational data with confounding variables.
method Kernel embeddings in reproducing kernel Hilbert spaces (RKHS).
result Robust nonparametric framework for causal inference.
Embedding complex objects as vectors in low dimensional spaces is a longstanding problem in machine learning. We propose in this work an extension of that approach, which consists in embedding objects as elliptical probability distributions, namely distributions whose densities have elliptical level sets. We endow thes…
MCE reduces embedding instability in nonlinear dimensionality reduction.
problem Embedding instability caused by random initialization.
method Median of multiple embeddings (MCE) based on large deviation theory.
result MCE achieves consistency at an exponential rate and effectively mitigates instability.
Kernel embeddings separate distinct probability distributions, simplifying testing.
problem Testing equality of non-atomic probability distributions.
method Kernel covariance embeddings and Gaussian measures in reproducing kernel Hilbert spaces.
result Testing for singularity between Gaussian measures is equivalent to testing for equality of non-atomic probability distributions.
A Hilbert space embedding of a distribution---in short, a kernel mean embedding---has recently emerged as a powerful tool for machine learning and inference. The basic idea behind this framework is to map distributions into a reproducing kernel Hilbert space (RKHS) in which the whole arsenal of kernel methods can be ex…
We unify subsampling methods for network embeddings and prove their asymptotic distribution.
problem Understanding and improving the performance of network embeddings learned via subsampling.
method Unified framework for node2vec-like methods, proving asymptotic distribution under exchangeable graph assumption.
result Asymptotic distribution of learned embedding vectors decouples and provides rates of convergence.
Paper proposes a new method to learn distribution kernels via entropy maximization.
problem Challenges in applying kernel methods to distribution regression tasks.
method Proposes a novel objective for unsupervised learning of data-dependent distribution kernels based on entropy maximization.
result Demonstrates the effectiveness of the learned kernel across different modalities.
The paper characterizes Pfaffian embeddings from 2,3,5-manifolds to 7-dimensional isotropic spaces.
problem Characterizing Pfaffian embeddings from (2,3,5)-into flat (4,7)-geometries.
method Analyzing Pfaffian embeddings with specific geometric constraints.
result A generic (2,3,5)-manifold does not embed, with the first obstruction being a double root in the Cartan quartic.
Study horizontal discs in fat distributions, proving their existence.
problem Existence of embedded horizontal discs in fat distributions.
method Analyzing nonlinear PDEs and proving local invertibility.
result Existence of germs of embedded horizontal discs.
New framework uses geometry of embeddings to predict robustness.
problem Monitoring robustness in models without OOD labels.
method Constructs graphs from embeddings, measures spectral complexity and curvature.
result Representation geometry predicts robustness reliably.
Trains word embeddings from music and text data to link music contexts.
problem Varying vocabulary size and musical relevance in word embeddings.
method Combines general text and music-specific data to train word embeddings.
result Trained embeddings better associate music contexts with compositions.
A nonparametric approach for policy learning for POMDPs is proposed. The approach represents distributions over the states, observations, and actions as embeddings in feature spaces, which are reproducing kernel Hilbert spaces. Distributions over states given the observations are obtained by applying the kernel Bayes' …
Supervised deep-embedding methods project inputs of a domain to a representational space in which same-class instances lie near one another and different-class instances lie far apart. We propose a probabilistic method that treats embeddings as random variables. Extending a state-of-the-art deterministic method, Protot…
asp2vec learns dynamic node aspect distributions for better network embedding.
problem Lack of multi-aspect node representations in network embedding.
method Dynamic aspect assignment via Gumbel-Softmax and aspect regularization.
result Improved network embedding quality through dynamic aspect modeling.
DEN learns diverse tasks to generalize to unseen tasks.
problem Generalization from a diverse set of classification tasks with limited data.
method Three-block architecture: covariate transformation, distribution embedding, and classification.
result DEN outperforms existing methods in various synthetic and real tasks.
Improved learning theory for kernel distribution regression with two-stage sampling.
problem Distribution regression problem and two-stage sampling setting.
method Kernel methods, near-unbiased condition, new error bounds, convergence rates.
result Strictly improved convergence rates for three important classes of kernels.
We address the problem of unsupervised domain adaptation (UDA) by learning a cross-domain agnostic embedding space, where the distance between the probability distributions of the two source and target visual domains is minimized. We use the output space of a shared cross-domain deep encoder to model the embedding spac…
We propose a novel Wasserstein method with a distillation mechanism, yielding joint learning of word embeddings and topics. The proposed method is based on the fact that the Euclidean distance between word embeddings may be employed as the underlying distance in the Wasserstein topic model. The word distributions of to…
We demonstrate an equivalence between reproducing kernel Hilbert space (RKHS) embeddings of conditional distributions and vector-valued regressors. This connection introduces a natural regularized loss function which the RKHS embeddings minimise, providing an intuitive understanding of the embeddings and a justificatio…
Word embeddings provide point representations of words containing useful semantic information. We introduce multimodal word distributions formed from Gaussian mixtures, for multiple word meanings, entailment, and rich uncertainty information. To learn these distributions, we propose an energy-based max-margin objective…
CPME embeds counterfactual outcomes in RKHS for flexible policy evaluation.
problem Estimating counterfactual policy outcomes for decision-making.
method Counterfactual Policy Mean Embedding (CPME) framework in RKHS, plug-in and doubly robust estimators, kernel test statistic.
result Doubly robust estimator improves convergence rates and asymptotic normality.
ELM improves neural model embeddings for long-tail learning.
problem Learning skewed label distributions in neural models.
method Enforces margins in logit space and regularizes embedding distribution.
result ELM reduces generalization gap and tightens tail class embeddings.
Understanding how species are distributed across landscapes over time is a fundamental question in biodiversity research. Unfortunately, most species distribution models only target a single species at a time, despite strong ecological evidence that species are not independently distributed. We propose Deep Multi-Speci…
Many applications today, such as NLP, network analysis, and code analysis, rely on semantically embedding objects into low-dimensional fixed-length vectors. Such embeddings naturally provide a way to perform useful downstream tasks, such as identifying relations among objects or predicting objects for a given context, …
This work lists and describes the main recent strategies for building fixed-length, dense and distributed representations for words, based on the distributional hypothesis. These representations are now commonly called word embeddings and, in addition to encoding surprisingly good syntactic and semantic information, ha…
Personalized activity recognition improves performance for diverse users.
problem Poor performance of impersonal algorithms for individual users.
method Personalized activity recognition using deep embeddings from a fully convolutional neural network with triplet loss.
result Novel subject triplet loss provides the best performance overall.