Paper debiases text embeddings using context injection.
problem Text embeddings capture and perpetuate biases.
method Review 19 embedding models, quantify biases, and test context injection.
result Higher performing models are more prone to biases but better at incorporating context.
Word embeddings are a powerful approach for capturing semantic similarity among terms in a vocabulary. In this paper, we develop exponential family embeddings, a class of methods that extends the idea of word embeddings to other types of high-dimensional data. As examples, we studied neural data with real-valued observ…
Estimator Vectors learns OOV word embeddings using subword and context clues.
problem Lack of OOV word representations in neural network models.
method Jointly learns word, subword, and context clue representations.
result Strong estimates for OOV words via combined subword and context clue embeddings.
Low dimensional embeddings that capture the main variations of interest in collections of data are important for many applications. One way to construct these embeddings is to acquire estimates of similarity from the crowd. However, similarity is a multi-dimensional concept that varies from individual to individual. Ex…
With a simple architecture and the ability to learn meaningful word embeddings efficiently from texts containing billions of words, word2vec remains one of the most popular neural language models used today. However, as only a single embedding is learned for every word in the vocabulary, the model fails to optimally re…
CAEL-MIPS learns embeddings to improve MIPS for better OPE in contextual bandits.
problem High variance in IPS weighting for OPE in large action spaces.
method Context-Action Embedding Learning (CAEL) for MIPS to minimize MSE.
result CAEL-MIPS outperforms baselines in MSE for OPE in contextual bandits.
Trains word embeddings from music and text data to link music contexts.
problem Varying vocabulary size and musical relevance in word embeddings.
method Combines general text and music-specific data to train word embeddings.
result Trained embeddings better associate music contexts with compositions.
Unsupervised model learns word and context embeddings from character sequences.
problem Learning meaningful word and context embeddings from unlabeled data.
method Character-aware neural architecture that jointly learns word and context embeddings.
result Compact encoders achieve high performance in downstream tasks.
Normal forms and isotropic embeddings via Euler-like vector fields.
problem Proving normal forms results for geometric structures.
method Construction of Euler-like vector fields compatible with geometric structures.
result Illustrated in various examples, including Morse-Bott, Weinstein, and Zung's theorems.
IITK wins FinSim 2020 task on financial hypernym detection.
problem Classifying financial terms into hypernym concepts in an external ontology.
method Used context-dependent and context-independent word embeddings (Word2vec and BERT) for classification.
result Ranked 1st based on mean rank and accuracy metrics.
Traditional semantic similarity models often fail to encapsulate the external context in which texts are situated. However, textual datasets generated on mobile platforms can help us build a truer representation of semantic similarity by introducing multimodal data. This is especially important in sparse datasets, maki…
Word2Vec embedding for Italian language developed.
problem Lack of word embedding for Italian language.
method Parameter tuning of Word2Vec for Italian language.
result An embedding for Italian language created.
DCE learns customer embeddings from digital activity and financial context.
problem Comprehensive customer understanding in financial services.
method Leverages customers' digital activity and financial context to learn dense representations.
result DCE showed performance lift in three prediction problems.
We break down transformer embeddings into interpretable components revealing hidden geometric structures.
problem Understanding the hidden geometry and interpretability of transformer models.
method Decomposed transformer embeddings into position, context, and residual components.
result Pervasive mathematical structure in transformer embeddings, including position and context vectors.
This paper improves continuous adversarial training for LLMs using in-context learning theory.
problem Efficiently defending large language models (LLMs) against jailbreak attacks.
method The paper presents a theoretical analysis of continuous adversarial training (CAT) for LLMs based on in-context learning (ICL) theory, proving a robust generalization bound and proposing an improved regularization term.
result The robust generalization bound explains why CAT can defend against jailbreak prompts and shows that LLM robustness is related to embedding matrix singular values.
We propose a principle for exploring context in machine learning models. Starting with a simple assumption that each observation may or may not depend on its context, a conditional probability distribution is decomposed into two parts: context-free and context-sensitive. Then by employing the log-linear word production…
CADE learns dual node representations for better generalization.
problem Transductive graph embeddings cannot generalize to unseen nodes or across different graphs.
method CADE combines real-time neighborhoods with neighbor-attentioned representation, preserving known node memory.
result CADE outperforms state-of-the-art methods in generalization and context-awareness.
This paper takes a step towards theoretical analysis of the relationship between word embeddings and context embeddings in models such as word2vec. We start from basic probabilistic assumptions on the nature of word vectors, context vectors, and text generation. These assumptions are well supported either empirically o…
Most existing word embedding approaches do not distinguish the same words in different contexts, therefore ignoring their contextual meanings. As a result, the learned embeddings of these words are usually a mixture of multiple meanings. In this paper, we acknowledge multiple identities of the same word in different co…
A new model learns demand patterns from data, reducing complexity and improving accuracy.
problem Forecasting short-term demand from spatiotemporal data with complex patterns.
method Temporal-Guided Network (TGNet) using graph networks and temporal-guided embedding.
result TGNet achieves competitive performance with fewer parameters compared to state-of-the-art models.
Trip recommendation is an important location-based service that helps relieve users from the time and efforts for trip planning. It aims to recommend a sequence of places of interest (POIs) for a user to visit that maximizes the user's satisfaction. When adding a POI to a recommended trip, it is essential to understand…
MCPCA analyzes shared factors across multiple data contexts.
problem No tools to recover shared factors across multiple contexts.
method Developed a theoretical and algorithmic framework (MCPCA).
result Reveals shared axes of variation across subsets of contexts.
We present a probabilistic language model for time-stamped text data which tracks the semantic evolution of individual words over time. The model represents words and contexts by latent trajectories in an embedding space. At each moment in time, the embedding vectors are inferred from a probabilistic version of word2ve…
We present a framework for building unsupervised representations of entities and their compositions, where each entity is viewed as a probability distribution rather than a vector embedding. In particular, this distribution is supported over the contexts which co-occur with the entity and are embedded in a suitable low…
LaT-PFN model predicts time series with zero-shot capability.
problem Zero-shot time series forecasting.
method In-context latent space learning with JEPA and PFN integration.
result Superior zero-shot predictions compared to baselines.
A model for POI recommendation using relation embedding.
problem Challenges in POI recommendation due to sparse user-POI matrix and varying context.
method Translation-based relation embedding using Knowledge Graph Embedding techniques, combined matrix factorization framework.
result Demonstrates effectiveness of the proposed model on real-world datasets.
This research improves graph embeddings by optimizing node sampling with centrality weights.
problem Improving the accuracy and efficiency of graph embeddings using Skip-Gram methods.
method Implemented and analyzed four graph embedding techniques with different centrality-weighted sampling distributions.
result Centrality-weighted sampling leads to improved accuracy and faster learning times.
New method interprets deep embeddings for diabetes patient clustering.
problem Interpreting deep embeddings for disease progression.
method Patient clustering approach using deep embeddings.
result Clinically meaningful insights into diabetes progression patterns.
asp2vec learns dynamic node aspect distributions for better network embedding.
problem Lack of multi-aspect node representations in network embedding.
method Dynamic aspect assignment via Gumbel-Softmax and aspect regularization.
result Improved network embedding quality through dynamic aspect modeling.
Word embedding maps words into a low-dimensional continuous embedding space by exploiting the local word collocation patterns in a small context window. On the other hand, topic modeling maps documents onto a low-dimensional topic space, by utilizing the global word collocation patterns in the same document. These two …
In automatic speech recognition (ASR) what a user says depends on the particular context she is in. Typically, this context is represented as a set of word n-grams. In this work, we present a novel, all-neural, end-to-end (E2E) ASR sys- tem that utilizes such context. Our approach, which we re- fer to as Contextual Lis…
Integrates contextual constraints into embedding models for better recommendation quality.
problem Contextual constraints lead to incomplete or low-quality recommendations when applied independently.
method Merges constraint application and retrieval into one operation in the embedding space.
result Significant improvements in predictive performance compared to context-aware and standard models.
Persona2vec learns multiple node roles in graphs.
problem Graphs often have nodes with multiple overlapping roles.
method Persona2vec learns multiple node representations based on structural contexts.
result Persona2vec outperforms state-of-the-art models in link prediction.
Models use embeddings and attention for better claim severity prediction.
problem Improving predictive modeling of claim severity with categorical data.
method Developed neural networks and attention-based architectures with embeddings.
result Attention-based models enhance predictive performance with contextual augmentation.
Fruit fly brain network learns word embeddings using sparse binary codes.
problem Learning semantic word representations from text.
method Inspired by mushroom body neural network, sparse binary hash codes.
result Fruit fly network achieves comparable NLP performance with reduced resources.
We present an approach to learning multi-sense word embeddings relying both on monolingual and bilingual information. Our model consists of an encoder, which uses monolingual and bilingual context (i.e. a parallel sentence) to choose a sense for a given word, and a decoder which predicts context words based on the chos…
GSNE improves house price predictions by embedding geo-spatial context.
problem Lack of contextual information in house price prediction models.
method Geo-Spatial Network Embedding (GSNE) using graph neural networks.
result GSNE embeddings consistently improve house price prediction performance.
This paper assesses biases in contextualized word representations.
problem Analyzing biases in contextualized word representations.
method Proposes assessing bias at the contextual word level, capturing contextual effects of bias.
result Demonstrates evidence of bias in contextual word models, including racial bias and exacerbated effects for intersectional minorities.
Study embedding calculus and link invariants using functor calculus.
problem Detect Milnor invariants using embedding towers of string links.
method Use functor calculus and Goodwillie-Weiss embedding calculus.
result Embedding tower detects Milnor invariants.
Paper proposes a new topology for AML analysis using Poincaré embeddings.
problem Complex money laundering schemes and regulatory constraints hinder AML analysis and information sharing.
method Proposes a new topology for AML analysis using Poincaré embeddings.
result Demonstrates improved AML analysis and information sharing through Poincaré embeddings.
The paper introduces a new method for graph embedding using exponential family distributions.
problem Representing networks in a low dimensional latent space for various applications.
method Introduces the exponential family graph embedding model, generalizing random walk-based techniques to exponential family conditional distributions.
result The proposed techniques outperform existing methods in link prediction and node classification tasks.
In this paper, we present our method of using fixed-size ordinally forgetting encoding (FOFE) to solve the word sense disambiguation (WSD) problem. FOFE enables us to encode variable-length sequence of words into a theoretically unique fixed-size representation that can be fed into a feed forward neural network (FFNN),…
Ordinal embedding methods estimate perceptual scales from relative judgments.
problem Measuring subjective sensation using relative judgments.
method Ordinal embedding from machine learning applied to method of triads.
result Ordinal embedding allows estimating perceptual scales from few judgments, non-monotonous functions, and multi-dimensional scales.
Paper proves embedding theorem for conformally compact manifolds.
problem Embedding conformally compact manifolds into hyperbolic spaces.
method Proves analogous Nash Embedding Theorem for conformally compact manifolds.
result Conformally compact manifolds can be isometrically embedded into hyperbolic spaces.
Unified framework for word embedding models using noise examples.
problem Improving word embedding models with negative sampling.
method Formulated a Word-Context Classification (WCC) framework that generalizes SkipGram word embedding models.
result The best noise distribution is the data distribution, improving both performance and training speed.
Improved object detection for scientific document images.
problem Current object detectors fail to accurately localize regions in scientific document images.
method Revised R-CNN model with region embedding for fine-grained proposals.
result 17% mAP improvement over standard object detection models.
A new method combines multiple node embeddings using tensor decomposition.
problem Generating accurate node embeddings for complex networks.
method TenSemble2Vec: combines multiple embeddings via tensor decomposition.
result Improves node embeddings by leveraging complementary information from different methods.
We find necessary and sufficient conditions for existence of a locally isometric embedding of a vacuum space-time into a conformally-flat 5-space. We explicitly construct such embeddings for any spherically symmetric Lorentzian metric in 3+1 dimensions as a hypersurface in R4,1. For the Schwarzschild metric the…