Locally-contextual CRFs improve sequence labeling performance.
problem Improving sequence labeling with contextual embeddings.
method Locally-contextual nonlinear CRFs using deep neural networks.
result Consistently outperforms linear chain CRF and previous state of the art.
Predict stock movement with news headlines using BERT embeddings.
problem Predicting stock price movement after financial news.
method Fine-Tuned Contextualized-Embedding Recurrent Neural Network (FT-CE-RNN) using BERT.
result Obtains state-of-the-art results on stock movement prediction task.
This paper assesses biases in contextualized word representations.
problem Analyzing biases in contextualized word representations.
method Proposes assessing bias at the contextual word level, capturing contextual effects of bias.
result Demonstrates evidence of bias in contextual word models, including racial bias and exacerbated effects for intersectional minorities.
CAEL-MIPS learns embeddings to improve MIPS for better OPE in contextual bandits.
problem High variance in IPS weighting for OPE in large action spaces.
method Context-Action Embedding Learning (CAEL) for MIPS to minimize MSE.
result CAEL-MIPS outperforms baselines in MSE for OPE in contextual bandits.
Faster, context-aware topic modeling approach unveiled.
problem Slow and heavy preprocessing in topic modeling.
method Conceptualizes topics as independent axes, decomposes embeddings using ICA.
result 4.5x faster than BERTopic on average, with diverse and coherent topics.
SLiCE learns contextual node embeddings for link prediction in heterogeneous networks.
problem Link prediction requires specific contextual information not captured by static node embeddings.
method Self-supervised pre-training with localized attention mechanisms.
result SLiCE significantly outperforms existing methods on link prediction tasks.
SCNode improves node embeddings for GNNs in both homophilic and heterophilic graphs.
problem Challenges in node representation quality and generalization in GNNs, especially in heterophilic graphs.
method SCNode integrates spatial and contextual information to create more discriminative and structurally aware node embeddings.
result SCNode achieves superior performance over conventional GNN models on benchmark datasets.
Study examines biases in clinical word embeddings, revealing performance gaps across groups.
problem Biases in clinical word embeddings leading to performance differences across groups.
method Pretrained BERT models on MIMIC-III, fill-in-the-blank method, fairness evaluation on clinical tasks.
result Classifiers trained from BERT representations exhibit statistically significant differences in performance across groups.
Integrates contextual constraints into embedding models for better recommendation quality.
problem Contextual constraints lead to incomplete or low-quality recommendations when applied independently.
method Merges constraint application and retrieval into one operation in the embedding space.
result Significant improvements in predictive performance compared to context-aware and standard models.
Method learns audio embeddings with contextualized tags.
problem Align audio and tags for cross-modal tasks.
method Audio autoencoder, word embeddings, multi-head self-attention, contrastive loss.
result Multi-head self-attention improves audio representations.
New algorithm tackles dynamic query routing to multiple embedding models.
problem Dynamic query routing to multiple embedding models under adversarial conditions.
method Formalized as adversarial contextual linear bandit with low-rank experts, proposed HPG algorithm.
result HPG algorithm achieves linearized policy regret of i l d e O ( s M T ) ilde{\mathcal O}(s\sqrt{M T}) i l d e O ( s M T ) . BERT embeddings improve sequence quality metrics.
problem Measuring the quality of generated sequences against references.
method Employ contextual BERT embeddings for sequence-level reward.
result Contextual embeddings provide a more effective learning signal.
New approach combines semi-supervised learning and bandits for better predictions.
problem Online semi-supervised learning with bandit feedback for applications like clinical trials and ad recommendations.
method Adjusted Graph Convolutional Network (GCN) for contextual bandits, with semi-supervised missing rewards imputation.
result Developed multi-GCN embedded contextual bandit algorithms verified on real-world datasets.
Import2vec creates embeddings for software libraries to improve learning tasks.
problem Developing semantic representations for software libraries.
method Applied word embedding techniques from NLP to library packages.
result Library vectors capture meaningful relationships among libraries.
This paper explores the complexity of learning representations in contextual linear bandits.
problem Understanding the complexity of representation learning in contextual linear bandits.
method Systematic approach to representation learning in contextual linear bandits, focusing on instance-dependent perspective.
result Representation learning is fundamentally more complex than linear bandits, with some cases being arbitrarily harder.
IITK wins FinSim 2020 task on financial hypernym detection.
problem Classifying financial terms into hypernym concepts in an external ontology.
method Used context-dependent and context-independent word embeddings (Word2vec and BERT) for classification.
result Ranked 1st based on mean rank and accuracy metrics.
A simple modification enables a universal NMT model with language-specific parameters.
problem Creating a universal NMT model that can adapt to different languages and domains.
method Introducing a contextual parameter generator (CPG) that dynamically adjusts model parameters based on source and target language embeddings.
result The system achieves state-of-the-art performance and zero-shot translation, demonstrating the effectiveness of the CPG.
BERTopic improves financial text analysis with FinTextSim's contextual embeddings.
problem Analyzing financial text data for insights and predictions.
method Integrates BERTopic with FinTextSim for topic modeling and clustering.
result BERTopic performs better with FinTextSim's embeddings, improving topic clarity and reducing misclassification.
This paper introduces a new task to better understand Transformers in quantitative contexts.
problem Understanding Transformers in high-stakes quantitative and scientific applications.
method Introduces a novel contextual counting task and analyzes it with causal and non-causal Transformer architectures.
result Causal attention is better suited for the contextual counting task, and no positional embeddings lead to the best accuracy.
Novel approach to contextual bandits using self-supervised learning.
problem Exploiting rich data representations in computer vision without explicit labels.
method Combining contextual bandit objective with self-supervision objective.
result Substantial gains in cumulative reward on computer vision datasets.
Models use embeddings and attention for better claim severity prediction.
problem Improving predictive modeling of claim severity with categorical data.
method Developed neural networks and attention-based architectures with embeddings.
result Attention-based models enhance predictive performance with contextual augmentation.
MEANTIME improves sequential recommendation by using multi-temporal embeddings and attention mechanisms.
problem Limited use of timestamp information and information bottleneck in sequential recommendation models.
method MEANTIME employs multiple types of temporal embeddings and attention mechanisms to capture diverse patterns from user behavior sequences.
result MEANTIME outperforms state-of-the-art sequential recommendation methods.
In automatic speech recognition (ASR) what a user says depends on the particular context she is in. Typically, this context is represented as a set of word n-grams. In this work, we present a novel, all-neural, end-to-end (E2E) ASR sys- tem that utilizes such context. Our approach, which we re- fer to as Contextual Lis…
This study examines gender bias in Dutch newspapers from 1950-1990 using word embeddings.
problem Examining gender bias in historical newspapers.
method Word embeddings to measure bias changes over time.
result Clear differences in gender bias and changes within newspapers over time.
New OPE estimator improves offline policy evaluation for large action spaces.
problem Existing OPE estimators fail with large action spaces, leading to extreme bias and variance.
method Proposes a new estimator using marginalized importance weights and action embeddings.
result Empirical performance improvement enables reliable OPE even with many actions.
LaT-PFN model predicts time series with zero-shot capability.
problem Zero-shot time series forecasting.
method In-context latent space learning with JEPA and PFN integration.
result Superior zero-shot predictions compared to baselines.
A new linear contextual bandit algorithm with improved regret bound.
problem Efficiently solving linear contextual bandit problems with reduced regret.
method Proposes a novel estimator embedded with exploration and a self-normalized bound.
result Regret bound matches lower bound of Ω ( d T ) Ω(\sqrt{dT}) Ω ( d T ) up to logarithmic factors. The AI2 Reasoning Challenge (ARC), a new benchmark dataset for question answering (QA) has been recently released. ARC only contains natural science questions authored for human exams, which are hard to answer and require advanced logic reasoning. On the ARC Challenge Set, existing state-of-the-art QA systems fail to s…
SARD improves deep learning clinical prediction performance.
problem Deep learning models struggle to match linear models in healthcare predictions.
method Reverse Distillation to initialize deep models, combined with contextual and temporal embeddings.
result SARD outperforms state-of-the-art methods on clinical prediction outcomes.
Proposes a joint POI embedding model for better trip recommendations.
problem Trip recommendation needs to consider multiple contextual factors for better user satisfaction.
method Jointly learns the impact of POI popularity, co-occurring POIs, and user preferences.
result Proposed algorithms outperform state-of-the-art in trip recommendation quality.
Contextualizing financial news improves stock price predictions.
problem Predicting stock prices from financial news requires understanding historical context.
method Proposed a method using a large language model for main articles and a small model for historical context.
result Historical context significantly improves model performance across methods and time horizons.
Proposes a new method for conversational agents using deep learning.
problem Building coherent and non-monotonous conversational agents with proper discourse and coverage.
method End-to-end multi-stream deep learning architecture leveraging contextual and syntactic information.
result Significantly improved next sentence prediction task.
Improves topic modeling using LLM embeddings and Poisson process.
problem Traditional topic modeling ignores word context; LLMs offer better contextual embeddings.
method Convert documents to word embeddings, model as Poisson process, estimate topics using flexible algorithm.
result Method integrates LLMs without fine-tuning, offers advantages over traditional methods.
Word embedding models offer continuous vector representations that can capture rich contextual semantics based on their word co-occurrence patterns. While these word vectors can provide very effective features used in many NLP tasks such as clustering similar words and inferring learning relationships, many challenges …
Proposes CGA model for logical queries over KGs.
problem Handling logical queries over incomplete KGs with unequal query path contributions.
method Multi-head graph attention with initial neighborhood aggregation for center node prediction.
result CGA model outperforms baselines on DB18, WikiGeo19, and Bio datasets.
Proposes MDR estimator for unbiased OPE with large action spaces.
problem Severe bias and variance tradeoffs in OPE with large action spaces.
method Marginalized Doubly Robust (MDR) estimator, reducing variance and bias.
result MDR estimator is unbiased under weaker assumptions than MIPS.
Most existing word embedding approaches do not distinguish the same words in different contexts, therefore ignoring their contextual meanings. As a result, the learned embeddings of these words are usually a mixture of multiple meanings. In this paper, we acknowledge multiple identities of the same word in different co…
AEALT uses autoencoders to reduce text embedding dimensions for improved efficiency.
problem High dimensionality of text embeddings hinders downstream tasks.
method Factor-augmented supervised learning with autoencoders.
result AEALT outperforms conventional deep-learning approaches.
Conventional text classification models make a bag-of-words assumption reducing text into word occurrence counts per document. Recent algorithms such as word2vec are capable of learning semantic meaning and similarity between words in an entirely unsupervised manner using a contextual window and doing so much faster th…
MBExplainer provides explanations for models combining graph embeddings and tabular features.
problem Explaining models using a mix of graph embeddings and tabular features.
method Model-agnostic approach using Shapley values and Monte Carlo Tree Search.
result MBExplainer efficiently finds human-readable explanations for model predictions.
Neural model improves text segmentation accuracy.
problem Manual feature engineering and large memory requirements in text segmentation.
method Attention-based bidirectional LSTM model with CNN sentence embeddings and contextual prediction.
result Improves WinDiff score by ~7% on three benchmark datasets.
Kernel-based algorithm optimizes cellular network configuration through multi-task learning.
problem Optimizing network configuration based on field experience and minimizing exploration cost.
method Kernel-based multi-BS contextual bandit algorithm leveraging conditional kernel embedding for multi-task learning.
result The proposed algorithm reduces exploration cost and improves network performance.
GICDM corrects hubness in embedding spaces for better generative model evaluation.
problem Hubness phenomenon distorts distances in high-dimensional embedding spaces.
method Generative ICDM (GICDM) using multi-scale extension to correct neighborhood estimation.
result GICDM resolves hubness-induced failures and improves metric behavior.
Paper introduces G-LowTESTR for efficient tensor bandits.
problem Efficient decision-making in multi-dimensional data with non-linear reward functions.
method Generalized low-rank tensor contextual bandits model and G-LowTESTR algorithm.
result G-LowTESTR achieves superior regret bound compared to vectorization and matricization methods.
Contextual bandit learning is an increasingly popular approach to optimizing recommender systems via user feedback, but can be slow to converge in practice due to the need for exploring a large feature space. In this paper, we propose a coarse-to-fine hierarchical approach for encoding prior knowledge that drastically …
Paper tackles Arabic question similarity, outperforming state-of-the-art.
problem Detecting semantically similar questions in Arabic is challenging.
method Utilizes contextualized word representations (ELMo embeddings) trained on MSA and dialectic sentences, combined with a pairwise similarity layer.
result Achieves 93% F1-score on Modern Standard Arabic benchmark and 82% on dialectical benchmark.
Model predicts political ideology using context vectors to mitigate bias and scarcity.
problem Scarcity and selection bias in political ideology prediction.
method Proposes a statistical model decomposing embeddings into context and position vectors, training an end-to-end model for deployment.
result Model can predict ideological labels even with minimal biased data, outperforming state-of-the-art methods.
Contextualized ML learns context-dependent effects using deep learning.
problem Learning heterogeneous and context-dependent effects in data.
method Applying deep learning to the meta-relationship between contextual information and context-specific parametric models.
result Unified framework for cluster analysis and cohort modeling.