Layer-wise Relevance Propagation (LRP) and saliency maps have been recently used to explain the predictions of Deep Learning models, specifically in the domain of text classification. Given different attribution-based explanations to highlight relevant words for a predicted class label, experiments based on word deleti…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “word relevance”
Estimating the difficulty level of math word problems is an important task for many educational applications. Identification of relevant and irrelevant sentences in math word problems is an important step for calculating the difficulty levels of such problems. This paper addresses a novel application of text categoriza…
Bayesian algorithm improves word representations using semantic taxonomy.
problem Improving word representations in semantic taxonomy.
method Bayesian Hierarchical Words Representation (BHWR) learning algorithm combining Variational Bayes and semantic taxonomy modeling.
result BHWR produces better representations for rare words.
Text documents can be described by a number of abstract concepts such as semantic category, writing style, or sentiment. Machine learning (ML) models have been trained to automatically map documents to these abstract concepts, allowing to annotate very large text collections, more than could be processed by a human in …
Improved TF-IDF for word relevance in health-care social media documents.
problem Determining word relevance in informal documents.
method Semantic Sensitive TF-IDF (STF-IDF) method.
result Decreased TF-IDF mean error rate by 50% to 13.7%.
Recently, several methods have been proposed to explain the predictions of recurrent neural networks (RNNs), in particular of LSTMs. The goal of these methods is to understand the network's decisions by assigning to each input variable, e.g., a word, a relevance indicating to which extent it contributed to a particular…
Model analyzes cooccurrence data for recommender systems and item relevance.
problem High-dimensional cooccurrence data from online platforms.
method Shared parameter Alternating Tweedie (SA-Tweedie) model with Fisher scoring and learning rate adjustment.
result SA-Tweedie model outperforms other methods in optimizing parameters.
POLAR framework interprets word embeddings using polar opposites.
problem Lack of interpretability in pre-trained word embeddings.
method Adopt semantic differentials and polar opposites to transform embeddings.
result Interpretable word embeddings maintain performance comparable to original embeddings.
Layer-wise relevance propagation (LRP) is a recently proposed technique for explaining predictions of complex non-linear classifiers in terms of input variables. In this paper, we apply LRP for the first time to natural language processing (NLP). More precisely, we use it to explain the predictions of a convolutional n…
Finding Social Media Trolls: Dynamic Keyword Selection Methods for Rapidly-Evolving Online Debatescs.LG
Online harassment is a significant social problem. Prevention of online harassment requires rapid detection of harassing, offensive, and negative social media posts. In this paper, we propose the use of word embedding models to identify offensive and harassing social media messages in two aspects: detecting fast-changi…
Monitoring the biomedical literature for cases of Adverse Drug Reactions (ADRs) is a critically important and time consuming task in pharmacovigilance. The development of computer assisted approaches to aid this process in different forms has been the subject of many recent works. One particular area that has shown pro…
Click-through rate (CTR) is a key signal of relevance for search engine results, both organic and sponsored. CTR of a result has two core components: (a) the probability of examination of a result by a user, and (b) the perceived relevance of the result given that it has been examined by the user. There has been consid…
The paper develops a decision support system for hierarchical text classification of conference proceedings.
problem Classifying documents with a fixed hierarchical structure of topics.
method Developed a weighted hierarchical similarity function to calculate topic relevance, using entropy of words to estimate weights.
result The weighted hierarchical similarity function improves ranking accuracy compared to other methods.
Supervised topic models can help clinical researchers find interpretable cooccurence patterns in count data that are relevant for diagnostics. However, standard formulations of supervised Latent Dirichlet Allocation have two problems. First, when documents have many more words than labels, the influence of the labels w…
Finding relevant information from large document collections such as the World Wide Web is a common task in our daily lives. Estimation of a user's interest or search intention is necessary to recommend and retrieve relevant information from these collections. We introduce a brain-information interface used for recomme…
Text corpora are widely used resources for measuring societal biases and stereotypes. The common approach to measuring such biases using a corpus is by calculating the similarities between the embedding vector of a word (like nurse) and the vectors of the representative words of the concepts of interest (such as gender…
While ubiquitous, textual sources of information such as company reports, social media posts, etc. are hardly included in prediction algorithms for time series, despite the relevant information they may contain. In this work, openly accessible daily weather reports from France and the United-Kingdom are leveraged to pr…
WEEND uses a neural network to recognize speech and assign speakers to words.
problem End-to-end neural diarization without additional ASR and orchestration.
method Multi-task learning with an auxiliary network for ASR and speaker diarization.
result WEEND outperforms turn-based diarization and can handle 5-minute audio.
Recently, a technique called Layer-wise Relevance Propagation (LRP) was shown to deliver insightful explanations in the form of input space relevances for understanding feed-forward neural network classification decisions. In the present work, we extend the usage of LRP to recurrent neural networks. We propose a specif…
We propose a parsimonious topic model for text corpora. In related models such as Latent Dirichlet Allocation (LDA), all words are modeled topic-specifically, even though many words occur with similar frequencies across different topics. Our modeling determines salient words for each topic, which have topic-specific pr…
LLMs learn new tasks from unstructured data, but it depends on word co-occurrence and positional information.
problem Understanding how LLMs can learn new tasks from unstructured data without explicit training.
method Examined the capabilities of LLMs trained on unstructured data, focusing on sequence model requirements and training data structure.
result Many ICL capabilities can emerge from word co-occurrence in unstructured data, but positional information is crucial for certain tasks.
News embeddings improve volatility forecasts.
problem Improving volatility forecasting accuracy.
method Transformed news text into embeddings, evaluated standalone and combined with benchmarks.
result News contains useful predictive information, especially for stock-related content.
The paper models and predicts co-occurrence counts using Gamma regression.
problem Predicting relevance between items or users from high-dimensional sparse co-occurrence count data.
method Shared parameter alternating zero-inflated Gamma regression models (SA-ZIG) with Fisher scoring and learning rate adjustment.
result SA-ZIG with learning rate adjustment performs satisfactorily in predicting relevance.
In this paper we present a model for unsupervised topic discovery in texts corpora. The proposed model uses documents, words, and topics lookup table embedding as neural network model parameters to build probabilities of words given topics, and probabilities of topics given documents. These probabilities are used to re…
Exclusive Topic Modelingstat.ML
ETM identifies field-specific keywords in text classification.
problem Unsupervised text classification with field-specific keywords.
method Weighted Lasso penalty and pairwise Kullback-Leibler divergence penalty for topic separation.
result ETM improves topic coherence by 22% and 10% compared to LDA.
Sommelier recommends machine learning algorithms for datasets based on scholarly knowledge.
problem Identifying the best machine learning algorithms for a dataset.
method Word embedding representations of scholarly knowledge to recommend algorithms.
result Sommelier's recommendations achieve on average 97.7% of optimal accuracy.
We study the restless bandit associated with an extremely simple scalar Kalman filter model in discrete time. Under certain assumptions, we prove that the problem is indexable in the sense that the Whittle index is a non-decreasing function of the relevant belief state. In spite of the long history of this problem, thi…
A bag-of-concepts model improves relation extraction in a narrow knowledge domain with limited datacs.LG
This paper focuses on a traditional relation extraction task in the context of limited annotated data and a narrow knowledge domain. We explore this task with a clinical corpus consisting of 200 breast cancer follow-up treatment letters in which 16 distinct types of relations are annotated. We experiment with an approa…
In latent Dirichlet allocation (LDA), topics are multinomial distributions over the entire vocabulary. However, the vocabulary usually contains many words that are not relevant in forming the topics. We adopt a variable selection method widely used in statistical modeling as a dimension reduction tool and combine it wi…
This paper presents our system details and results of participation in the RDoC Tasks of BioNLP-OST 2019. Research Domain Criteria (RDoC) construct is a multi-dimensional and broad framework to describe mental health disorders by combining knowledge from genomics to behaviour. Non-availability of RDoC labelled dataset …
Word meaning changes over time, depending on linguistic and extra-linguistic factors. Associating a word's correct meaning in its historical context is a central challenge in diachronic research, and is relevant to a range of NLP tasks, including information retrieval and semantic search in historical texts. Bayesian m…
Exponential Family Attentionstat.ML
EFA extends self-attention to handle mixed data types and dynamic relevance.
problem Handling high-dimensional, mixed data types with dynamic relevance.
method Probabilistic generative model using self-attention and latent factor model.
result EFA consistently outperforms existing models in complex latent structure capture and reconstruction.
Motivation: Prediction of the interaction affinity between proteins and compounds is a major challenge in the drug discovery process. WideDTA is a deep-learning based prediction model that employs chemical and biological textual sequence information to predict binding affinity. Results: WideDTA uses four text-based inf…
We propose an adversarial learning approach for generating multi-turn dialogue responses. Our proposed framework, hredGAN, is based on conditional generative adversarial networks (GANs). The GAN's generator is a modified hierarchical recurrent encoder-decoder network (HRED) and the discriminator is a word-level bidirec…
In this paper we address the problem of discovering a small set of frequent serial episodes from sequential data so as to adequately characterize or summarize the data. We discuss an algorithm based on the Minimum Description Length (MDL) principle and the algorithm is a slight modification of an earlier method, called…
Providing explanations along with predictions is crucial in some text processing tasks. Therefore, we propose a new self-interpretable model that performs output prediction and simultaneously provides an explanation in terms of the presence of particular concepts in the input. To do so, our model's prediction relies so…
Topological Data Analysis in Text Classification: Extracting Features with Additive Informationcs.LG
Paper explores applying TDA to text classification, improving model performance.
problem Applying TDA to text classification is challenging due to the complexity of text geometry.
method Used word embeddings and TF-IDF vectors to extract topological features from text.
result Topological features improve classification results, especially in ensemble models.
Search-oriented conversational systems rely on information needs expressed in natural language (NL). We focus here on the understanding of NL expressions for building keyword-based queries. We propose a reinforcement-learning-driven translation model framework able to 1) learn the translation from NL expressions to que…
IITK wins FinSim 2020 task on financial hypernym detection.
problem Classifying financial terms into hypernym concepts in an external ontology.
method Used context-dependent and context-independent word embeddings (Word2vec and BERT) for classification.
result Ranked 1st based on mean rank and accuracy metrics.
New method considers subjectivity in text analysis using 'Room Theory'.
problem Detecting emotions in text considering subjective context.
method Framework Theory, Word2Vec, similarity measure between words.
result Measures relative relevance of emotions for a document.
Company2Vec creates embeddings from company websites for fine-grained business analytics.
problem Lack of fine-grained company labels for analytics.
method Word2Vec and dimensionality reduction on company website data.
result Semantic company embeddings for various applications.
The paper introduces natural α-embeddings for item representations.
problem Overcoming computational limitations in item encoding.
method Interpreting item embeddings in an Information Geometric framework using α-geometry of exponential families.
result A family of natural α-embeddings in the tangent space of the probability simplex.
We prove that the mirror map is trivial for the canonical formal families of Calabi-Yau varieties constructed by Gross and the second author. In other words, the natural coordinate in a canonical Calabi-Yau family is a canonical coordinate in the sense of Hodge theory. This implies that the higher weight periods direct…
Extreme multi-label text classification (XMTC) addresses the problem of tagging each text with the most relevant labels from an extreme-scale label set. Traditional methods use bag-of-words (BOW) representations without context information as their features. The state-ot-the-art deep learning-based method, AttentionXML…
Separable Non-negative Matrix Factorization (SNMF) is an important method for topic modeling, where "separable" assumes every topic contains at least one anchor word, defined as a word that has non-zero probability only on that topic. SNMF focuses on the word co-occurrence patterns to reveal topics by two steps: anchor…
Most existing word embedding approaches do not distinguish the same words in different contexts, therefore ignoring their contextual meanings. As a result, the learned embeddings of these words are usually a mixture of multiple meanings. In this paper, we acknowledge multiple identities of the same word in different co…
New algorithm selects relevant variables in high-dimensional graphical models.
problem Automatic selection of relevant variables in high-dimensional graphical models.
method Extends Chow and Liu's algorithm using mutual information and entropy coefficient of determination.
result Outperforms existing methods in selecting variables with explanatory power.
There are certain families of words and word sequences (words in the generators of a two-generator group) that arise frequently in the Teichm{ü}ller theory of hyperbolic three-manifolds and Kleinian and Fuchsian groups and in the discreteness problem for two generator matrix groups. We survey some of the families of su…