Topic models are probabilistic models for discovering topical themes in collections of documents. In real world applications, these models provide us with the means of organizing what would otherwise be unstructured collections. They can help us cluster a huge collection into different topics or find a subset of the co…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Topic modeling”
Document clustering and topic modeling are two closely related tasks which can mutually benefit each other. Topic modeling can project documents into a topic space which facilitates effective document clustering. Cluster labels discovered by document clustering can be incorporated into topic models to extract local top…
A new model CDTM improves text classification by concentrating document topics.
problem Unsupervised text classification with diverse topic distributions.
method Imposes an exponential entropy penalty on document topic distribution to encourage concentration.
result More coherent topics and concentrated, sparse document-topic distributions.
Recently, considerable research effort has been devoted to developing deep architectures for topic models to learn topic structures. Although several deep models have been proposed to learn better topic proportions of documents, how to leverage the benefits of deep structures for learning word distributions of topics h…
We introduce Gaussian Process Topic Models (GPTMs), a new family of topic models which can leverage a kernel among documents while extracting correlated topics. GPTMs can be considered a systematic generalization of the Correlated Topic Models (CTMs) using ideas from Gaussian Process (GP) based embedding. Since GPTMs w…
New Gamma-Poisson model improves topic selection for short text.
problem Topic modelling for short text using Poisson distribution.
method Gamma-Poisson mixture model with collapsed Gibbs sampler.
result Gamma-Poisson model selects more accurate number of topics.
We propose a parsimonious topic model for text corpora. In related models such as Latent Dirichlet Allocation (LDA), all words are modeled topic-specifically, even though many words occur with similar frequencies across different topics. Our modeling determines salient words for each topic, which have topic-specific pr…
Topic modeling analyzes documents to learn meaningful patterns of words. However, existing topic models fail to learn interpretable topics when working with large and heavy-tailed vocabularies. To this end, we develop the Embedded Topic Model (ETM), a generative model of documents that marries traditional topic models …
Scientific documents rely on both mathematics and text to communicate ideas. Inspired by the topical correspondence between mathematical equations and word contexts observed in scientific texts, we propose a novel topic model that jointly generates mathematical equations and their surrounding text (TopicEq). Using an e…
The top word list, i.e., the top-M words with highest marginal probability in a given topic, is the standard topic representation in topic models. Most of recent automatical topic labeling algorithms and popular topic quality metrics are based on it. However, we find, empirically, words in this type of top word list ar…
Improved neural topic model for semi-supervised learning.
problem Representing textual data in an interpretable manner with limited labeled data.
method Label-Indexed Neural Topic Model (LI-NTM) that combines deep generative models with semi-supervised learning.
result LI-NTM outperforms existing models in document reconstruction and classifier performance.
We propose a novel model for a topic-aware chatbot by combining the traditional Recurrent Neural Network (RNN) encoder-decoder model with a topic attention layer based on Nonnegative Matrix Factorization (NMF). After learning topic vectors from an auxiliary text corpus via NMF, the decoder is trained so that it is more…
We introduce the author-topic model, a generative model for documents that extends Latent Dirichlet Allocation (LDA; Blei, Ng, & Jordan, 2003) to include authorship information. Each author is associated with a multinomial distribution over topics and each topic is associated with a multinomial distribution over words.…
AOBTM adapts online topic modeling for short app reviews, revealing coherent topics over time.
problem Challenges in inferring latent topics from short, dynamic app reviews over multiple versions.
method Adaptive Online Biterm Topic Model (AOBTM) that addresses sparsity and considers statistical data from previous versions.
result AOBTM finds more coherent topics and outperforms state-of-the-art baselines.
A new neural topic model using optimal transport improves document representation and topic coherence.
problem Challenges in achieving good document representation and coherent/diverse topics in existing NTMs.
method Proposes a neural topic model via optimal transport, learning topic distribution by minimising OT distance to document word distributions.
result Significantly outperforms state-of-the-art NTMs on discovering coherent and diverse topics.
This research proposes a new (old) metric for evaluating goodness of fit in topic models, the coefficient of determination, or . Within the context of topic modeling, has the same interpretation that it does when used in a broader class of statistical models. Reporting with topic models addresses two c…
E-LDA: Toward Interpretable LDA Topic Models with Strong Guarantees in Logarithmic Parallel Timecs.LG
E-LDA offers faster, interpretable LDA topic models.
problem Inferring topics in LDA topic models with strong guarantees.
method Non-gradient combinatorial approach for faster convergence.
result Logarithmic parallel computation time and interpretability.
CFTM uses fractional Brownian motion for dynamic topic modeling.
problem Identifying long-term dependency or roughness in topic and word distributions over time.
method Continuous Time Fractional Topic Model (cFTM) incorporating fractional Brownian motion.
result cFTM captures long-term dependency or roughness in topic and word distributions.
Faster, context-aware topic modeling approach unveiled.
problem Slow and heavy preprocessing in topic modeling.
method Conceptualizes topics as independent axes, decomposes embeddings using ICA.
result 4.5x faster than BERTopic on average, with diverse and coherent topics.
Supervised topic models are often sought to balance prediction quality and interpretability. However, when models are (inevitably) misspecified, standard approaches rarely deliver on both. We introduce a novel approach, the prediction-focused topic model, that uses the supervisory signal to retain only vocabulary terms…
Topic models are typically evaluated with respect to the global topic distributions that they generate, using metrics such as coherence, but without regard to local (token-level) topic assignments. Token-level assignments are important for downstream tasks such as classification. Even recent models, which aim to improv…
New NMF algorithms improve topic model fits.
problem Improving topic model fits for large datasets.
method Leveraging recent NMF optimization methods to fit topic models efficiently.
result Better topic model fits and faster computation times.
Paper proposes a Renyi entropy-based method for tuning hierarchical topic models.
problem Tuning hierarchical topic models, especially determining the number of topics at each level, is challenging.
method The paper introduces a Renyi entropy-based metric for quality assessment and a practical tuning concept.
result The proposed method can estimate the number of topics for two hierarchical levels in hARTM model.
This paper improves topic modeling by embedding words and topics together.
problem Topic models struggle with short documents and approximate inference.
method Model each document as a mixture of word embeddings and each topic as a mixture of topic embeddings.
result The method optimizes topic embeddings to minimize semantic differences between words and topics.
Social media conversations unfold based on complex interactions between users, topics and time. While recent models have been proposed to capture network strengths between users, users' topical preferences and temporal patterns between posting and response times, interaction patterns between topics has not been studied…
In this paper we present a model for unsupervised topic discovery in texts corpora. The proposed model uses documents, words, and topics lookup table embedding as neural network model parameters to build probabilities of words given topics, and probabilities of topics given documents. These probabilities are used to re…
Certain type of documents such as tweets are collected by specifying a set of keywords. As topics of interest change with time it is beneficial to adjust keywords dynamically. The challenge is that these need to be specified ahead of knowing the forthcoming documents and the underlying topics. The future topics should …
Correlated topic modeling has been limited to small model and problem sizes due to their high computational cost and poor scaling. In this paper, we propose a new model which learns compact topic embeddings and captures topic correlations through the closeness between the topic vectors. Our method enables efficient inf…
Geometric method captures rare topics and temporal alignment in co-author networks.
problem Missing rare topics and smooth temporal alignment in topic modeling.
method Integrates multimodal text and co-author network data using Hellinger distances and Ward's linkage.
result Effective identification of rare topics and visualization of topic drift over time.
nnLDA combines neural and probabilistic methods for better topic modeling with side information.
problem Lack of integration of auxiliary information in traditional topic models.
method nnLDA integrates side information through a neural prior mechanism, optimizing both neural and probabilistic components.
result nnLDA outperforms traditional models in topic coherence, perplexity, and classification.
Neural model predicts survival outcomes and reveals feature relationships.
problem Predicting time-to-event outcomes and understanding feature relationships in clinical data.
method Survival and topic modeling combined in a neural network framework.
result Neural survival-supervised topic models achieve competitive accuracy with interpretability.
A single, stationary topic model such as latent Dirichlet allocation is inappropriate for modeling corpora that span long time periods, as the popularity of topics is likely to change over time. A number of models that incorporate time have been proposed, but in general they either exhibit limited forms of temporal var…
In this technical report, we present jLDADMM---an easy-to-use Java toolkit for conventional topic models. jLDADMM is released to provide alternatives for topic modeling on normal or short texts. It provides implementations of the Latent Dirichlet Allocation topic model and the one-topic-per-document Dirichlet Multinomi…
Enhances topic-metadata relationship modeling using Bayesian methods.
problem Estimating relationships between latent topics and metadata in topic modeling.
method Proposes modifications to the method of composition, using Beta regression and a fully Bayesian approach.
result Improves quantification of uncertainty in topic-metadata relationships.
We present a new topic model that generates documents by sampling a topic for one whole sentence at a time, and generating the words in the sentence using an RNN decoder that is conditioned on the topic of the sentence. We argue that this novel formalism will help us not only visualize and model the topical discourse s…
We develop new models and algorithms for learning the temporal dynamics of the topic polytopes and related geometric objects that arise in topic model based inference. Our model is nonparametric Bayesian and the corresponding inference algorithm is able to discover new topics as the time progresses. By exploiting the c…
Modeling lead-lag relationship between two text corpora for improved topic modeling.
problem Recognizing the relationship between multiple text corpora for better topic modeling.
method Proposed a jointly dynamic topic model and embedding extension for large-scale text corpus.
result The proposed model can well recognize the lead-lag relationship between two text corpora and improve topic learning.
Paper presents a multi-label topic model for financial texts with high performance and insights into market reactions.
problem Analyzing financial text data for market reactions and understanding topic interactions.
method Trained a multi-label topic model on a financial text database, achieved high macro F1 score, and investigated topic interactions.
result Model achieves high performance (macro F1 > 85%) and reveals significant market reactions to topic co-occurrences.
TopicResponse: A Marriage of Topic Modelling and Rasch Modelling for Automatic Measurement in MOOCscs.LG
This paper explores the suitability of using automatically discovered topics from MOOC discussion forums for modelling students' academic abilities. The Rasch model from psychometrics is a popular generative probabilistic model that relates latent student skill, latent item difficulty, and observed student-item respons…
We develop the multilingual topic model for unaligned text (MuTo), a probabilistic model of text that is designed to analyze corpora composed of documents in two languages. From these documents, MuTo uses stochastic EM to simultaneously discover both a matching between the languages and multilingual latent topics. We d…
Electronic Health Record (EHR) data can be represented as discrete counts over a high dimensional set of possible procedures, diagnoses, and medications. Supervised topic models present an attractive option for incorporating EHR data as features into a prediction problem: given a patient's record, we estimate a set of …
Topic models have proven to be a useful tool for discovering latent structures in document collections. However, most document collections often come as temporal streams and thus several aspects of the latent structure such as the number of topics, the topics' distribution and popularity are time-evolving. Several mode…
Traditional Relational Topic Models provide a way to discover the hidden topics from a document network. Many theoretical and practical tasks, such as dimensional reduction, document clustering, link prediction, benefit from this revealed knowledge. However, existing relational topic models are based on an assumption t…
The logistic normal distribution has recently been adapted via the transformation of multivariate Gaus- sian variables to model the topical distribution of documents in the presence of correlations among topics. In this paper, we propose a probit normal alternative approach to modelling correlated topical structures. O…
Overcomplete latent representations have been very popular for unsupervised feature learning in recent years. In this paper, we specify which overcomplete models can be identified given observable moments of a certain order. We consider probabilistic admixture or topic models in the overcomplete regime, where the numbe…
GraphSTONE uses topic models to capture graph structures, improving GCN performance.
problem GCNs focus too much on node features and not enough on graph structures.
method GraphSTONE employs topic models of graphs to capture structural topics, which guide the aggregation of node features.
result GraphSTONE outperforms GCNs in performance, efficiency, and interpretability.
This paper proposes the continuous semantic topic embedding model (CSTEM) which finds latent topic variables in documents using continuous semantic distance function between the topics and the words by means of the variational autoencoder(VAE). The semantic distance could be represented by any symmetric bell-shaped geo…
We propose new algorithms for topic modeling when the number of topics is unknown. Our approach relies on an analysis of the concentration of mass and angular geometry of the topic simplex, a convex polytope constructed by taking the convex hull of vertices representing the latent topics. Our algorithms are shown in pr…