TopicEq model generates equations and text from scientific papers.
problem Communicating ideas in scientific texts using both mathematics and text.
method Joint topic and equation generation model using correlated topic model and RNN.
result Joint model outperforms existing topic and equation models for scientific texts.
SenGen generates sentences conditioned on topics, improving topic visualization.
problem Improve topic visualization and interpretability in documents.
method Variational auto-encoder with RNN decoder conditioned on topics.
result Preliminary experiments show promise but also challenges remain.
Generative topic embedding combines local and global patterns for document representation.
problem Representing documents in a continuous space using both local and global patterns.
method Proposes a variational inference model to generate topic embeddings and document representations.
result Performs better than existing methods in document classification tasks.
New Gamma-Poisson model improves topic selection for short text.
problem Topic modelling for short text using Poisson distribution.
method Gamma-Poisson mixture model with collapsed Gibbs sampler.
result Gamma-Poisson model selects more accurate number of topics.
Proposes a new multi-layer model for topic distributions.
problem Leveraging deep structures for learning word distributions of topics.
method A multi-layer generative process on word distributions of topics, where each topic is drawn from a mixture of topics from the layer above.
result Discover interpretable topic hierarchies and improve topic models' accuracy and interpretability.
Improved neural topic model for semi-supervised learning.
problem Representing textual data in an interpretable manner with limited labeled data.
method Label-Indexed Neural Topic Model (LI-NTM) that combines deep generative models with semi-supervised learning.
result LI-NTM outperforms existing models in document reconstruction and classifier performance.
We introduce Gaussian Process Topic Models (GPTMs), a new family of topic models which can leverage a kernel among documents while extracting correlated topics. GPTMs can be considered a systematic generalization of the Correlated Topic Models (CTMs) using ideas from Gaussian Process (GP) based embedding. Since GPTMs w…
ETM discovers interpretable topics in large vocabularies.
problem Existing topic models fail with large, heavy-tailed vocabularies.
method Generative model combining topic models and word embeddings with variational inference.
result ETM discovers interpretable topics even with large vocabularies.
New method learns more diverse topics from text documents considering paragraph structure.
problem Classic Topic Models ignore word position and use symmetric priors, limiting topic diversity.
method Exploits paragraph structure to distinguish between general and specific topics.
result Shows improved topic diversity and relevance in structured documents.
CorEx learns topics without assumptions, incorporating human input.
problem Complexity and detailed assumptions in topic modeling.
method Information-theoretic framework, anchor words for minimal human input.
result Topics comparable to LDA with minimal human intervention.
We introduce the author-topic model, a generative model for documents that extends Latent Dirichlet Allocation (LDA; Blei, Ng, & Jordan, 2003) to include authorship information. Each author is associated with a multinomial distribution over topics and each topic is associated with a multinomial distribution over words.…
This paper improves topic modeling by embedding words and topics together.
problem Topic models struggle with short documents and approximate inference.
method Model each document as a mixture of word embeddings and each topic as a mixture of topic embeddings.
result The method optimizes topic embeddings to minimize semantic differences between words and topics.
Proposes HMHP for joint modeling of user-topic interactions.
problem Complex interactions between users, topics and time on social media.
method Hidden Markov Hawkes Process (HMHP) incorporating topical Markov Chains.
result HMHP outperforms state-of-the-art models in generalization and accuracy.
New metric correlates local topic quality with human judgments.
problem Evaluation of topic models focuses on global metrics, ignoring token-level assignments.
method Proposed a human evaluation task and automated metrics to assess local topic quality.
result Consistency metric correlates best with human judgments of local topic quality.
Overcomplete latent representations have been very popular for unsupervised feature learning in recent years. In this paper, we specify which overcomplete models can be identified given observable moments of a certain order. We consider probabilistic admixture or topic models in the overcomplete regime, where the numbe…
LLMs encode latent topic distributions, suggesting Bayesian inference.
problem Capturing topic structure from large language models.
method Connecting LLM optimization to implicit Bayesian inference and de Finetti's theorem.
result LLMs recover latent topic distributions, matching LDA-generated topics.
A new model generates summaries by conditioning on input text and latent topics.
problem Improving abstractive summarization quality.
method Conditioning decoder output on both input text and latent topics identified by LDA.
result Strongly improved ROUGE scores on CNN/Daily Mail and WikiHow datasets.
Supervised topic models utilize document's side information for discovering predictive low dimensional representations of documents. Existing models apply the likelihood-based estimation. In this paper, we present a general framework of max-margin supervised topic models for both continuous and categorical response var…
A new method for topic detection using hierarchical latent tree models.
problem Hierarchical topic detection in document collections.
method Graphical models (HLTMs) with binary variables at different levels representing word co-occurrence patterns and document clusters.
result Captures both general and specific topics at different levels of a hierarchical structure.
A new method improves topic modeling accuracy using semantic filtering.
problem Improving topic modeling accuracy in text documents.
method Three-step process: generate word/word-pair, apply TF-IDF, merge similar semantic pairs.
result Improves topic accuracy by up to 12.99% compared to state-of-the-art models.
Paper proposes a method to discover topic evolutions from text data.
problem Difficulty in identifying new research topics from large text data.
method Uses sparseness-constrained Non-negative Matrix Factorization with generalized Jensen-Shannon divergence.
result Extracts more prominent topics and visualizes term-topic relationships.
Paper develops a framework for generating coherent image captions using visual features and hierarchical topics.
problem Generating semantically coherent paragraphs to describe image content.
method Plug-and-play hierarchical-topic-guided image paragraph generation framework integrating visual extractor and deep topic model.
result Proposed models can distill interpretable multi-layer semantic topics and generate diverse and coherent captions.
A dynamic keyword selection model for topic modeling of tweets.
problem Adjusting keywords dynamically to mimic past topics with novelty.
method Generative process selects keywords and documents, trained with variational lower bound and stochastic gradient optimization.
result Keyword-based topic model outperforms a sophisticated baseline model by 67%.
New topic modeling method scales to large datasets.
problem Large-scale topic modeling with high co-occurrence data.
method Introduced Full Dependence Mixture (FDM) model for direct topic learning.
result FDM model performs comparably or better than benchmarks on large datasets.
A single, stationary topic model such as latent Dirichlet allocation is inappropriate for modeling corpora that span long time periods, as the popularity of topics is likely to change over time. A number of models that incorporate time have been proposed, but in general they either exhibit limited forms of temporal var…
New NMF algorithms improve topic model fits.
problem Improving topic model fits for large datasets.
method Leveraging recent NMF optimization methods to fit topic models efficiently.
result Better topic model fits and faster computation times.
Spectral methods learn flexible topic models with topic correlations.
problem Learning topic models with arbitrary topic correlations.
method Flexible topic model using Normalized Infinitely Divisible (NID) distributions, learned via spectral methods.
result Improved perplexity on real datasets compared to baseline.
Paper proposes models to identify topic sparsity in social media texts.
problem Topic sparsity in online social media content.
method Sparsemax and relaxed Wasserstein for topic sparsity.
result Proposed models achieve better topic sparsity identification and performance.
nnLDA combines neural and probabilistic methods for better topic modeling with side information.
problem Lack of integration of auxiliary information in traditional topic models.
method nnLDA integrates side information through a neural prior mechanism, optimizing both neural and probabilistic components.
result nnLDA outperforms traditional models in topic coherence, perplexity, and classification.
CSTEM models document topics using VAE with semantic distance.
problem Inability of previous topic models to explain semantic relations correctly.
method Continuous semantic topic embedding model using variational autoencoder and Mahalanobis distance.
result Improves topic coherence and semantic relation explanation.
Top2Vec finds topic vectors from documents and words without needing stop words or custom settings.
problem Topic modeling weaknesses, including needing known topics, stop words, and custom settings.
method Joint document and word semantic embedding to find topic vectors automatically.
result Top2Vec finds more informative and representative topics than probabilistic models.
Combines topic modelling and Rasch modelling for MOOC student assessment.
problem Discovering meaningful topics that fit the Rasch model for MOOC student abilities.
method Combines Rasch model with non-negative matrix factorisation for topic discovery.
result Demonstrated suitability of the approach on MOOC data.
D-ETM models document topics over time using embeddings and variational inference.
problem Capturing evolving topic patterns in sequential documents.
method Combines D-LDA and word embeddings, using random walk priors and variational inference.
result D-ETM outperforms D-LDA on document completion tasks, learning more diverse and coherent topics.
A new method for document network embedding interprets and generalizes well.
problem Lack of interpretability and generalization to new documents in existing methods.
method Introduces Topic-Word Attention (TWA) and Inductive Document Network Embedding (IDNE) to generate document representations.
result Achieves state-of-the-art performance on various networks and produces meaningful representations.
The problem of topic modeling can be seen as a generalization of the clustering problem, in that it posits that observations are generated due to multiple latent factors (e.g., the words in each document are generated as a mixture of several active topics, as opposed to just one). This increased representational power …
Traditional Relational Topic Models provide a way to discover the hidden topics from a document network. Many theoretical and practical tasks, such as dimensional reduction, document clustering, link prediction, benefit from this revealed knowledge. However, existing relational topic models are based on an assumption t…
Corrected moment-based methods improve inference in topic model regression.
problem Inferential difficulties in topic model plug-in workflow for regression.
method Corrected spectral moment methods for LDA, response-weighted word moments.
result Direct identification of regression coefficients without estimating topic shares.
Develops a flexible deep autoencoding topic model with scalable hybrid Bayesian inference.
problem Flexible and interpretable document analysis models.
method DATM with hybrid Bayesian inference, including topic-layer-adaptive stochastic gradient Riemannian MCMC and Weibull variational encoder.
result Demonstrates scalability and efficacy on big corpora in unsupervised and supervised learning tasks.
GraphSTONE uses topic models to capture graph structures, improving GCN performance.
problem GCNs focus too much on node features and not enough on graph structures.
method GraphSTONE employs topic models of graphs to capture structural topics, which guide the aggregation of node features.
result GraphSTONE outperforms GCNs in performance, efficiency, and interpretability.
New algorithm improves topic modeling for short texts.
problem Inference of latent topics from short texts is challenging.
method Developed a stochastic divergence minimization algorithm for Biterm Topic Model (BTM).
result The new algorithm estimates latent topics more accurately and efficiently.
Proposes a new training framework for better interpretable latent models.
problem Lack of balance between generative explanations and label predictions in semi-supervised models.
method Prediction-constrained training objective integrating supervisory signals and stochastic gradient descent.
result Improved prediction quality and interpretable topics compared to previous models.
Paper finds better words for topic models by reranking top words.
problem Top words in topic models are not always representative.
method Reranking words by considering marginal probability over every topic.
result Reranked top words are more representative of topics.
Self-supervised learning excels in topic modeling by being less model-specific.
problem How self-supervised learning discovers useful representations in topic models.
method Applying self-supervised learning objectives to topic model-generated data.
result Self-supervised learning objectives can recover useful posterior information for topic models, outperforming misspecified models.
Study uses social media analytics to identify exercise-related topics.
problem Understanding exercise-related discussions on social media.
method Data collection, topic modeling, and data annotation.
result 86% of detected topics were meaningful after annotation.
Paper shows topic models are unstable and proposes ensemble learning to reduce it.
problem Topic models can vary significantly with different initializations.
method Proposes ensemble learning strategies to reduce instability in matrix factorization topic models.
result Ensemble learning reduces instability in topic models.
Study shows LDA topic models converge at rate n^-1/4 without strict topic separability.
problem Convergence rates of Latent Dirichlet Allocation (LDA) topic models.
method Maximum likelihood estimator, Wasserstein's distance metric, without separability or non-degeneracy assumptions.
result Maximum likelihood estimator converges at rate n^-1/4, optimal in worst case.
Topic-aware chatbot learns from NMF topic vectors.
problem Improving chatbot relevance based on user topics.
method Combines RNN with NMF for topic learning and attention.
result Chatbot provides more relevant answers based on topic.
Efficiently models correlated topics with topic embeddings.
problem High computational cost and poor scaling in correlated topic modeling.
method Compact topic embeddings and efficient inference in low-dimensional space.
result Handles larger model and data scales without sacrificing performance.