A new model CDTM improves text classification by concentrating document topics.
problem Unsupervised text classification with diverse topic distributions.
method Imposes an exponential entropy penalty on document topic distribution to encourage concentration.
result More coherent topics and concentrated, sparse document-topic distributions.
We introduce the author-topic model, a generative model for documents that extends Latent Dirichlet Allocation (LDA; Blei, Ng, & Jordan, 2003) to include authorship information. Each author is associated with a multinomial distribution over topics and each topic is associated with a multinomial distribution over words.…
Proposes a new multi-layer model for topic distributions.
problem Leveraging deep structures for learning word distributions of topics.
method A multi-layer generative process on word distributions of topics, where each topic is drawn from a mixture of topics from the layer above.
result Discover interpretable topic hierarchies and improve topic models' accuracy and interpretability.
LLMs encode latent topic distributions, suggesting Bayesian inference.
problem Capturing topic structure from large language models.
method Connecting LLM optimization to implicit Bayesian inference and de Finetti's theorem.
result LLMs recover latent topic distributions, matching LDA-generated topics.
New Gamma-Poisson model improves topic selection for short text.
problem Topic modelling for short text using Poisson distribution.
method Gamma-Poisson mixture model with collapsed Gibbs sampler.
result Gamma-Poisson model selects more accurate number of topics.
New method identifies quark and gluon jets from collider data.
problem Determine quark and gluon jet distributions from collider data.
method Apply topic modeling to jet distributions, using parton shower and theoretical predictions.
result Determined separate quark and gluon jet distributions and spectra.
A new parallel MCMC algorithm improves topic modeling without communication.
problem Quasi-ergodicity problem in topic modeling due to multimodal topic distributions.
method Developed an embarrassingly parallel MCMC algorithm for sLDA by switching topic combination and labeling prediction.
result Out-of-sample prediction performance is comparable to non-parallel sLDA but computation time is significantly reduced.
Spectral methods learn flexible topic models with topic correlations.
problem Learning topic models with arbitrary topic correlations.
method Flexible topic model using Normalized Infinitely Divisible (NID) distributions, learned via spectral methods.
result Improved perplexity on real datasets compared to baseline.
We describe our language-independent unsupervised word sense induction system. This system only uses topic features to cluster different word senses in their global context topic space. Using unlabeled data, this system trains a latent Dirichlet allocation (LDA) topic model then uses it to infer the topics distribution…
A new topic model uses word embeddings on a sphere for better topic coherence.
problem Traditional topic models ignore semantic word correlations.
method Proposes von Mises-Fisher distribution for word density on a unit sphere, using Hierarchical Dirichlet Process and Stochastic Variational Inference.
result The model outperforms existing methods in topic coherence.
CFTM uses fractional Brownian motion for dynamic topic modeling.
problem Identifying long-term dependency or roughness in topic and word distributions over time.
method Continuous Time Fractional Topic Model (cFTM) incorporating fractional Brownian motion.
result cFTM captures long-term dependency or roughness in topic and word distributions.
Top2Vec finds topic vectors from documents and words without needing stop words or custom settings.
problem Topic modeling weaknesses, including needing known topics, stop words, and custom settings.
method Joint document and word semantic embedding to find topic vectors automatically.
result Top2Vec finds more informative and representative topics than probabilistic models.
A novel method learns word embeddings and topics using Wasserstein distance.
problem Learning word embeddings and topics from text data.
method Distilled Wasserstein learning framework for joint word embedding and topic modeling.
result Superior performance on disease network construction, mortality prediction, and procedure recommendation.
A new neural topic model using optimal transport improves document representation and topic coherence.
problem Challenges in achieving good document representation and coherent/diverse topics in existing NTMs.
method Proposes a neural topic model via optimal transport, learning topic distribution by minimising OT distance to document word distributions.
result Significantly outperforms state-of-the-art NTMs on discovering coherent and diverse topics.
Paper proposes a method to improve language model performance on unknown distributions.
problem Language models trained on diverse data can perform poorly on unseen distributions.
method Distributionally robust optimization (DRO) to minimize worst-case performance over a mixture of potential test distributions.
result Topic CVaR approach reduces perplexity by 5.5 points compared to standard maximum likelihood.
Paper proposes models to identify topic sparsity in social media texts.
problem Topic sparsity in online social media content.
method Sparsemax and relaxed Wasserstein for topic sparsity.
result Proposed models achieve better topic sparsity identification and performance.
We describe a new method for visualizing topics, the distributions over terms that are automatically extracted from large text corpora using latent variable models. Our method finds significant n-grams related to a topic, which are then used to help understand and interpret the underlying distribution. Compared with …
Model clusters authors and topics in short texts like social media posts.
problem Analysis of short texts is difficult due to brevity and lack of context.
method Expands Latent Dirichlet Allocation to model word dependencies and cluster users.
result Improves topic and user clustering, outperforming traditional methods.
ETM discovers interpretable topics in large vocabularies.
problem Existing topic models fail with large, heavy-tailed vocabularies.
method Generative model combining topic models and word embeddings with variational inference.
result ETM discovers interpretable topics even with large vocabularies.
Efficiently trains HDP topic models on large datasets using a sparse data-parallel sampler.
problem Scaling non-parametric topic models to large datasets.
method Data-parallel training with a doubly sparse sampler for HDP topic models.
result Trains HDP topic models on a 8m document, 768m token PubMed corpus in under 4 days.
Develops a flexible deep autoencoding topic model with scalable hybrid Bayesian inference.
problem Flexible and interpretable document analysis models.
method DATM with hybrid Bayesian inference, including topic-layer-adaptive stochastic gradient Riemannian MCMC and Weibull variational encoder.
result Demonstrates scalability and efficacy on big corpora in unsupervised and supervised learning tasks.
mixEHR model reveals latent disease topics from EHR data.
problem Mining heterogeneous, non-randomly missing EHR data.
method Generative model integrating collaborative filtering and latent topic models.
result mixEHR reveals meaningful multi-disease insights and outperforms previous methods.
Neural model predicts survival outcomes and reveals feature relationships.
problem Predicting time-to-event outcomes and understanding feature relationships in clinical data.
method Survival and topic modeling combined in a neural network framework.
result Neural survival-supervised topic models achieve competitive accuracy with interpretability.
Topic models, such as Latent Dirichlet Allocation (LDA), posit that documents are drawn from admixtures of distributions over words, known as topics. The inference problem of recovering topics from admixtures, is NP-hard. Assuming separability, a strong assumption, [4] gave the first provable algorithm for inference. F…
Paper proposes a new model to infer topic-based connection structures from noisy adjacency matrices.
problem Traditional community detection assumes static connection structures, but real-world connections vary based on topic.
method Introduces latent model with influence and receptivity vectors for each node, estimating topic distributions from observed data.
result The model can estimate topic-based connection structures with theoretical guarantees and outperforms existing methods.
The logistic normal distribution has recently been adapted via the transformation of multivariate Gaus- sian variables to model the topical distribution of documents in the presence of correlations among topics. In this paper, we propose a probit normal alternative approach to modelling correlated topical structures. O…
New metric correlates local topic quality with human judgments.
problem Evaluation of topic models focuses on global metrics, ignoring token-level assignments.
method Proposed a human evaluation task and automated metrics to assess local topic quality.
result Consistency metric correlates best with human judgments of local topic quality.
This paper improves topic model estimation for sparse distributions and applies it to Wasserstein distances.
problem Estimating sparse topic distributions in topic models with high-dimensional data.
method MLE for topic weights when A is known, plug-in estimator for unknown A. result MLE can be exactly sparse and contain true zero pattern of topic weights.
A single, stationary topic model such as latent Dirichlet allocation is inappropriate for modeling corpora that span long time periods, as the popularity of topics is likely to change over time. A number of models that incorporate time have been proposed, but in general they either exhibit limited forms of temporal var…
Scalable algorithm for extracting topic hierarchies from large text corpora.
problem Efficient inference for hierarchical topic models on large datasets.
method Partially collapsed Gibbs sampling (PCGS) algorithm combined with efficient distributed implementation.
result 111 times more efficient than previous implementation for hLDA.
WHAI combines autoencoding and MCMC for scalable topic modeling.
problem Training scalable deep topic models for big corpora.
method Develops a hybrid autoencoding inference network for deep latent Dirichlet allocation.
result WHAI achieves both scalability and speed in topic inference.
We examine the problem of learning a probabilistic model for melody directly from musical sequences belonging to the same genre. This is a challenging task as one needs to capture not only the rich temporal structure evident in music, but also the complex statistical dependencies among different music components. To ad…
The simplicial condition and other stronger conditions that imply it have recently played a central role in developing polynomial time algorithms with provable asymptotic consistency and sample complexity guarantees for topic estimation in separable topic models. Of these algorithms, those that rely solely on the simpl…
In latent Dirichlet allocation (LDA), topics are multinomial distributions over the entire vocabulary. However, the vocabulary usually contains many words that are not relevant in forming the topics. We adopt a variable selection method widely used in statistical modeling as a dimension reduction tool and combine it wi…
Enhances topic models to better handle polysemous words.
problem Lack of polysemy handling in Gaussian latent Dirichlet allocation.
method Introduces a hierarchical structure to capture polysemy in Gaussian latent Dirichlet allocation.
result Significantly improves polysemy detection and provides more parsimonious topic representations.
Detects changes in topic proportions over time in large text datasets.
problem Unsupervised detection of structural changes in topic distributions over time.
method Specialised temporal topic model with changepoint detection, approximate inference using sample splitting and likelihood ratio statistic.
result Automated detection of changepoints in topic proportions, facilitating interpretable results.
A new topic modeling method that minimizes topic simplex volume.
problem Topic modeling efficiency and accuracy.
method Reformulates LDA as minimizing topic simplex volume, uses convex relaxation and ADMM.
result Relaxed problem has same global minimum as original under assumptions.
A new model for analyzing large SSDs using tags and text.
problem Modeling large-scale semi-structured documents efficiently.
method Tag-Weighted Topic Model (TWTM) with variational inference and MapReduce.
result Efficient and robust model for tags, text, and document modeling.
We present an LDA approach to entity disambiguation. Each topic is associated with a Wikipedia article and topics generate either content words or entity mentions. Training such models is challenging because of the topic and vocabulary size, both in the millions. We tackle these problems using a novel distributed infer…
Develops efficient algorithm for topic discovery with novel geometric insights.
problem Discovering topics from documents with shared latent factors.
method Geometric insights from normalized word co-occurrence matrix and isotropic random projections.
result Provably efficient algorithm with polynomial computation and sample complexity bounds.
nnLDA combines neural and probabilistic methods for better topic modeling with side information.
problem Lack of integration of auxiliary information in traditional topic models.
method nnLDA integrates side information through a neural prior mechanism, optimizing both neural and probabilistic components.
result nnLDA outperforms traditional models in topic coherence, perplexity, and classification.
Recent advances in topic models have explored complicated structured distributions to represent topic correlation. For example, the pachinko allocation model (PAM) captures arbitrary, nested, and possibly sparse correlations between topics using a directed acyclic graph (DAG). While PAM provides more flexibility and gr…
Topic models have proven to be a useful tool for discovering latent structures in document collections. However, most document collections often come as temporal streams and thus several aspects of the latent structure such as the number of topics, the topics' distribution and popularity are time-evolving. Several mode…
D-ETM models document topics over time using embeddings and variational inference.
problem Capturing evolving topic patterns in sequential documents.
method Combines D-LDA and word embeddings, using random walk priors and variational inference.
result D-ETM outperforms D-LDA on document completion tasks, learning more diverse and coherent topics.
CSTEM models document topics using VAE with semantic distance.
problem Inability of previous topic models to explain semantic relations correctly.
method Continuous semantic topic embedding model using variational autoencoder and Mahalanobis distance.
result Improves topic coherence and semantic relation explanation.
A new nonparametric model discovers hidden topics in document networks without knowing the number of topics in advance.
problem Discovering hidden topics in document networks without knowing the number of topics in advance.
method Proposes a nonparametric relational topic model using dependent gamma processes to represent topic interests of documents and their network structure.
result Can discover hidden topics and their number simultaneously using a posterior inference algorithm.
Deep model captures interactions in online forums.
problem Modeling discursive interactions in online forums.
method Coupled Distributed Topics model with deep architecture and GPU-based inference.
result Model outperforms existing methods in online discourse analysis.
New tools for estimating and inferring Wasserstein distance in topic models.
problem Estimating and inferring the Wasserstein distance between mixing measures in topic models.
method New canonical interpretation and tools for inference on Wasserstein distance in topic models.
result First minimax lower bounds and fully data-driven inferential tools for the Wasserstein distance in topic models.