A dynamic keyword selection model for topic modeling of tweets.
problem Adjusting keywords dynamically to mimic past topics with novelty.
method Generative process selects keywords and documents, trained with variational lower bound and stochastic gradient optimization.
result Keyword-based topic model outperforms a sophisticated baseline model by 67%.
D-ETM models document topics over time using embeddings and variational inference.
problem Capturing evolving topic patterns in sequential documents.
method Combines D-LDA and word embeddings, using random walk priors and variational inference.
result D-ETM outperforms D-LDA on document completion tasks, learning more diverse and coherent topics.
CFTM uses fractional Brownian motion for dynamic topic modeling.
problem Identifying long-term dependency or roughness in topic and word distributions over time.
method Continuous Time Fractional Topic Model (cFTM) incorporating fractional Brownian motion.
result cFTM captures long-term dependency or roughness in topic and word distributions.
Topic models are probabilistic models for discovering topical themes in collections of documents. In real world applications, these models provide us with the means of organizing what would otherwise be unstructured collections. They can help us cluster a huge collection into different topics or find a subset of the co…
New models discover new topics over time in topic modeling.
problem Discovering new topics over time in topic modeling.
method Nonparametric Bayesian models and Hungarian matching algorithm.
result Significantly faster than existing methods, discovering new topics in large datasets.
In dynamic topic modeling, the proportional contribution of a topic to a document depends on the temporal dynamics of that topic's overall prevalence in the corpus. We extend the Dynamic Topic Model of Blei and Lafferty (2006) by explicitly modeling document level topic proportions with covariates and dynamic structure…
In this paper, we develop the continuous time dynamic topic model (cDTM). The cDTM is a dynamic topic model that uses Brownian motion to model the latent topics through a sequential collection of documents, where a "topic" is a pattern of word use that we expect to evolve over the course of the collection. We derive an…
Dynamic topic models (DTMs) are very effective in discovering topics and capturing their evolution trends in time series data. To do posterior inference of DTMs, existing methods are all batch algorithms that scan the full dataset before each update of the model and make inexact variational approximations with mean-fie…
New method improves dynamic topic modeling for large-scale data.
problem Lack of temporal information in dynamic topic modeling.
method Nonnegative CP tensor decomposition (NNCPD) for data tensor.
result Significantly improved results compared to NMF-based methods.
The last decade has seen great progress in both dynamic network modeling and topic modeling. This paper draws upon both areas to create a Bayesian method that allows topic discovery to inform the latent network model and the network structure to facilitate topic identification. We apply this method to the 467 top polit…
Topic models have proven to be a useful tool for discovering latent structures in document collections. However, most document collections often come as temporal streams and thus several aspects of the latent structure such as the number of topics, the topics' distribution and popularity are time-evolving. Several mode…
Modeling lead-lag relationship between two text corpora for improved topic modeling.
problem Recognizing the relationship between multiple text corpora for better topic modeling.
method Proposed a jointly dynamic topic model and embedding extension for large-scale text corpus.
result The proposed model can well recognize the lead-lag relationship between two text corpora and improve topic learning.
We develop dependent hierarchical normalized random measures and apply them to dynamic topic modeling. The dependency arises via superposition, subsampling and point transition on the underlying Poisson processes of these measures. The measures used include normalised generalised Gamma processes that demonstrate power …
DAPPER improves scalability of DAP topic model for large corpora.
problem Scaling complex models like DAP to large text corpora.
method Adapted approximate inference techniques for DAP, developing CVI-based EM.
result Significant improvements in model fit and training time without compromising structure.
Topic model captures health journeys of multiple authors.
problem Challenges in topic modeling health journals due to asynchronous writing.
method Dynamic Author-Persona topic model (DAP) with regularized variational inference.
result Significant improvements over competing models, especially with regularization.
New DTMs model text evolution with Gaussian processes and scalable inference.
problem Challenges in modeling text evolution with continuous stochastic processes.
method Extended tractable priors to Gaussian processes and developed scalable inference methods.
result Found interesting patterns in large-scale datasets not accessible before.
Dynamic topic model improves mental health note analysis for children.
problem Lack of longitudinal topic models for psychiatric clinical notes.
method Developed a dynamic topic model with consistent topics and individualized temporal dependencies.
result Achieved a 38% increase in topic coherence.
Topic modeling, a method for extracting the underlying themes from a collection of documents, is an increasingly important component of the design of intelligent systems enabling the sense-making of highly dynamic and diverse streams of text data. Traditional methods such as Dynamic Topic Modeling (DTM) do not lend the…
Study reveals AI's spontaneous topic changes in text prediction.
problem AI's inability to spontaneously switch topics like humans.
method Defined topic as Token Priority Graphs (TPGs) and analyzed self-attention models.
result AI can only switch topics if lower-priority tokens outnumber higher-priority ones.
Proposes a new RNN model for grouped sequential data with varying time intervals.
problem Implicitly models fixed time intervals between observations and lacks group-level effects.
method Mixed membership framework for RNN, learning group-level base parameter.
result Demonstrates dynamic topic modeling with evolving topic distributions over time.
nnLDA combines neural and probabilistic methods for better topic modeling with side information.
problem Lack of integration of auxiliary information in traditional topic models.
method nnLDA integrates side information through a neural prior mechanism, optimizing both neural and probabilistic components.
result nnLDA outperforms traditional models in topic coherence, perplexity, and classification.
AOBTM adapts online topic modeling for short app reviews, revealing coherent topics over time.
problem Challenges in inferring latent topics from short, dynamic app reviews over multiple versions.
method Adaptive Online Biterm Topic Model (AOBTM) that addresses sparsity and considers statistical data from previous versions.
result AOBTM finds more coherent topics and outperforms state-of-the-art baselines.
In retrospective assessments, internet news reports have been shown to capture early reports of unknown infectious disease transmission prior to official laboratory confirmation. In general, media interest and reporting peaks and wanes during the course of an outbreak. In this study, we quantify the extent to which med…
Semi-supervised and unsupervised systems provide operators with invaluable support and can tremendously reduce the operators load. In the light of the necessity to process large volumes of video data and provide autonomous decisions, this work proposes new learning algorithms for activity analysis in video. The activit…
This paper proposes a novel dynamic Hierarchical Dirichlet Process topic model that considers the dependence between successive observations. Conventional posterior inference algorithms for this kind of models require processing of the whole data through several passes. It is computationally intractable for massive or …
Proposes a new RNN for language generation capturing long-range dependencies.
problem Capturing long-range word dependencies and sentence order in text corpora.
method Recurrent Hierarchical Topic-Guided RNN with dynamic deep topic model.
result Outperforms larger-context RNN-based language models and learns interpretable topics.
This study investigates the content of the published scientific literature in the fields of operations research and management science (OR/MS) since the early 1950s. Our study is based on 80,757 published journal abstracts from 37 of the leading OR/MS journals. We have developed a topic model, using Latent Dirichlet Al…
Neural differential equations combine deep learning and differential equations for modeling complex systems.
problem Modeling complex systems with high capacity and efficiency.
method Combining neural networks and differential equations, focusing on neural ordinary, controlled, and stochastic differential equations.
result NDEs offer high-capacity function approximation, strong priors, and handle irregular data efficiently.
Proposes a method for time-evolving and difficulty-level topic discovery.
problem Discovering evolving and advanced topics in dynamic corpora.
method Constrained Coupled Matrix-Tensor Factorization with expertise-level constraints.
result Identifies evolving and difficulty-level topics in community-contributed content.
A novel dynamic Bayesian nonparametric topic model for anomaly detection in video is proposed in this paper. Batch and online Gibbs samplers are developed for inference. The paper introduces a new abnormality measure for decision making. The proposed method is evaluated on both synthetic and real data. The comparison w…
This research tackles unsupervised topic extraction in noisy social media data.
problem Capturing customer insights from social media data is challenging due to noise and heterogeneity.
method The research presents three nonparametric approaches based on the Variational Autoencoder framework: Embedded Dirichlet Process, Embedded Hierarchical Dirichlet Process, and time-aware Dynamic Embedded Dirichlet Process.
result The models achieve equal to better performance than state-of-the-art methods in topic extraction from noisy social media data.
Linking manifold structures to group dynamics.
problem Understanding geometric structures on manifolds and group dynamics.
method Study of geometric structures and representations of discrete groups into Lie groups.
result Recent findings on connections between manifold structures and group dynamics.
Detection of emerging topics are now receiving renewed interest motivated by the rapid growth of social networks. Conventional term-frequency-based approaches may not be appropriate in this context, because the information exchanged are not only texts but also images, URLs, and videos. We focus on the social aspects of…
Analyzes word2vec-like models revealing linear subspaces learned during training.
problem Understanding representation learning in word embeddings.
method Analytical solution of word2vec loss dynamics and final embeddings.
result Models learn orthogonal linear subspaces incrementally, representing interpretable concepts.
Nested Chinese Restaurant Process (nCRP) topic models are powerful nonparametric Bayesian methods to extract a topic hierarchy from a given text corpus, where the hierarchical structure is automatically determined by the data. Hierarchical Latent Dirichlet Allocation (hLDA) is a popular instance of nCRP topic models. H…
Hybrid approach combines topic and graph embeddings for legal document clustering.
problem Challenges in classifying legal texts due to domain-specific language and limited labeled data.
method Combines unsupervised topic and graph embeddings with a supervised model.
result Improves clustering quality over text-only or graph-only embeddings.
Transformers learn topic structure through embedding and attention mechanisms.
problem Understanding how transformers capture semantic structure in text.
method Combination of mathematical analysis and experiments on Wikipedia and synthetic data.
result Embedding and attention layers encode topic structure in transformers.
We present the nested Chinese restaurant process (nCRP), a stochastic process which assigns probability distributions to infinitely-deep, infinitely-branching trees. We show how this stochastic process can be used as a prior distribution in a Bayesian nonparametric model of document collections. Specifically, we presen…
New BAM model connects tensor factorization and topic models using Polya Urns.
problem Efficiently modeling and analyzing nonnegative tensors and topic distributions.
method Dynamic generative model BAM based on Poisson process and Polya-Bayes process.
result Developed efficient simulation algorithms for NTF and topic models.
Proposes a new multi-layer model for topic distributions.
problem Leveraging deep structures for learning word distributions of topics.
method A multi-layer generative process on word distributions of topics, where each topic is drawn from a mixture of topics from the layer above.
result Discover interpretable topic hierarchies and improve topic models' accuracy and interpretability.
Efficiently models correlated topics with topic embeddings.
problem High computational cost and poor scaling in correlated topic modeling.
method Compact topic embeddings and efficient inference in low-dimensional space.
result Handles larger model and data scales without sacrificing performance.
Document clustering and topic modeling are two closely related tasks which can mutually benefit each other. Topic modeling can project documents into a topic space which facilitates effective document clustering. Cluster labels discovered by document clustering can be incorporated into topic models to extract local top…
ETM discovers interpretable topics in large vocabularies.
problem Existing topic models fail with large, heavy-tailed vocabularies.
method Generative model combining topic models and word embeddings with variational inference.
result ETM discovers interpretable topics even with large vocabularies.
A new model CDTM improves text classification by concentrating document topics.
problem Unsupervised text classification with diverse topic distributions.
method Imposes an exponential entropy penalty on document topic distribution to encourage concentration.
result More coherent topics and concentrated, sparse document-topic distributions.
Paper finds better words for topic models by reranking top words.
problem Top words in topic models are not always representative.
method Reranking words by considering marginal probability over every topic.
result Reranked top words are more representative of topics.
Predictive topic models retain only relevant terms for better prediction and topic coherence.
problem Misspecification of topic models leads to poor prediction and topic coherence.
method Uses supervisory signal to select vocabulary terms improving prediction performance.
result Prediction-focused topic models learn more coherent topics while maintaining competitive predictions.
New metric R2 improves topic model evaluation.
problem Lack of standard cross-contextual evaluation metrics for topic modeling.
method Introduces R2 as a coefficient of determination for topic models. result Improves topic model evaluation by providing a standard metric.
New metric correlates local topic quality with human judgments.
problem Evaluation of topic models focuses on global metrics, ignoring token-level assignments.
method Proposed a human evaluation task and automated metrics to assess local topic quality.
result Consistency metric correlates best with human judgments of local topic quality.