Bayesian dynamic topic model improves topic prevalence prediction.
problem Estimating document-specific topic proportions in dynamic topic models.
method Developed a Bayesian dynamic topic model with covariates and dynamic structure, including polynomial trends and periodicity. Used MCMC algorithm with Polya-Gamma data augmentation and Gaussian approximation.
result Explicitly modeling polynomial and periodic behavior improves topic prevalence prediction.
Bayesian method models dynamic text networks and topics.
problem Discovering community structure and topics in evolving text networks.
method Bayesian approach combining network and topic modeling.
result Complex community structure identified, dependent on blogger interests.
Paper presents a scalable inference algorithm for dynamic topic models.
problem Scalability issue in existing dynamic topic models.
method Gibbs Sampling with Stochastic Gradient Langevin Dynamics and Metropolis-Hastings.
result Learned the dynamics of 1,000 topics from 2.6 million documents in less than half an hour.
A dynamic keyword selection model for topic modeling of tweets.
problem Adjusting keywords dynamically to mimic past topics with novelty.
method Generative process selects keywords and documents, trained with variational lower bound and stochastic gradient optimization.
result Keyword-based topic model outperforms a sophisticated baseline model by 67%.
D-ETM models document topics over time using embeddings and variational inference.
problem Capturing evolving topic patterns in sequential documents.
method Combines D-LDA and word embeddings, using random walk priors and variational inference.
result D-ETM outperforms D-LDA on document completion tasks, learning more diverse and coherent topics.
CFTM uses fractional Brownian motion for dynamic topic modeling.
problem Identifying long-term dependency or roughness in topic and word distributions over time.
method Continuous Time Fractional Topic Model (cFTM) incorporating fractional Brownian motion.
result cFTM captures long-term dependency or roughness in topic and word distributions.
Topic models are probabilistic models for discovering topical themes in collections of documents. In real world applications, these models provide us with the means of organizing what would otherwise be unstructured collections. They can help us cluster a huge collection into different topics or find a subset of the co…
New models discover new topics over time in topic modeling.
problem Discovering new topics over time in topic modeling.
method Nonparametric Bayesian models and Hungarian matching algorithm.
result Significantly faster than existing methods, discovering new topics in large datasets.
In this paper, we develop the continuous time dynamic topic model (cDTM). The cDTM is a dynamic topic model that uses Brownian motion to model the latent topics through a sequential collection of documents, where a "topic" is a pattern of word use that we expect to evolve over the course of the collection. We derive an…
CLDA improves topic modeling for large, dynamic datasets.
problem Dynamic topic modeling for large, diverse text streams.
method Data decomposition followed by topic modeling on segments, then clustering.
result Very fast runtime and insight into topic composition over time.
New method improves dynamic topic modeling for large-scale data.
problem Lack of temporal information in dynamic topic modeling.
method Nonnegative CP tensor decomposition (NNCPD) for data tensor.
result Significantly improved results compared to NMF-based methods.
Topic models have proven to be a useful tool for discovering latent structures in document collections. However, most document collections often come as temporal streams and thus several aspects of the latent structure such as the number of topics, the topics' distribution and popularity are time-evolving. Several mode…
Modeling lead-lag relationship between two text corpora for improved topic modeling.
problem Recognizing the relationship between multiple text corpora for better topic modeling.
method Proposed a jointly dynamic topic model and embedding extension for large-scale text corpus.
result The proposed model can well recognize the lead-lag relationship between two text corpora and improve topic learning.
New learning algorithms for dynamic topic modeling in video analysis.
problem Efficiently processing large volumes of video data for autonomous decisions.
method Two novel learning algorithms based on expectation maximisation and variational Bayes inference.
result Comparison of learning algorithms on real video data.
We develop dependent hierarchical normalized random measures and apply them to dynamic topic modeling. The dependency arises via superposition, subsampling and point transition on the underlying Poisson processes of these measures. The measures used include normalised generalised Gamma processes that demonstrate power …
Dynamic topic model detects abnormal behavior in video sequences.
problem Detecting abnormal behavior in sequential video data.
method Dynamic Hierarchical Dirichlet Process with online inference algorithms.
result The dynamic model improves abnormal behavior detection performance.
DAPPER improves scalability of DAP topic model for large corpora.
problem Scaling complex models like DAP to large text corpora.
method Adapted approximate inference techniques for DAP, developing CVI-based EM.
result Significant improvements in model fit and training time without compromising structure.
Topic model captures health journeys of multiple authors.
problem Challenges in topic modeling health journals due to asynchronous writing.
method Dynamic Author-Persona topic model (DAP) with regularized variational inference.
result Significant improvements over competing models, especially with regularization.
New DTMs model text evolution with Gaussian processes and scalable inference.
problem Challenges in modeling text evolution with continuous stochastic processes.
method Extended tractable priors to Gaussian processes and developed scalable inference methods.
result Found interesting patterns in large-scale datasets not accessible before.
Dynamic topic model improves mental health note analysis for children.
problem Lack of longitudinal topic models for psychiatric clinical notes.
method Developed a dynamic topic model with consistent topics and individualized temporal dependencies.
result Achieved a 38% increase in topic coherence.
Study reveals AI's spontaneous topic changes in text prediction.
problem AI's inability to spontaneously switch topics like humans.
method Defined topic as Token Priority Graphs (TPGs) and analyzed self-attention models.
result AI can only switch topics if lower-priority tokens outnumber higher-priority ones.
Study analyzes OR/MS literature using topic models.
problem Understanding the content and evolution of OR/MS literature.
method Developed a topic model using Latent Dirichlet Allocation (LDA).
result Identified temporal dynamics and unique scopes of journals.
Study uses news trends to predict infectious disease outbreaks.
problem Predicting infectious disease outbreaks using news reports.
method Supervised temporal topic models to transform news articles into trends.
result Temporal topic trends from disease news capture outbreak dynamics.
Proposes a new RNN model for grouped sequential data with varying time intervals.
problem Implicitly models fixed time intervals between observations and lacks group-level effects.
method Mixed membership framework for RNN, learning group-level base parameter.
result Demonstrates dynamic topic modeling with evolving topic distributions over time.
nnLDA combines neural and probabilistic methods for better topic modeling with side information.
problem Lack of integration of auxiliary information in traditional topic models.
method nnLDA integrates side information through a neural prior mechanism, optimizing both neural and probabilistic components.
result nnLDA outperforms traditional models in topic coherence, perplexity, and classification.
AOBTM adapts online topic modeling for short app reviews, revealing coherent topics over time.
problem Challenges in inferring latent topics from short, dynamic app reviews over multiple versions.
method Adaptive Online Biterm Topic Model (AOBTM) that addresses sparsity and considers statistical data from previous versions.
result AOBTM finds more coherent topics and outperforms state-of-the-art baselines.
Proposes a new RNN for language generation capturing long-range dependencies.
problem Capturing long-range word dependencies and sentence order in text corpora.
method Recurrent Hierarchical Topic-Guided RNN with dynamic deep topic model.
result Outperforms larger-context RNN-based language models and learns interpretable topics.
Neural differential equations combine deep learning and differential equations for modeling complex systems.
problem Modeling complex systems with high capacity and efficiency.
method Combining neural networks and differential equations, focusing on neural ordinary, controlled, and stochastic differential equations.
result NDEs offer high-capacity function approximation, strong priors, and handle irregular data efficiently.
Proposes a method for time-evolving and difficulty-level topic discovery.
problem Discovering evolving and advanced topics in dynamic corpora.
method Constrained Coupled Matrix-Tensor Factorization with expertise-level constraints.
result Identifies evolving and difficulty-level topics in community-contributed content.
Bayesian model detects video anomalies.
problem Anomaly detection in video data.
method Dynamic Bayesian nonparametric topic model with Gibbs samplers.
result Dynamic model outperforms static model in anomaly detection.
Scalable algorithm for extracting topic hierarchies from large text corpora.
problem Efficient inference for hierarchical topic models on large datasets.
method Partially collapsed Gibbs sampling (PCGS) algorithm combined with efficient distributed implementation.
result 111 times more efficient than previous implementation for hLDA.
This research tackles unsupervised topic extraction in noisy social media data.
problem Capturing customer insights from social media data is challenging due to noise and heterogeneity.
method The research presents three nonparametric approaches based on the Variational Autoencoder framework: Embedded Dirichlet Process, Embedded Hierarchical Dirichlet Process, and time-aware Dynamic Embedded Dirichlet Process.
result The models achieve equal to better performance than state-of-the-art methods in topic extraction from noisy social media data.
Linking manifold structures to group dynamics.
problem Understanding geometric structures on manifolds and group dynamics.
method Study of geometric structures and representations of discrete groups into Lie groups.
result Recent findings on connections between manifold structures and group dynamics.
Detection of emerging topics are now receiving renewed interest motivated by the rapid growth of social networks. Conventional term-frequency-based approaches may not be appropriate in this context, because the information exchanged are not only texts but also images, URLs, and videos. We focus on the social aspects of…
Analyzes word2vec-like models revealing linear subspaces learned during training.
problem Understanding representation learning in word embeddings.
method Analytical solution of word2vec loss dynamics and final embeddings.
result Models learn orthogonal linear subspaces incrementally, representing interpretable concepts.
Hybrid approach combines topic and graph embeddings for legal document clustering.
problem Challenges in classifying legal texts due to domain-specific language and limited labeled data.
method Combines unsupervised topic and graph embeddings with a supervised model.
result Improves clustering quality over text-only or graph-only embeddings.
Transformers learn topic structure through embedding and attention mechanisms.
problem Understanding how transformers capture semantic structure in text.
method Combination of mathematical analysis and experiments on Wikipedia and synthetic data.
result Embedding and attention layers encode topic structure in transformers.
We present the nested Chinese restaurant process (nCRP), a stochastic process which assigns probability distributions to infinitely-deep, infinitely-branching trees. We show how this stochastic process can be used as a prior distribution in a Bayesian nonparametric model of document collections. Specifically, we presen…
New BAM model connects tensor factorization and topic models using Polya Urns.
problem Efficiently modeling and analyzing nonnegative tensors and topic distributions.
method Dynamic generative model BAM based on Poisson process and Polya-Bayes process.
result Developed efficient simulation algorithms for NTF and topic models.
Proposes a new multi-layer model for topic distributions.
problem Leveraging deep structures for learning word distributions of topics.
method A multi-layer generative process on word distributions of topics, where each topic is drawn from a mixture of topics from the layer above.
result Discover interpretable topic hierarchies and improve topic models' accuracy and interpretability.
Efficiently models correlated topics with topic embeddings.
problem High computational cost and poor scaling in correlated topic modeling.
method Compact topic embeddings and efficient inference in low-dimensional space.
result Handles larger model and data scales without sacrificing performance.
Document clustering and topic modeling are two closely related tasks which can mutually benefit each other. Topic modeling can project documents into a topic space which facilitates effective document clustering. Cluster labels discovered by document clustering can be incorporated into topic models to extract local top…
ETM discovers interpretable topics in large vocabularies.
problem Existing topic models fail with large, heavy-tailed vocabularies.
method Generative model combining topic models and word embeddings with variational inference.
result ETM discovers interpretable topics even with large vocabularies.
A new model CDTM improves text classification by concentrating document topics.
problem Unsupervised text classification with diverse topic distributions.
method Imposes an exponential entropy penalty on document topic distribution to encourage concentration.
result More coherent topics and concentrated, sparse document-topic distributions.
Paper finds better words for topic models by reranking top words.
problem Top words in topic models are not always representative.
method Reranking words by considering marginal probability over every topic.
result Reranked top words are more representative of topics.
Predictive topic models retain only relevant terms for better prediction and topic coherence.
problem Misspecification of topic models leads to poor prediction and topic coherence.
method Uses supervisory signal to select vocabulary terms improving prediction performance.
result Prediction-focused topic models learn more coherent topics while maintaining competitive predictions.
New metric correlates local topic quality with human judgments.
problem Evaluation of topic models focuses on global metrics, ignoring token-level assignments.
method Proposed a human evaluation task and automated metrics to assess local topic quality.
result Consistency metric correlates best with human judgments of local topic quality.
New metric R2 improves topic model evaluation.
problem Lack of standard cross-contextual evaluation metrics for topic modeling.
method Introduces R2 as a coefficient of determination for topic models. result Improves topic model evaluation by providing a standard metric.