New metric correlates local topic quality with human judgments.
problem Evaluation of topic models focuses on global metrics, ignoring token-level assignments.
method Proposed a human evaluation task and automated metrics to assess local topic quality.
result Consistency metric correlates best with human judgments of local topic quality.
Proposes a new method for evaluating and constructing hierarchical topic models.
problem Evaluation and construction of hierarchical topic models.
method Represent HTM as layers and edges, introduce quality measures, and develop a heterogeneous algorithm.
result The proposed heterogeneous algorithm significantly outperforms baseline approaches.
Predictive topic models retain only relevant terms for better prediction and topic coherence.
problem Misspecification of topic models leads to poor prediction and topic coherence.
method Uses supervisory signal to select vocabulary terms improving prediction performance.
result Prediction-focused topic models learn more coherent topics while maintaining competitive predictions.
Paper finds better words for topic models by reranking top words.
problem Top words in topic models are not always representative.
method Reranking words by considering marginal probability over every topic.
result Reranked top words are more representative of topics.
Predicts EHR features that improve prediction, learning coherent topics.
problem Balancing prediction quality and coherence in EHR topic models.
method Prediction-focused topic model that uses supervisory signal to retain relevant features.
result Prediction-focused topic models learn more coherent topics while maintaining competitive predictions.
We introduce Gaussian Process Topic Models (GPTMs), a new family of topic models which can leverage a kernel among documents while extracting correlated topics. GPTMs can be considered a systematic generalization of the Correlated Topic Models (CTMs) using ideas from Gaussian Process (GP) based embedding. Since GPTMs w…
A new method selects anchor words for better topic discovery in text corpora.
problem Selecting anchor words for improved topic modeling in text corpora.
method Proposes a new greedy method to find a minimum edge-weight anchor clique in a word similarity graph.
result The proposed method outperforms existing methods on topic quality and is faster.
Nested Chinese Restaurant Process (nCRP) topic models are powerful nonparametric Bayesian methods to extract a topic hierarchy from a given text corpus, where the hierarchical structure is automatically determined by the data. Hierarchical Latent Dirichlet Allocation (hLDA) is a popular instance of nCRP topic models. H…
ETM discovers interpretable topics in large vocabularies.
problem Existing topic models fail with large, heavy-tailed vocabularies.
method Generative model combining topic models and word embeddings with variational inference.
result ETM discovers interpretable topics even with large vocabularies.
Inference is an integral part of probabilistic topic models, but is often non-trivial to derive an efficient algorithm for a specific model. It is even much more challenging when we want to find a fast inference algorithm which always yields sparse latent representations of documents. In this article, we introduce a si…
Topic discovery has witnessed a significant growth as a field of data mining at large. In particular, time-evolving topic discovery, where the evolution of a topic is taken into account has been instrumental in understanding the historical context of an emerging topic in a dynamic corpus. Traditionally, time-evolving t…
Correlated topic modeling has been limited to small model and problem sizes due to their high computational cost and poor scaling. In this paper, we propose a new model which learns compact topic embeddings and captures topic correlations through the closeness between the topic vectors. Our method enables efficient inf…
Topic models have emerged as fundamental tools in unsupervised machine learning. Most modern topic modeling algorithms take a probabilistic view and derive inference algorithms based on Latent Dirichlet Allocation (LDA) or its variants. In contrast, we study topic modeling as a combinatorial optimization problem, and p…
A new neural topic model using optimal transport improves document representation and topic coherence.
problem Challenges in achieving good document representation and coherent/diverse topics in existing NTMs.
method Proposes a neural topic model via optimal transport, learning topic distribution by minimising OT distance to document word distributions.
result Significantly outperforms state-of-the-art NTMs on discovering coherent and diverse topics.
E-LDA offers faster, interpretable LDA topic models.
problem Inferring topics in LDA topic models with strong guarantees.
method Non-gradient combinatorial approach for faster convergence.
result Logarithmic parallel computation time and interpretability.
While generative models such as Latent Dirichlet Allocation (LDA) have proven fruitful in topic modeling, they often require detailed assumptions and careful specification of hyperparameters. Such model complexity issues only compound when trying to generalize generative models to incorporate human input. We introduce …
Paper proposes a Renyi entropy-based method for tuning hierarchical topic models.
problem Tuning hierarchical topic models, especially determining the number of topics at each level, is challenging.
method The paper introduces a Renyi entropy-based metric for quality assessment and a practical tuning concept.
result The proposed method can estimate the number of topics for two hierarchical levels in hARTM model.
One of the core problems in statistical models is the estimation of a posterior distribution. For topic models, the problem of posterior inference for individual texts is particularly important, especially when dealing with data streams, but is often intractable in the worst case. As a consequence, existing methods for…
Hybrid approach combines topic and graph embeddings for legal document clustering.
problem Challenges in classifying legal texts due to domain-specific language and limited labeled data.
method Combines unsupervised topic and graph embeddings with a supervised model.
result Improves clustering quality over text-only or graph-only embeddings.
BERTopic improves financial text analysis with FinTextSim's contextual embeddings.
problem Analyzing financial text data for insights and predictions.
method Integrates BERTopic with FinTextSim for topic modeling and clustering.
result BERTopic performs better with FinTextSim's embeddings, improving topic clarity and reducing misclassification.
Quality-designed consumer products are easy to recognize. Wouldn't it be great if the quality of financial products became just as apparent? This paper is addressed to financial practitioners. It provides an informal introduction to Quantitative Structuring -- a technology of manufacturing quality financial products (i…
Supervisory signals have the potential to make low-dimensional data representations, like those learned by mixture and topic models, more interpretable and useful. We propose a framework for training latent variable models that explicitly balances two goals: recovery of faithful generative explanations of high-dimensio…
Generative models predict page quality without training, useful for low-resource settings.
problem Detecting low-quality content in web articles.
method Human evaluation and analysis of 500 million web articles.
result Generative models can predict page quality without training, useful for low-resource settings.
Modeling lead-lag relationship between two text corpora for improved topic modeling.
problem Recognizing the relationship between multiple text corpora for better topic modeling.
method Proposed a jointly dynamic topic model and embedding extension for large-scale text corpus.
result The proposed model can well recognize the lead-lag relationship between two text corpora and improve topic learning.
We present a new algorithm for identifying the transition and emission probabilities of a hidden Markov model (HMM) from the emitted data. Expectation-maximization becomes computationally prohibitive for long observation records, which are often required for identification. The new algorithm is particularly suitable fo…
Topic models such as Latent Dirichlet Allocation (LDA) have been widely used in information retrieval for tasks ranging from smoothing and feedback methods to tools for exploratory search and discovery. However, classical methods for inferring topic models do not scale up to the massive size of today's publicly availab…
Paper introduces new indicators for forecasting crude oil prices using short news headlines.
problem Forecasting crude oil prices from short, noisy news headlines using LDA.
method Developed two novel indicators for topic and sentiment from short text data, and applied AdaBoost.RT.
result AdaBoost.RT with the proposed indicators outperforms benchmarks in crude oil forecasting.
Automatically assesses the quality of online health articles.
problem Lack of automated tools to evaluate the quality of online health information.
method Data mining approach using 10 quality criteria and feature selection.
result Classifier achieved 84%-90% accuracy on 10 criteria.
Improved topic modeling captures temporal relationships in speech.
problem Lack of temporal information in LDA for speech analysis.
method Temporal Markov chain extension to LDA for acoustic unit discovery.
result Improved phone segmentation results compared to base LDA.
Mixture models and topic models generate each observation from a single cluster, but standard variational posteriors for each observation assign positive probability to all possible clusters. This requires dense storage and runtime costs that scale with the total number of clusters, even though typically only a few clu…
CPGBN analyzes text sequences, capturing word order for better topic extraction.
problem Lack of word order in traditional text representations.
method CPFA processes words as sequences, CPGBN adds hierarchical topic modeling.
result CPGBN extracts high-quality latent representations capturing word order.
Improved spectral methods of moments for robust latent variable model learning.
problem Limited robustness of spectral methods of moments to model misspecification.
method Hierarchical approach using approximate joint diagonalization instead of tensor decomposition.
result Our method outperforms previous tensor decomposition methods in speed and model quality.
Quality-Diversity algorithms explore multiple high-performing solutions in a search space.
problem Finding multiple high-performing solutions in complex optimization problems.
method Evolutionary computation approach focusing on behavioral space and holistic solution distribution.
result Quality-Diversity algorithms provide a comprehensive view of high-performing solutions in a search space.
LR-Robot automates SLRs with AI, expert oversight, and multidimensional analysis.
problem Efficient but contextually limited outputs from existing SLR frameworks.
method Human-in-the-loop process, structured knowledge sources, retrieval-augmented generation.
result Empirical demonstration of AI-driven literature synthesis in option pricing.
The paper proposes a new method to evaluate LLM agent responses using ECDF clustering.
problem The standard evaluation of LLM agent responses via majority voting obscures response quality and distribution.
method The paper introduces a novel evaluation framework based on ECDF of cosine similarities and clustering of ECDFs using distances and k-medoids algorithm. result ECDF clustering reveals interpretable group structures in LLM responses, offering insights into agent settings.
Survey on random features for kernel approximation, focusing on algorithms, theory, and practical applications.
problem Efficiently approximating kernel methods for large-scale problems.
method Random features techniques to speed up kernel methods.
result Need for a high number of random features for good approximation quality.
Develops platforms to analyze social media data for human behavior and emotions.
problem Understanding human behavior and emotions from social media data.
method Self-structuring incremental machine learning, event detection, natural language processing.
result Captured salient topics and events from social media data, validated against news.
Low-cost sensors improve air quality prediction accuracy significantly.
problem Improving air quality monitoring networks with affordable sensors.
method Developed a high-resolution air quality prediction engine using low-cost sensors and official data.
result The use of low-cost sensors improves prediction accuracy by 25% and 15% for PM2.5 and PM10 respectively in densely monitored areas.
Emergency Department (ED) crowding is a worldwide issue that affects the efficiency of hospital management and the quality of patient care. This occurs when the request for an admit ward-bed to receive a patient is delayed until an admission decision is made by a doctor. To reduce the overcrowding and waiting time of E…
DAISYnt evaluates synthetic data quality and privacy in regulated domains.
problem Balancing data quality and privacy in regulated domains.
method Developed a suite of advanced tests (DAISYnt) to evaluate synthetic data quality and privacy.
result DAISYnt sets a de facto standard for synthetic data evaluation in regulated domains.
We develop a unified and systematic framework for performing online nonnegative matrix factorization under a wide variety of important divergences. The online nature of our algorithm makes it particularly amenable to large-scale data. We prove that the sequence of learned dictionaries converges almost surely to the set…
Novel Orlicz regrets consistently bound environmental variable statistics.
problem Consistent evaluation of stochastic environmental variables like water quality indices.
method Proposed novel Orlicz regrets for upper and lower bounds.
result Explicit linkage between Orlicz regrets and divergence risk measures.
Proposes a new multi-layer model for topic distributions.
problem Leveraging deep structures for learning word distributions of topics.
method A multi-layer generative process on word distributions of topics, where each topic is drawn from a mixture of topics from the layer above.
result Discover interpretable topic hierarchies and improve topic models' accuracy and interpretability.
Topic-aware chatbot learns from NMF topic vectors.
problem Improving chatbot relevance based on user topics.
method Combines RNN with NMF for topic learning and attention.
result Chatbot provides more relevant answers based on topic.
Introduces TT-NF for more compact neural field representations.
problem Finding more compact and easy-to-fit neural field representations.
method Tensor Train parameterization trained with backpropagation.
result Low-rank compression improves downstream task quality metrics.
A new model CDTM improves text classification by concentrating document topics.
problem Unsupervised text classification with diverse topic distributions.
method Imposes an exponential entropy penalty on document topic distribution to encourage concentration.
result More coherent topics and concentrated, sparse document-topic distributions.
Proposes HMHP for joint modeling of user-topic interactions.
problem Complex interactions between users, topics and time on social media.
method Hidden Markov Hawkes Process (HMHP) incorporating topical Markov Chains.
result HMHP outperforms state-of-the-art models in generalization and accuracy.
Document clustering and topic modeling are two closely related tasks which can mutually benefit each other. Topic modeling can project documents into a topic space which facilitates effective document clustering. Cluster labels discovered by document clustering can be incorporated into topic models to extract local top…