A new method for scalable inference in deep discrete LVMs.
problem Challenges in scalable inference for deep discrete latent variable models.
method Topic-layer-adaptive stochastic gradient Riemannian MCMC (TLASGR) for DLDA.
result State-of-the-art results on big data sets.
Enhances topic models to better handle polysemous words.
problem Lack of polysemy handling in Gaussian latent Dirichlet allocation.
method Introduces a hierarchical structure to capture polysemy in Gaussian latent Dirichlet allocation.
result Significantly improves polysemy detection and provides more parsimonious topic representations.
Logistic LDA applies LDA to non-categorical data.
problem Applying LDA to non-categorical data is challenging.
method Logistic LDA, a discriminative variant of LDA.
result Logistic LDA can learn from unlabeled data.
LDTA expands LDA's topic modeling capacity with tree-structured priors.
problem Limited expressiveness of Dirichlet priors in LDA for complex topic relationships.
method Introduces Latent Dirichlet-Tree Allocation (LDTA) with Dirichlet-Tree (DT) priors, and develops universal mean-field variational inference and Expectation Propagation.
result LDTA enables expressive, tree-structured priors over topic proportions, expanding modeling capacity of LDA.
A new GAN model LDAGAN uses Latent Dirichlet Allocation to model multimodal images.
problem Ignoring the structure and multimodal characteristics of vision data in GANs.
method Introduced a Dirichlet prior for multimodal image generation leading to LDAGAN. LDAGAN defines generative modes for each sample and uses a VEM algorithm for adversarial training.
result Experimental results show LDAGAN outperforms other GANs on real-world datasets.
NMF with specific constraints is equivalent to LDA.
problem Dimensionality reduction of non-negative data.
method NMF with ℓ1 normalization constraints and Dirichlet prior. result NMF with these constraints is equivalent to LDA.
Paper calculates the exact error of LDA models.
problem Bayesian generalization error in Latent Dirichlet Allocation (LDA).
method Theoretical analysis of learning coefficient using algebraic geometry.
result Exact asymptotic form of LDA's generalization error.
Improves efficiency of LDA model inference.
problem Efficiency of collapsed Gibbs sampling for LDA models.
method Introduces blocking scheme to improve chain mixing efficiency.
result Significant improvements in chain mixing compared to single-site sampler.
Derives VMP for LDA, simplifying inference for topic modeling.
problem Manual derivation of VMP equations for LDA is challenging and time-consuming.
method Detailed derivation of VMP update equations for LDA.
result Enables easier implementation of VMP for LDA models.
Decentralized LDA for privacy preserving topic modeling.
problem Learning topics from decentralized networks without sharing sensitive information.
method Adapted LDA model for decentralized optimization.
result Similar topic parameters and performance at each node as with stochastic methods.
CoNN uses cooperative neural networks to leverage prior independence structure for improved text classification.
problem Improving text classification accuracy by exploiting prior independence structure.
method CoNN employs a set of cooperatively trained neural networks to capture latent representations based on prior independence structure.
result Demonstrated a 23% reduction in error on the MultiSent dataset compared to state-of-the-art methods.
A new algorithm improves topic model accuracy in small data sets.
problem Limited data leads to inaccurate topic model identification.
method Developed a novel variational message passing algorithm (ALBU) for LDA.
result ALBU learns latent distributions more accurately than Variational Bayes (VB) in small data sets.
Paper tackles privacy issues in LDA model training.
problem Privacy leakage in LDA model training.
method Developed privacy monitoring and locally private LDA training algorithms.
result Demonstrated effectiveness of locally private LDA training on crowdsourced data.
PM-LDA models allow partial topic memberships for image patches.
problem Crisp segmentation limits topic models to one topic per image patch.
method Developed PM-LDA for partial topic memberships and an estimation algorithm.
result PM-LDA can produce both crisp and soft segmentations.
New algorithm for latent variable models using spectral decomposition.
problem Unsupervised learning of latent variable models from unlabeled data.
method Spectral decomposition for robust unsupervised learning.
result Efficient technique to learn parameters of text mining models.
Geometric Dirichlet Means algorithm improves topic inference efficiency.
problem Improving topic inference in Latent Dirichlet Allocation models.
method Optimization of a geometric loss function for weighted clustering with geometric corrections.
result Achieves comparable accuracy to Gibbs sampling but with computational efficiency.
nnLDA combines neural and probabilistic methods for better topic modeling with side information.
problem Lack of integration of auxiliary information in traditional topic models.
method nnLDA integrates side information through a neural prior mechanism, optimizing both neural and probabilistic components.
result nnLDA outperforms traditional models in topic coherence, perplexity, and classification.
The paper proves variational Bayes methods are statistically optimal under certain conditions.
problem Justification of variational Bayes methods for parameter estimation.
method General conditions for optimal risk bounds in mean-field variational Bayesian inference.
result Optimal risk bounds for variational Bayes estimates are achievable under specific conditions.
Subset LLDA improves scalability for large label sets in multi-label classification.
problem Scalability issues in Labeled Latent Dirichlet Allocation (LLDA) for large label sets.
method Subset LLDA, a simple variant of LLDA, addressing scalability issues.
result Subset LLDA outperforms LLDA and extreme multi-label classification algorithms on large label sets.
We introduce incremental variational inference and apply it to latent Dirichlet allocation (LDA). Incremental variational inference is inspired by incremental EM and provides an alternative to stochastic variational inference. Incremental LDA can process massive document collections, does not require to set a learning …
New model identifies microbial subcommunities robustly, accounting for cross-sample heterogeneity.
problem Inference in LDA is sensitive to the number of subcommunities and often creates artificial ones.
method Incorporates logistic-tree normal (LTN) model into LDA to account for cross-sample heterogeneity.
result Restores robustness of inference and identifies meaningful subcommunities.
Study shows LDA topic models converge at rate n^-1/4 without strict topic separability.
problem Convergence rates of Latent Dirichlet Allocation (LDA) topic models.
method Maximum likelihood estimator, Wasserstein's distance metric, without separability or non-degeneracy assumptions.
result Maximum likelihood estimator converges at rate n^-1/4, optimal in worst case.
Study finds common poetic themes across languages over time.
problem Understanding thematic evolution in different poetic traditions.
method Applied Latent Dirichlet Allocation (LDA) to poetry corpora of four languages.
result Identified common themes and their temporal trends across poetic traditions.
Applying traditional collaborative filtering to digital publishing is challenging because user data is very sparse due to the high volume of documents relative to the number of users. Content based approaches, on the other hand, is attractive because textual content is often very informative. In this paper we describe …
Topic models (e.g., pLSA, LDA, SLDA) have been widely used for segmenting imagery. These models are confined to crisp segmentation. Yet, there are many images in which some regions cannot be assigned a crisp label (e.g., transition regions between a foggy sky and the ground or between sand and water at a beach). In the…
Bayesian inference for topics in documents with many potential causes.
problem Estimating topic distributions in documents with many potential causes and few observations.
method Exact Bayesian inference using a linear-time algorithm with a simple formula.
result Exact Bayesian inference can be computed in linear time for a given upper bound on observations.
Improved topic modeling captures temporal relationships in speech.
problem Lack of temporal information in LDA for speech analysis.
method Temporal Markov chain extension to LDA for acoustic unit discovery.
result Improved phone segmentation results compared to base LDA.
ALBU improves LDA performance on small datasets.
problem Limited data impairs variational Bayes in LDA.
method Approximate Loopy Belief message passing algorithm.
result ALBU learns latent distributions more accurately than VB, especially for smaller data sets.
Model estimates corporate credibility using NLP and neural networks.
problem Estimating corporate credibility in Chinese listed companies.
method Latent Dirichlet Allocation + Residual Convolutional Neural Network.
result Model ranks companies based on transparency.
A new sampler speeds up LDA topic modeling for big data.
problem Training LDA on large corpora is slow and requires dense memory storage.
method Uses a Pólya-urn-based approximation in a sparse partially collapsed sampler.
result The new sampler is faster and asymptotically exact.
In latent Dirichlet allocation (LDA), topics are multinomial distributions over the entire vocabulary. However, the vocabulary usually contains many words that are not relevant in forming the topics. We adopt a variable selection method widely used in statistical modeling as a dimension reduction tool and combine it wi…
Paper shows LDA and SMF have similar generalization errors.
problem LDA and SMF's generalization performance is unknown.
method Algebraic and geometric method to show equivalence of LDA and SMF.
result LDA and SMF have asymptotically same Bayesian generalization error.
LDA identifies latent topics in CFPB consumer complaints over time.
problem Identify latent topics in CFPB consumer complaints for better regulation effectiveness.
method Latent Dirichlet Allocation (LDA) for topic modeling of consumer complaints.
result Time trends of latent topics reveal regulatory effectiveness and consumer protection issues.
Feature extraction has gained increasing attention in the field of machine learning, as in order to detect patterns, extract information, or predict future observations from big data, the urge of informative features is crucial. The process of extracting features is highly linked to dimensionality reduction as it impli…
CLDA improves topic modeling for large, dynamic datasets.
problem Dynamic topic modeling for large, diverse text streams.
method Data decomposition followed by topic modeling on segments, then clustering.
result Very fast runtime and insight into topic composition over time.
Bayesian nonparametric model learns new categories without predefined limits.
problem Learning new categories unseen in labeled training data.
method Hierarchical Dirichlet process and latent Dirichlet allocation for automatic category inference.
result Nonparametric approach yields comparable performance to parametric methods with pre-specified new categories.
The paper uses WeChat data to map urban cultural resource needs.
problem Optimizing cultural resource allocation in cities.
method Data-driven framework using WeChat user check-ins and temporal LDA model.
result Identifies urban regions lacking cultural resources.
The paper analyzes the mixing time of a simple LDA model.
problem Analyzing the mixing time of a simple LDA model.
method Study of the Gibbs sampler for the posterior distribution of LDA.
result The mixing time is at most of order m2logm. Proposes a method to evaluate meta-learning performance based on task similarity.
problem Meta-learning performance evaluation ignores task similarity, leading to biased results.
method Generative approach using Latent Dirichlet Allocation to analyze task similarity.
result The proposed method provides an insightful evaluation of meta-learning algorithms, matching common intuition.
Paper presents a reparameterized DP-DLGMM for clustering.
problem Non-parametric DP priors in DLGMM are hard to couple with variational inference.
method Closed-form updates for DP-DLGMM's variational posterior.
result Model generates realistic samples and performs competitively in semi-supervised settings.
Supervised topic models improve clinical diagnostics by interpreting cooccurence patterns in count data.
problem Standard supervised Latent Dirichlet Allocation (sLDA) struggles with documents having many more words than labels and lacks effective use of supervised labels.
method Investigates penalized optimization methods to train interpretable sLDA models using recognition networks for faster inference.
result Preliminary results show improved predictions on heldout data for predicting anti-depressant medication success based on patient history.
Hyper-parameters play a major role in the learning and inference process of latent Dirichlet allocation (LDA). In order to begin the LDA latent variables learning process, these hyper-parameters values need to be pre-determined. We propose an extension for LDA that we call 'Latent Dirichlet allocation Gibbs Newton' (LD…
A new method improves topic modeling accuracy using semantic filtering.
problem Improving topic modeling accuracy in text documents.
method Three-step process: generate word/word-pair, apply TF-IDF, merge similar semantic pairs.
result Improves topic accuracy by up to 12.99% compared to state-of-the-art models.
Latent Dirichlet Allocation models discrete data as a mixture of discrete distributions, using Dirichlet beliefs over the mixture weights. We study a variation of this concept, in which the documents' mixture weight beliefs are replaced with squashed Gaussian distributions. This allows documents to be associated with e…
Proposes a new model to predict antidepressants from EHRs, improving accuracy and interpretability.
problem Improving prediction accuracy and interpretability in topic models for clinical data.
method Prediction-constrained latent Dirichlet allocation framework balancing generative likelihood and prediction accuracy.
result Improved prediction of depression medications from EHRs compared to previous methods.
The paper proposes a new recommender system combining ratings and textual reviews.
problem Lack of using textual reviews in recommender systems.
method Combines Latent Factor Model with Latent Dirichlet Allocation for textual reviews.
result Combining textual reviews with ratings improves recommendation quality.
Improved Gibbs sampling yields higher likelihood solutions.
problem Gibbs sampling often returns suboptimal solutions due to bottlenecks.
method Interdependent Gibbs Samplers combining multiple samplers with coupling.
result High likelihood solutions significantly more often than regular Gibbs sampler.
Understanding the adaptation process of plants to drought stress is essential in improving management practices, breeding strategies as well as engineering viable crops for a sustainable agriculture in the coming decades. Hyper-spectral imaging provides a particularly promising approach to gain such understanding since…