arXiv research

The paper tackles learning mixtures of two multinomial logits, showing identifiability and presenting an algorithm.

Paper introduces a new text clustering model using Beta-Liouville priors.

New Gamma-Poisson model improves topic selection for short text.

Evolutionary clustering aims at capturing the temporal evolution of clusters. This issue is particularly important in the context of social media data that are naturally temporally driven. In this paper, we propose a new probabilistic model-based evolutionary clustering technique. The Temporal Multinomial Mixture (TMM)…

Researchers show mixtures of ranking models are generally identifiable.

We improve MoE models for classification with rigorous guarantees and practical methods.

The paper establishes convergence rates for MoE models in classification problems.

Proposes a new model for clustering passenger trajectories with graphs.

New bounds on sample size for identifying mixture models with grouped samples.

Image of an entity can be defined as a structured and dynamic representation which can be extracted from the opinions of a group of users or population. Automatic extraction of such an image has certain importance in political science and sociology related studies, e.g., when an extended inquiry from large-scale data i…

The traditional Minkowski distances are induced by the corresponding Minkowski norms in real-valued vector spaces. In this work, we propose novel statistical symmetric distances based on the Minkowski's inequality for probability densities belonging to Lebesgue spaces. These statistical Minkowski distances admit closed…

Motivated by generating personalized recommendations using ordinal (or preference) data, we study the question of learning a mixture of MultiNomial Logit (MNL) model, a parameterized class of distributions over permutations, from partial ordinal or preference data (e.g. pair-wise comparisons). Despite its long standing…

In this technical report, we present jLDADMM---an easy-to-use Java toolkit for conventional topic models. jLDADMM is released to provide alternatives for topic modeling on normal or short texts. It provides implementations of the Latent Dirichlet Allocation topic model and the one-topic-per-document Dirichlet Multinomi…

Improved regret bound for MNL MDPs with variance-aware approach.

When estimating finite mixture models, it is common to make assumptions on the mixture components, such as parametric assumptions. In this work, we make no distributional assumptions on the mixture components and instead assume that observations from the mixture model are grouped, such that observations in the same gro…

Develops a functional mix-of-experts model for multiclass classification.

Efficiently models categorical data with low to medium class overlap, improving accuracy over standard distributions.

We introduce the author-topic model, a generative model for documents that extends Latent Dirichlet Allocation (LDA; Blei, Ng, & Jordan, 2003) to include authorship information. Each author is associated with a multinomial distribution over topics and each topic is associated with a multinomial distribution over words.…

We present a mixed multinomial logit (MNL) model, which leverages the truncated stick-breaking process representation of the Dirichlet process as a flexible nonparametric mixing distribution. The proposed model is a Dirichlet process mixture model and accommodates discrete representations of heterogeneity, like a laten…

Improved logistic MoE with sigmoid gate shows better sample efficiency.

MM (majorization--minimization) algorithms are an increasingly popular tool for solving optimization problems in machine learning and statistical estimation. This article introduces the MM algorithm framework in general and via three popular example applications: Gaussian mixture regressions, multinomial logistic regre…

Bayesian framework for analyzing heterogeneous covariance data with a novel MoE-Wishart model.

A method for identifying NPWARX models with arbitrary domains using probabilistic mixture models.

Mixtures of Unigrams are one of the simplest and most efficient tools for clustering textual data, as they assume that documents related to the same topic have similar distributions of terms, naturally described by Multinomials. When the classification task is particularly challenging, such as when the document-term ma…

PIANO speeds up multinomial logistic regression solving.

To model categorical response variables given their covariates, we propose a permuted and augmented stick-breaking (paSB) construction that one-to-one maps the observed categories to randomly permuted latent sticks. This new construction transforms multinomial regression into regression analysis of stick-specific binar…

This paper analyzes EM algorithm for softmax mixture models in high dimensions.

The vast majority of the neural network literature focuses on predicting point values for a given set of response variables, conditioned on a feature vector. In many cases we need to model the full joint conditional distribution over the response variables rather than simply making point predictions. In this paper, we …

Many practical modeling problems involve discrete data that are best represented as draws from multinomial or categorical distributions. For example, nucleotides in a DNA sequence, children's names in a given state and year, and text documents are all commonly modeled with multinomial distributions. In all of these cas…

New method tests risk measures for various distortions.

Paper introduces new methods for modeling categorical data.

We use the theory of normal variance-mean mixtures to derive a data augmentation scheme for models that include gamma functions. Our methodology applies to many situations in statistics and machine learning, including Multinomial-Dirichlet distributions, Negative binomial regression, Poisson-Gamma hierarchical models, …

Sparse multinomial logistic regression for multiclass classification with feature selection.

Proposes a new model for clustering passenger trips considering hierarchical and multi-dimensional data.

Proposes a new model for context-dependent decision-making.

Beam search improves UQ in LLMs by reducing duplicates and variance.

Count data take on non-negative integer values and are challenging to properly analyze using standard linear-Gaussian methods such as linear regression and principal components analysis. Generalized linear models enable direct modeling of counts in a regression context using distributions such as the Poisson and negati…

Efficient RL algorithm for multinomial logistic MDPs with provable guarantees.

Log-concavity proven for multinomial likelihoods under specific constraints.

New conjugate priors improve Bayesian inference for multinomial probit models.

We consider a problem of clustering a sequence of multinomial observations by way of a model selection criterion. We propose a form of a penalty term for the model selection procedure. Our approach subsumes both the conventional AIC and BIC criteria but also extends the conventional criteria in a way that it can be app…

The paper proposes an efficient method to scale Bayesian inference for mixed multinomial logit models to very large datasets.

Two algorithms achieve optimal regret with limited adaptivity in multinomial logistic bandits.

New method for epidemic model inference using multinomial approximations.

The paper proves asymptotic normality for multinomial logistic regression on null covariates.

Develops Bayesian inference methods for gamma models.

Deviance-style normalization for sparse, jointly overdispersed count matrices

We propose the nuclear norm penalty as an alternative to the ridge penalty for regularized multinomial regression. This convex relaxation of reduced-rank multinomial regression has the advantage of leveraging underlying structure among the response categories to make better predictions. We apply our method, nuclear pen…