A new Bayesian multinomial regression model using permuted and augmented stick-breaking.
problem Modeling categorical response variables given covariates.
method Permuted and augmented stick-breaking (paSB) construction.
result Transforms multinomial regression into regression of stick-specific binary variables.
This article proposes Multinomial Probit Bayesian Additive Regression Trees (MPBART) as a multinomial probit extension of BART - Bayesian Additive Regression Trees (Chipman et al (2010)). MPBART is flexible to allow inclusion of predictors that describe the observed units as well as the available choice alternatives. T…
Paper develops Bayesian inference for discrete-choice mnp models with Gaussian priors.
problem Estimating parameters of discrete-choice multinomial probit models with Gaussian priors.
method Adapts Fasano and Durante's results to a specific mnp model with zero mean and independent Gaussian priors, simplifying posterior distribution parameters and providing a new variational algorithm.
result Simplified expressions for posterior distribution parameters and a novel variational algorithm.
NPMR uses nuclear norm penalty for multinomial regression, predicting baseball outcomes.
problem Predicting at bat outcomes in baseball with improved accuracy.
method Nuclear penalized multinomial regression (NPMR) applied to MLB data.
result NPMR provides better prediction probabilities for batter-pitcher matchups.
PIANO speeds up multinomial logistic regression solving.
problem Handling large datasets and many classes in logistic regression.
method Parallel iterative algorithm based on Majorization Minimization.
result PIANO converges to a stationary point of Multinomial and Sparse Multinomial Logistic Regression.
New algorithms improve MPBART for HIV patient data.
problem Improving inference for multinomial outcomes in HIV patient data.
method Introduced two new algorithms for fitting MPBART.
result Better performance in MCMC convergence and predictive accuracy.
Combines response categories in multinomial logistic regression models.
problem Handling multiple response categories in logistic regression models.
method Penalized likelihood method with alternating direction method of multipliers.
result Encourages response category combination in the model.
The paper proposes an efficient method to scale Bayesian inference for mixed multinomial logit models to very large datasets.
problem Efficiency in Bayesian inference for mixed multinomial logit models on large datasets.
method Amortized Variational Inference with stochastic backpropagation, automatic differentiation, and GPU acceleration.
result The proposed method achieves significant computational speedups over traditional methods for large datasets.
We speed up cross-validation in multinomial logistic regression with an ℓ1-regularization formula.
problem Slow cross-validation in multinomial logistic regression with ℓ1-regularization. method Perturbative approach using large data size and model dimensionality.
result Significant reduction in computational time for cross-validation.
Proved a combinatorial conjecture in machine learning.
problem None explicitly stated in the abstract.
method Binomial and multinomial sums identities.
result Proved a combinatorial conjecture.
Sparse multinomial logistic regression for multiclass classification with feature selection.
problem High-dimensional multiclass classification with a focus on sparse models.
method Penalized maximum likelihood with complexity penalty, feature selection using group Lasso and Slope classifiers.
result Achievement of minimax order in both small and large number of classes regimes.
Proposes a new model for context-dependent decision-making.
problem Constant preference parameters in decision models are too rigid.
method Introduces Context-aware Bayesian mixed multinomial logit model using neural networks.
result Models context-dependent intra-respondent heterogeneity effectively.
The paper proves asymptotic normality for multinomial logistic regression on null covariates.
problem Classical asymptotic normality results fail in high-dimensional multinomial logistic models.
method Developed asymptotic normality and chi-square results for multinomial logistic MLE on null covariates.
result Validated new methodology to test feature significance in high-dimensional classification problems.
The abstract reviews models for analyzing count data.
problem Challenges in analyzing count data with standard methods.
method Review of generalized linear models and multinomial models.
result Fundamental connections between multinomial and count models.
Develops Bayesian inference methods for gamma models.
problem Challenges in inference for models with gamma functions.
method Data augmentation scheme using Exponential Reciprocal Gamma distributions.
result Scalable EM and MCMC algorithms developed.
In this paper we purpose a blockwise descent algorithm for group-penalized multiresponse regression. Using a quasi-newton framework we extend this to group-penalized multinomial regression. We give a publicly available implementation for these in R, and compare the speed of this algorithm to a competing algorithm --- w…
New conjugate priors improve Bayesian inference for multinomial probit models.
problem Lack of tractable conjugate priors for efficient Bayesian inference in multinomial probit models.
method Unified skew-normal (SUN) distributions as conjugate priors, leading to improved posterior inference and classification.
result Improved computational methods for posterior inference and classification, especially in high dimensions.
VAEs improve collaborative filtering for implicit feedback.
problem Limited modeling capacity of linear factor models in collaborative filtering.
method Introduced a generative model with multinomial likelihood and used Bayesian inference for parameter estimation.
result Significantly outperforms state-of-the-art baselines on real-world datasets.
ADMM-Softmax improves classification accuracy for multiclass problems.
problem Multinomial logistic regression for classification tasks with many examples and features.
method Alternating direction method of multipliers (ADMM) for decoupling and solving the problem into efficient steps.
result ADMM-Softmax leads to improved generalization compared to other methods on two image classification problems.
Bayesian models predict Collatz stopping times with high accuracy.
problem Predicting the total stopping time of Collatz sequences.
method Developed two complementary models: a hierarchical Negative Binomial regression and a mechanistic generative approximation.
result Bayesian models outperform generative approximations in predicting Collatz stopping times.
Proposes PG-DA for Bayesian MMNL estimation to handle non-conjugacy.
problem Non-conjugacy in the Bayesian estimation of MMNL models.
method Pólygamma data augmentation technique applied to MMNL estimation.
result Similar posterior estimates for binary choice scenarios, but empirical identification issues for J≥3 alternatives. Deep generative model for healthcare data identifies coherent substructures and mutational clusters.
problem Analytical challenges in healthcare data, including sparsity, missingness, and small sample sizes.
method Proposes a deep generative Bayesian model with collapsed Gibbs sampling for multinomial count data.
result Identifies coherent substructures and biologically meaningful mutational clusters in cancer data.
Proposes a method to use external machine-learning predictions in multinomial logistic regression.
problem Improving statistical inference using summary-level external machine-learning predictions.
method Empirical-likelihood framework incorporating moment constraints from external nonparametric machine-learning predictions.
result Fused estimator achieves strict efficiency gain over primary-only estimator under mild conditions.
Many practical modeling problems involve discrete data that are best represented as draws from multinomial or categorical distributions. For example, nucleotides in a DNA sequence, children's names in a given state and year, and text documents are all commonly modeled with multinomial distributions. In all of these cas…
The paper studies causal effects of multiple treatments in healthcare databases with rare outcomes.
problem Estimating causal effects of multiple treatments in healthcare databases with rare outcomes.
method The paper designs three sets of simulations and compares the operating characteristics of three types of methods: Bayesian Additive Regression Trees (BART), regression adjustment on multivariate spline of generalized propensity scores (RAMS), and inverse probability of treatment weighting (IPTW) with multinomial logistic regression or generalized boosted models.
result BART and RAMS provide lower bias and mean squared error compared to IPTW methods.
Bayesian model predicts fashion size recommendations and returns.
problem Predicting optimal fashion sizes and handling returns efficiently.
method Hierarchical Bayesian model for size and return events.
result Model incorporates domain expertise and article characteristics.
We propose a new data-augmentation strategy for fully Bayesian inference in models with binomial likelihoods. The approach appeals to a new class of Polya-Gamma distributions, which are constructed in detail. A variety of examples are presented to show the versatility of the method, including logistic regression, negat…
We improve random forest consistency and performance with DMRF, a new variant.
problem Improving the consistency and performance of random forest models.
method Developed DMRF, a data-driven multinomial random forest, by modifying proof methods and improving data utilization.
result DMRF achieves strong consistency with probability 1, surpassing previous models in classification tasks.
Scaling multinomial logistic regression to datasets with very large number of data points and classes is challenging. This is primarily because one needs to compute the log-partition function on every data point. This makes distributing the computation hard. In this paper, we present a distributed stochastic gradient d…
Enhances random forest consistency and introduces DMRF for improved performance.
problem Improving the consistency and efficiency of random forest algorithms.
method Strengthened proof methods and propose DMRF algorithm.
result DMRF achieves better theoretical and experimental performance than previous variants.
Develops unbiased variational inference method for better model performance.
problem Improving variational inference methods for better model performance.
method Defines an expressive variational family using a simple reparameterizable distribution and deep neural networks, directly optimizing the ELBO.
result Achieves tighter ELBO and better predictive performance than existing approaches at similar computational cost.
Improves change-point detection for high-dimensional time-series.
problem Uncertainty in latent variable estimation affects change-point detection.
method Proposes multinomial sampling to improve detection rate and reduce delay.
result Results outperform baseline method in experiments.
Approximates large Random Forest models to save space.
problem Large model size limits deployment on small devices.
method Approximate each tree using multinomial logistic regression.
result Reduced model size without significant loss of accuracy.
New method for epidemic model inference using multinomial approximations.
problem Inference in stochastic epidemic models with partial observations.
method Recursive multinomial approximations to integrate over unobserved variables.
result Accuracy demonstrated through real and simulated data.
Unbiased method for Bayesian posterior means using kinetic Langevin dynamics.
problem Estimating Bayesian posterior means efficiently and accurately.
method Combines advanced splitting methods with enhanced gradient approximations in a multilevel Monte Carlo approach.
result The method achieves unbiased estimates with finite variance and central limit theorem properties.
Paper introduces MPES for top-k ranking BO with preferential observations.
problem Handling top-k ranking and tie/indifference observations in Bayesian optimization.
method Designs a surrogate model and introduces MPES acquisition function.
result MPES outperforms existing acquisition functions in handling preferential observations.
For the problem of multi-class linear classification and feature selection, we propose approximate message passing approaches to sparse multinomial logistic regression (MLR). First, we propose two algorithms based on the Hybrid Generalized Approximate Message Passing (HyGAMP) framework: one finds the maximum a posterio…
The study analyzes when Bayesian averaging over decision trees is reliable.
problem When do Bayesian model averaging weights over decision trees provide reliable information?
method Closed-form solution for Bayesian decision trees with Catalan-exponential priors.
result Established a complete non-asymptotic theory of rational commitment thresholds.
Fenrir efficiently estimates Bayesian MLN-DLMs for scalable inference.
problem Computational challenges in Bayesian MLN-DLMs for longitudinal count compositional data.
method Novel algorithm for MAP estimation and accurate posterior marginal approximation.
result Fenrir can be three orders of magnitude more efficient than Stan.
The paper establishes convergence rates for MoE models in classification problems.
problem Understanding the behavior of MoE models in classification settings.
method Established convergence rates for density and parameter estimation in softmax gating multinomial logistic MoE models.
result Parameter estimation rates are significantly improved with a novel modified softmax gating function.
We describe algorithms for learning Bayesian networks from a combination of user knowledge and statistical data. The algorithms have two components: a scoring metric and a search procedure. The scoring metric takes a network structure, statistical data, and a user's prior knowledge, and returns a score proportional to …
The paper provides guarantees for a tangent transform algorithm in logistic regression models.
problem Finding theoretical guarantees for statistical optimality and algorithmic convergence in non-conjugate models.
method Exploiting convex duality and minorizing the marginal likelihood, the paper derives non-asymptotic upper bounds and convergence guarantees for a tangent transform algorithm in logistic regression models.
result The tangent transform algorithm is shown to be locally asymptotically stable without assumptions on the data-generating process.
The paper identifies universal features for high-dimensional data inference.
problem Identifying universal low-dimensional features from high-dimensional data for inference tasks.
method Introduces natural notions of universality and shows a local equivalence among them, using information geometry.
result Reveals the complementary roles of various data analysis techniques.
VB methods improve MMNL estimation speed and accuracy.
problem Scalable Bayesian estimation of MMNL models.
method Extending VB methods to include both fixed and random utility parameters, and conducting extensive simulations.
result VB methods, especially VB-NCVMP-Delta, are up to 16 times faster than MCMC and MSLE while maintaining similar accuracy.
FisherSFT selects informative examples to fine-tune LLMs efficiently.
problem Adapting large language models to new domains efficiently.
method Selects examples maximizing information gain using Hessian of log-likelihood.
result Empirically demonstrates improved performance with reduced computational cost.
We improve MoE models for classification with rigorous guarantees and practical methods.
problem Limited guarantees for stable maximum-likelihood training and model selection in softmax-gated MoE models.
method Derived a batch MM algorithm with closed-form updates, proved finite-sample rates, and developed a dendrogram selector.
result Achieved near-parametric optimal rates for parameter recovery and improved accuracy over baselines.
Bayesian model improves BCI performance for ALS users.
problem Classifying EEG signals for P300 BCIs with low SNR and complex correlations.
method GLASS model with Gaussian Latent channel and Sparse time-varying effects.
result GLASS substantially improves BCI performance in ALS users.
Efficiently models categorical data with low to medium class overlap, improving accuracy over standard distributions.
problem Poor parameter estimates and accuracy in multinomial and Dirichlet multinomial distributions when assumptions are violated.
method Introduces Beta-Liouville multinomial distribution and efficient estimation methods.
result Beta-Liouville multinomial outperforms standard distributions on two out of four datasets.