Multinomial Logistic Regression is a well-studied tool for classification and has been widely used in fields like image processing, computer vision and, bioinformatics, to name a few. Under a supervised classification scenario, a Multinomial Logistic Regression model learns a weight vector to differentiate between any …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “multinomial logistic regression”
Sparse multinomial logistic regression for multiclass classification with feature selection.
problem High-dimensional multiclass classification with a focus on sparse models.
method Penalized maximum likelihood with complexity penalty, feature selection using group Lasso and Slope classifiers.
result Achievement of minimax order in both small and large number of classes regimes.
The paper proves asymptotic normality for multinomial logistic regression on null covariates.
problem Classical asymptotic normality results fail in high-dimensional multinomial logistic models.
method Developed asymptotic normality and chi-square results for multinomial logistic MLE on null covariates.
result Validated new methodology to test feature significance in high-dimensional classification problems.
We propose a penalized likelihood method that simultaneously fits the multinomial logistic regression model and combines subsets of the response categories. The penalty is non differentiable when pairs of columns in the optimization variable are equal. This encourages pairwise equality of these columns in the estimator…
Fused Multinomial Logistic Regression Utilizing Summary-Level External Machine-learning Informationstat.ME
Proposes a method to use external machine-learning predictions in multinomial logistic regression.
problem Improving statistical inference using summary-level external machine-learning predictions.
method Empirical-likelihood framework incorporating moment constraints from external nonparametric machine-learning predictions.
result Fused estimator achieves strict efficiency gain over primary-only estimator under mild conditions.
We develop an approximate formula for evaluating a cross-validation estimator of predictive likelihood for multinomial logistic regression regularized by an -norm. This allows us to avoid repeated optimizations required for literally conducting cross-validation; hence, the computational time can be significantl…
DS-MLR: Exploiting Double Separability for Scaling up Distributed Multinomial Logistic Regressioncs.LG
Scaling multinomial logistic regression to datasets with very large number of data points and classes is challenging. This is primarily because one needs to compute the log-partition function on every data point. This makes distributing the computation hard. In this paper, we present a distributed stochastic gradient d…
We improve MoE models for classification with rigorous guarantees and practical methods.
problem Limited guarantees for stable maximum-likelihood training and model selection in softmax-gated MoE models.
method Derived a batch MM algorithm with closed-form updates, proved finite-sample rates, and developed a dendrogram selector.
result Achieved near-parametric optimal rates for parameter recovery and improved accuracy over baselines.
For the problem of multi-class linear classification and feature selection, we propose approximate message passing approaches to sparse multinomial logistic regression (MLR). First, we propose two algorithms based on the Hybrid Generalized Approximate Message Passing (HyGAMP) framework: one finds the maximum a posterio…
The paper establishes convergence rates for MoE models in classification problems.
problem Understanding the behavior of MoE models in classification settings.
method Established convergence rates for density and parameter estimation in softmax gating multinomial logistic MoE models.
result Parameter estimation rates are significantly improved with a novel modified softmax gating function.
We present ADMM-Softmax, an alternating direction method of multipliers (ADMM) for solving multinomial logistic regression (MLR) problems. Our method is geared toward supervised classification tasks with many examples and features. It decouples the nonlinear optimization problem in MLR into three steps that can be solv…
New algorithm for maximizing revenue in multinomial logistic bandits.
problem Maximizing revenue in scenarios with multiple outcomes.
method MNL-UCB algorithm based on upper confidence bounds.
result Achieves regret with small dependency on constants.
Approximates large Random Forest models to save space.
problem Large model size limits deployment on small devices.
method Approximate each tree using multinomial logistic regression.
result Reduced model size without significant loss of accuracy.
FisherSFT selects informative examples to fine-tune LLMs efficiently.
problem Adapting large language models to new domains efficiently.
method Selects examples maximizing information gain using Hessian of log-likelihood.
result Empirically demonstrates improved performance with reduced computational cost.
Efficient RL algorithm for multinomial logistic MDPs with provable guarantees.
problem Model-based RL for episodic MDPs with unknown transition probabilities.
method Upper confidence bound-based algorithm for exploration-exploitation balance.
result Achieves regret bound for multinomial logistic models.
Logistic regression models are a popular and effective method to predict the probability of categorical response data. However inference for these models can become computationally prohibitive for large datasets. Here we adapt ideas from symbolic data analysis to summarise the collection of predictor variables into his…
Trans-GCR uses GCR model for node classification, providing theoretical guarantees and superior performance.
problem Challenges in obtaining node classification labels in real-world scenarios.
method Graph Convolutional Multinomial Logistic Regression (GCR) model and transfer learning method based on GCR.
result Trans-GCR provides superior empirical performance and theoretical guarantees.
Two algorithms achieve optimal regret with limited adaptivity in multinomial logistic bandits.
problem Achieving optimal regret with limited adaptivity in multinomial logistic bandits.
method Presented two algorithms, B-MNL-CB and RS-MNL, for batched and rarely-switching paradigms.
result Achieved regret with limited adaptivity.
MM (majorization--minimization) algorithms are an increasingly popular tool for solving optimization problems in machine learning and statistical estimation. This article introduces the MM algorithm framework in general and via three popular example applications: Gaussian mixture regressions, multinomial logistic regre…
Improved Regret Bounds of (Multinomial) Logistic Bandits via Regret-to-Confidence-Set Conversionstat.ML
Improved regret bounds for logistic bandits via novel confidence set construction.
problem Dependencies in parameter space for logistic bandits, especially when .
method Regret-to-confidence-set conversion (R2CS) to construct convex confidence sets.
result Strict improvement in regret bound w.r.t. in logistic bandits.
Model predicts individual insurance claim reserves using activation patterns.
problem Accurately predicting individual claim reserves in insurance contracts.
method Multinomial logistic regression to model claim activation and development.
result The model generates accurate predictions of total and per coverage reserves.
Improved regret bound for multinomial logistic bandits with non-linearity.
problem Maximizing rewards in multinomial logistic bandits with non-linear feedback.
method Extended the definition of to multinomial setting and proposed an efficient algorithm.
result Minimax-optimal regret bound of , improving over existing guarantees.
A new algorithm reduces the time and space complexity for multinomial logistic bandits.
problem High-dimensional feedback in multinomial logistic bandits makes existing algorithms inefficient.
method Integrates frequent directions matrix sketching into OFUL-MLogB to reduce time and space complexity.
result Achieves a regret bound of .
Enhances neural networks in hyperbolic space for better data structure capture.
problem Capturing hierarchical data structures efficiently.
method Unified hyperbolic model for neural network components.
result Superior parameter efficiency and outperformance over Euclidean methods.
Researchers find a class/cross-class structure in deep learning spectra.
problem Understanding the structure of deep learning classifiers.
method Empirical spectral analysis of deepnet classifiers.
result The cross-class structure explains various features in deepnet spectra.
The paper provides guarantees for a tangent transform algorithm in logistic regression models.
problem Finding theoretical guarantees for statistical optimality and algorithmic convergence in non-conjugate models.
method Exploiting convex duality and minorizing the marginal likelihood, the paper derives non-asymptotic upper bounds and convergence guarantees for a tangent transform algorithm in logistic regression models.
result The tangent transform algorithm is shown to be locally asymptotically stable without assumptions on the data-generating process.
New method for multiclass classification reduces error bounds.
problem High-dimensional multiclass classification with sparse coefficients.
method Sparse multinomial logistic regression with convex penalties.
result Plug-in classifiers achieve minimax generalization error bounds.
Improved regret bound for MNL MDPs with variance-aware approach.
problem Optimal reinforcement learning for MNL MDPs with structured variance.
method Introducing a problem-dependent constant measuring average variance, proposing an algorithm with improved regret bound.
result Minimax optimal regret bound of for structured MDPs.
Improved logistic MoE with sigmoid gate shows better sample efficiency.
problem Improving sample efficiency in logistic MoE models.
method Comprehensive analysis of multinomial logistic MoE with modified sigmoid gate, incorporating temperature parameter and using Euclidean score.
result The sigmoid gate leads to lower sample complexity than softmax gate for both parameter and expert estimation.
New algorithms improve MPBART for HIV patient data.
problem Improving inference for multinomial outcomes in HIV patient data.
method Introduced two new algorithms for fitting MPBART.
result Better performance in MCMC convergence and predictive accuracy.
New method estimates density ratio for well-separated distributions using multi-class logistic regression.
problem Challenges in estimating density ratio for well-separated distributions.
method Uses multi-class logistic regression with auxiliary densities to estimate log(p/q).
result Demonstrates superior performance on density ratio estimation, mutual information, and representation learning tasks.
Improved online confidence bounds for multinomial logistic models in bandits.
problem Achieving optimal regret in multinomial logistic bandits with bounded parameters and outcomes.
method Deriving an improved online confidence bound and proposing OFU-MNL++ and OFU-MNL algorithms.
result Achieved variance-dependent optimal regret for MNL bandits.
Machine learning predicts US and EuroZone business cycles with high accuracy.
problem Predicting the business cycle phases in US and EuroZone.
method Three machine learning approaches were compared: Multinomial Logistic Regression (MLR) achieved the best results.
result MLR achieved 65.25% accuracy for EuroZone and 75% for US in predicting business cycle phases.
To model categorical response variables given their covariates, we propose a permuted and augmented stick-breaking (paSB) construction that one-to-one maps the observed categories to randomly permuted latent sticks. This new construction transforms multinomial regression into regression analysis of stick-specific binar…
We propose the nuclear norm penalty as an alternative to the ridge penalty for regularized multinomial regression. This convex relaxation of reduced-rank multinomial regression has the advantage of leveraging underlying structure among the response categories to make better predictions. We apply our method, nuclear pen…
We develop unbiased implicit variational inference (UIVI), a method that expands the applicability of variational inference by defining an expressive variational family. UIVI considers an implicit variational distribution obtained in a hierarchical manner using a simple reparameterizable distribution whose variational …
We propose a new data-augmentation strategy for fully Bayesian inference in models with binomial likelihoods. The approach appeals to a new class of Polya-Gamma distributions, which are constructed in detail. A variety of examples are presented to show the versatility of the method, including logistic regression, negat…
We present a framework for supervised subspace tracking, when there are two time series and , one being the high-dimensional predictors and the other being the response variables and the subspace tracking needs to take into consideration of both sequences. It extends the classic online subspace tracking work…
This paper tackles scalability issues in kernel logistic regression for large datasets.
problem Challenges in training large-scale kernel-based models for discrete choice modelling.
method Introduces Nyström approximation for Kernel Logistic Regression (KLR) on large datasets.
result The k-means Nyström KLR approach is a successful solution for large datasets, maintaining robust performance.
Many practical modeling problems involve discrete data that are best represented as draws from multinomial or categorical distributions. For example, nucleotides in a DNA sequence, children's names in a given state and year, and text documents are all commonly modeled with multinomial distributions. In all of these cas…
This article proposes Multinomial Probit Bayesian Additive Regression Trees (MPBART) as a multinomial probit extension of BART - Bayesian Additive Regression Trees (Chipman et al (2010)). MPBART is flexible to allow inclusion of predictors that describe the observed units as well as the available choice alternatives. T…
When searching for gene pathways leading to specific disease outcomes, additional information on gene characteristics is often available that may facilitate to differentiate genes related to the disease from irrelevant background when connections involving both types of genes are observed and their relationships to the…
The logistic normal distribution has recently been adapted via the transformation of multivariate Gaus- sian variables to model the topical distribution of documents in the presence of correlations among topics. In this paper, we propose a probit normal alternative approach to modelling correlated topical structures. O…
Due to its linear complexity, naive Bayes classification remains an attractive supervised learning method, especially in very large-scale settings. We propose a sparse version of naive Bayes, which can be used for feature selection. This leads to a combinatorial maximum-likelihood problem, for which we provide an exact…
Dropout has been witnessed with great success in training deep neural networks by independently zeroing out the outputs of neurons at random. It has also received a surge of interest for shallow learning, e.g., logistic regression. However, the independent sampling for dropout could be suboptimal for the sake of conver…
High dimensional regression benefits from sparsity promoting regularizations. Screening rules leverage the known sparsity of the solution by ignoring some variables in the optimization, hence speeding up solvers. When the procedure is proven not to discard features wrongly the rules are said to be \emph{safe}. In this …
New algorithm reduces reinforcement learning regret by adapting to interaction variability.
problem Existing reinforcement learning methods lack adaptability to interaction variability.
method Developed a variance-adaptive optimal algorithm for MNL function approximation.
result Achieved instance-wise optimal regret bounds, validating efficiency in practice.
We consider the problem of identifying universal low-dimensional features from high-dimensional data for inference tasks in settings involving learning. For such problems, we introduce natural notions of universality and we show a local equivalence among them. Our analysis is naturally expressed via information geometr…