arXiv research

PIANO speeds up multinomial logistic regression solving.

Sparse multinomial logistic regression for multiclass classification with feature selection.

For the problem of multi-class linear classification and feature selection, we propose approximate message passing approaches to sparse multinomial logistic regression (MLR). First, we propose two algorithms based on the Hybrid Generalized Approximate Message Passing (HyGAMP) framework: one finds the maximum a posterio…

The paper proves asymptotic normality for multinomial logistic regression on null covariates.

New method for multiclass classification reduces error bounds.

We propose a penalized likelihood method that simultaneously fits the multinomial logistic regression model and combines subsets of the response categories. The penalty is non differentiable when pairs of columns in the optimization variable are equal. This encourages pairwise equality of these columns in the estimator…

Proposes a method to use external machine-learning predictions in multinomial logistic regression.

We develop an approximate formula for evaluating a cross-validation estimator of predictive likelihood for multinomial logistic regression regularized by an $\ell_1$-norm. This allows us to avoid repeated optimizations required for literally conducting cross-validation; hence, the computational time can be significantl…

Scaling multinomial logistic regression to datasets with very large number of data points and classes is challenging. This is primarily because one needs to compute the log-partition function on every data point. This makes distributing the computation hard. In this paper, we present a distributed stochastic gradient d…

Unified framework for sparse logistic regression with nonconvex regularization.

Due to its linear complexity, naive Bayes classification remains an attractive supervised learning method, especially in very large-scale settings. We propose a sparse version of naive Bayes, which can be used for feature selection. This leads to a combinatorial maximum-likelihood problem, for which we provide an exact…

We improve MoE models for classification with rigorous guarantees and practical methods.

The paper establishes convergence rates for MoE models in classification problems.

The l1-regularized logistic regression (or sparse logistic regression) is a widely used method for simultaneous classification and feature selection. Although many recent efforts have been devoted to its efficient implementation, its application to high dimensional data still poses significant challenges. In this paper…

We present ADMM-Softmax, an alternating direction method of multipliers (ADMM) for solving multinomial logistic regression (MLR) problems. Our method is geared toward supervised classification tasks with many examples and features. It decouples the nonlinear optimization problem in MLR into three steps that can be solv…

New algorithm for maximizing revenue in multinomial logistic bandits.

We introduce Graph-Sparse Logistic Regression, a new algorithm for classification for the case in which the support should be sparse but connected on a graph. We val- idate this algorithm against synthetic data and benchmark it against L1-regularized Logistic Regression. We then explore our technique in the bioinformat…

Approximates large Random Forest models to save space.

FisherSFT selects informative examples to fine-tune LLMs efficiently.

Improved VB algorithm for high-dimensional logistic regression with theoretical guarantees.

Safe screening rules reduce computation time in logistic regression with $\ell_0-\ell_2$ regularization.

Efficient RL algorithm for multinomial logistic MDPs with provable guarantees.

Logistic regression models are a popular and effective method to predict the probability of categorical response data. However inference for these models can become computationally prohibitive for large datasets. Here we adapt ideas from symbolic data analysis to summarise the collection of predictor variables into his…

High dimensional regression benefits from sparsity promoting regularizations. Screening rules leverage the known sparsity of the solution by ignoring some variables in the optimization, hence speeding up solvers. When the procedure is proven not to discard features wrongly the rules are said to be \emph{safe}. In this …

Trans-GCR uses GCR model for node classification, providing theoretical guarantees and superior performance.

Proposes efficient Bayesian logistic regression for large sparse datasets.

Two algorithms achieve optimal regret with limited adaptivity in multinomial logistic bandits.

Flexible Bayesian approach for generalized linear models, especially for sparse logistic regression.

MM (majorization--minimization) algorithms are an increasingly popular tool for solving optimization problems in machine learning and statistical estimation. This article introduces the MM algorithm framework in general and via three popular example applications: Gaussian mixture regressions, multinomial logistic regre…

Improved regret bounds for logistic bandits via novel confidence set construction.

Model predicts individual insurance claim reserves using activation patterns.

The paper examines logistic regression in sparse network settings, improving inference under varying degrees of dyadic dependence.

In this work we propose to fit a sparse logistic regression model by a weakly convex regularized nonconvex optimization problem. The idea is based on the finding that a weakly convex function as an approximation of the $\ell_0$ pseudo norm is able to better induce sparsity than the commonly used $\ell_1$ norm. For a cl…

Improved CRT for sparse logistic regression in high dimensions.

Optimal sketching bounds for sparse linear regression under various loss functions are established.

We consider high-dimensional binary classification by sparse logistic regression. We propose a model/feature selection procedure based on penalized maximum likelihood with a complexity penalty on the model size and derive the non-asymptotic bounds for the resulting misclassification excess risk. The bounds can be reduc…

Improved regret bound for multinomial logistic bandits with non-linearity.

A new algorithm reduces the time and space complexity for multinomial logistic bandits.

Enhances neural networks in hyperbolic space for better data structure capture.

Researchers find a class/cross-class structure in deep learning spectra.

A neural network solves logistic regression with $\ell_1$ regularization efficiently.

$L_1$ regularized logistic regression has now become a workhorse of data mining and bioinformatics: it is widely used for many classification problems, particularly ones with many features. However, $L_1$ regularization typically selects too many features and that so-called false positives are unavoidable. In this pape…

The pseudo-likelihood method is one of the most popular algorithms for learning sparse binary pairwise Markov networks. In this paper, we formulate the $L_1$ regularized pseudo-likelihood problem as a sparse multiple logistic regression problem. In this way, many insights and optimization procedures for sparse logistic…

New algorithm learns sparse GLMs for binary outcomes efficiently.

The paper provides guarantees for a tangent transform algorithm in logistic regression models.

Logistic regression is commonly used for modeling dichotomous outcomes. In the classical setting, where the number of observations is much larger than the number of parameters, properties of the maximum likelihood estimator in logistic regression are well understood. Recently, Sur and Candes have studied logistic regre…

In high dimensional regression settings, sparsity enforcing penalties have proved useful to regularize the data-fitting term. A recently introduced technique called screening rules propose to ignore some variables in the optimization leveraging the expected sparsity of the solutions and consequently leading to faster s…

New weighted Lasso estimates improve logistic regression performance with measurement error.