arXiv research

FOLKLORE algorithm speeds up online multiclass logistic regression.

Improved multiclass logistic regression with lower computational complexity.

Learning linear predictors with the logistic loss---both in stochastic and online settings---is a fundamental task in machine learning and statistics, with direct connections to classification and boosting. Existing "fast rates" for this setting exhibit exponential dependence on the predictor norm, and Hazan et al. (20…

Sparse multinomial logistic regression for multiclass classification with feature selection.

New method for multiclass classification reduces error bounds.

Method extracts features from signals for classification with explainability.

Gaptron algorithm reduces mistakes in online multiclass classification.

New algorithm selects optimal subset for multiclass classifier training.

The paper revisits discriminative vs. generative classifiers, showing naive Bayes requires fewer samples.

Multilabel classification is an important problem in a wide range of domains such as text categorization and music annotation. In this paper, we present a probabilistic model, Multilabel Logistic Regression with Hidden variables (MLRH), which extends the standard logistic regression by introducing hidden variables. Hid…

Data labeling is currently a time-consuming task that often requires expert knowledge. In research settings, the availability of correctly labeled data is crucial to ensure that model predictions are accurate and useful. We propose relatively simple machine learning-based models that achieve high performance metrics in…

Develops a robust multiclass classification method for deep image classifiers.

Improved robustness in multivariate regression and classification with DRO under Wasserstein metric.

Conventional multiclass conditional probability estimation methods, such as Fisher's discriminate analysis and logistic regression, often require restrictive distributional model assumption. In this paper, a model-free estimation method is proposed to estimate multiclass conditional probability through a series of cond…

Generalized additive models (GAMs) are favored in many regression and binary classification problems because they are able to fit complex, nonlinear functions while still remaining interpretable. In the first part of this paper, we generalize a state-of-the-art GAM learning algorithm based on boosted trees to the multi…

New algorithm reduces online logistic regression regret without exponential constant.

We introduce a proximal version of the stochastic dual coordinate ascent method and show how to accelerate the method using an inner-outer iteration procedure. We analyze the runtime of the framework and obtain rates that improve state-of-the-art results for various key machine learning optimization problems including …

Proposes efficient Bayesian logistic regression for large sparse datasets.

Solving logistic regression with L1-regularization in distributed settings is an important problem. This problem arises when training dataset is very large and cannot fit the memory of a single machine. We present d-GLMNET, a new algorithm solving logistic regression with L1-regularization in the distributed settings. …

Study online multiclass classification under bandit feedback, extending previous results.

Extends boosting to multiclass online agnostic classification.

FIRAL is a scalable active learning algorithm for multiclass classification.

Extends quantum learning theory to multiclass and online settings.

Recent work has extended the theoretical analysis of boosting algorithms to multiclass problems and to online settings. However, the multiclass extension is in the batch setting and the online extensions only consider binary classification. We fill this gap in the literature by defining, and justifying, a weak learning…

Proposes RTL model for sentiment classification and key word detection in online reviews.

Online Platt Scaling adapts to varying data distributions.

Bayesian logistic regression improves clinical risk prediction models over time.

Study improves online learning with adaptable agents in various settings.

New research shows SVM and related methods can overfit without harm in multiclass classification.

The article describe the model, derivation, and implementation of variational Bayesian inference for linear and logistic regression, both with and without automatic relevance determination. It has the dual function of acting as a tutorial for the derivation of variational Bayesian inference for simple models, as well a…

Proposes a method to use external machine-learning predictions in multinomial logistic regression.

GD with large, adaptive stepsizes achieves optimal risk in logistic regression.

New algorithm learns multiclass concepts with finite Littlestone dimension.

New method estimates density ratio for well-separated distributions using multi-class logistic regression.

Establishes a condition for multiclass classification-calibration of Gamma-Phi losses.

We consider Online Convex Optimization (OCO) in the setting where the costs are $m$-strongly convex and the online learner pays a switching cost for changing decisions between rounds. We show that the recently proposed Online Balanced Descent (OBD) algorithm is constant competitive in this setting, with competitive rat…

Study online learning with set-valued feedback, showing differences between deterministic and randomized approaches.

We present an efficient second-order algorithm with $\tilde{O}(\frac{1}η\sqrt{T})$ regret for the bandit online multiclass problem. The regret bound holds simultaneously with respect to a family of loss functions parameterized by $η$, for a range of $η$ restricted by the norm of the competitor. The family of loss funct…

Lower bounds on online logistic regression regret rates derived.

We propose a Fourier-based learning algorithm for highly nonlinear multiclass classification. The algorithm is based on a smoothing technique to calculate the probability distribution of all classes. To obtain the probability distribution, the density distribution of each class is smoothed by a low-pass filter separate…

Develops a functional mix-of-experts model for multiclass classification.

Online L2D algorithm for multiclass classification with varying experts.

We present a framework for supervised subspace tracking, when there are two time series $x_t$ and $y_t$, one being the high-dimensional predictors and the other being the response variables and the subspace tracking needs to take into consideration of both sequences. It extends the classic online subspace tracking work…

We present a family of expectation-maximization (EM) algorithms for binary and negative-binomial logistic regression, drawing a sharp connection with the variational-Bayes algorithm of Jaakkola and Jordan (2000). Indeed, our results allow a version of this variational-Bayes approach to be re-interpreted as a true EM al…

Large stepsize GD for logistic regression converges faster than expected.

In this work we provide a theoretical framework for structured prediction that generalizes the existing theory of surrogate methods for binary and multiclass classification based on estimating conditional probabilities with smooth convex surrogates (e.g. logistic regression). The theory relies on a natural characteriza…

New method predicts customer churn using mixed-penalty logistic regression.

We develop efficient algorithms to train $\ell_1$-regularized linear classifiers with large dimensionality $d$ of the feature space, number of classes $k$, and sample size $n$. Our focus is on a special class of losses that includes, in particular, the multiclass hinge and logistic losses. Our approach combines several…