arXiv research

New method estimates density ratio for well-separated distributions using multi-class logistic regression.

A new approach for signal parametrization, which consists of a specific regression model incorporating a discrete hidden logistic process, is proposed. The model parameters are estimated by the maximum likelihood method performed by a dedicated Expectation Maximization (EM) algorithm. The parameters of the hidden logis…

A new approach for feature extraction from time series is proposed in this paper. This approach consists of a specific regression model incorporating a discrete hidden logistic process. The model parameters are estimated by the maximum likelihood method performed by a dedicated Expectation Maximization (EM) algorithm. …

A major challenge for building statistical models in the big data era is that the available data volume far exceeds the computational capability. A common approach for solving this problem is to employ a subsampled dataset that can be handled by available computational resources. In this paper, we propose a general sub…

We present a method for training multi-label, massively multi-class image classification models, that is faster and more accurate than supervision via a sigmoid cross-entropy loss (logistic regression). Our method consists in embedding high-dimensional sparse labels onto a lower-dimensional dense sphere of unit-normed …

Study develops a robust federated logistic regression for financial data analysis.

This paper revisits the special type of a neural network known under two names. In the statistics and machine learning community it is known as a multi-class logistic regression neural network. In the neural network community, it is simply the soft-max layer. The importance is underscored by its role in deep learning: …

Paper proposes MMVFL for multi-class VFL with multiple participants.

For the problem of multi-class linear classification and feature selection, we propose approximate message passing approaches to sparse multinomial logistic regression (MLR). First, we propose two algorithms based on the Hybrid Generalized Approximate Message Passing (HyGAMP) framework: one finds the maximum a posterio…

Package implements ABC-Boost for multi-class classification.

Regulatory compliance is an organization's adherence to laws, regulations, guidelines and specifications relevant to its business. Compliance officers responsible for maintaining adherence constantly struggle to keep up with the large amount of changes in regulatory requirements. Keeping up with the changes entail two …

Paper explains learning property of logistic and softmax losses for balanced and imbalanced class data.

Scaling multinomial logistic regression to datasets with very large number of data points and classes is challenging. This is primarily because one needs to compute the log-partition function on every data point. This makes distributing the computation hard. In this paper, we present a distributed stochastic gradient d…

We consider multi-agent stochastic optimization problems over reproducing kernel Hilbert spaces (RKHS). In this setting, a network of interconnected agents aims to learn decision functions, i.e., nonlinear statistical models, that are optimal in terms of a global convex functional that aggregates data across the networ…

Federated Learning with L0 constraint improves sparsity and performance.

We examine gradient descent on unregularized logistic regression problems, with homogeneous linear predictors on linearly separable datasets. We show the predictor converges to the direction of the max-margin (hard margin SVM) solution. The result also generalizes to other monotone decreasing loss functions with an inf…

New method deconfounds deep learning feature representations using counterfactual approach.

Study extends learnability equivalence to multi-class and regression, overcoming binary classification limits.

High dimensional regression benefits from sparsity promoting regularizations. Screening rules leverage the known sparsity of the solution by ignoring some variables in the optimization, hence speeding up solvers. When the procedure is proven not to discard features wrongly the rules are said to be \emph{safe}. In this …

Study local geometry of mixture models via spectral theory, revealing transitions in training dynamics.

Multi-class classification methods based on both labeled and unlabeled functional data sets are discussed. We present a semi-supervised logistic model for classification in the context of functional data analysis. Unknown parameters in our proposed model are estimated by regularization with the help of EM algorithm. A …

Paper finds a lower bound for estimating low-rank matrices in logistic regression.

A new framework for selecting base classes in multi-class classification boosts accuracy.

Novel bounds for logistic regression coreset construction and feature selection.

Unified framework for sparse logistic regression with nonconvex regularization.

Prevalidated ridge regression simplifies logistic regression for high-dimensional data.

Improved sketching for logistic and $\ell_1$ regression with near-linear dimensions.

Safe screening rules reduce computation time in logistic regression with $\ell_0-\ell_2$ regularization.

Improves logistic regression performance with nonconvex programming.

A common method of generalizing binary to multi-class classification is the error correcting code (ECC). ECCs may be optimized in a number of ways, for instance by making them orthogonal. Here we test two types of orthogonal ECCs on seven different datasets using three types of binary classifier and compare them with t…

Paper explores connections between loss functions and consistency in binary classification and regression.

Maximum likelihood estimator performance in logistic regression analyzed.

Time series are used in many domains including finance, engineering, economics and bioinformatics generally to represent the change of a measurement over time. Modeling techniques may then be used to give a synthetic representation of such data. A new approach for time series modeling is proposed in this paper. It cons…

We comment on the fact that gradient ascent for logistic regression has a connection with the perceptron learning algorithm. Logistic learning is the "soft" variant of perceptron learning.

Paper introduces imprecise logistic regression for handling uncertain data.

A new data-oblivious sketch for logistic regression reduces data size while maintaining approximation accuracy.

The l1-regularized logistic regression (or sparse logistic regression) is a widely used method for simultaneous classification and feature selection. Although many recent efforts have been devoted to its efficient implementation, its application to high dimensional data still poses significant challenges. In this paper…

PIANO speeds up multinomial logistic regression solving.

Analyzes SGD dynamics on multi-class problems with exact expressions.

Logistic regression is a widely used method in several fields. When applying logistic regression to imbalanced data, for which majority classes dominate over minority classes, all class labels are estimated as `majority class.' In this article, we use an F-measure optimization method to improve the performance of logis…

A new algorithm approximates logistic regression probabilities efficiently.

Revises logistic-softmax likelihood for Bayesian meta-learning in few-shot classification.

Study shows sample complexity for logistic regression with normal covariates.

Logistic regression gets a new, simpler uniform bound.

Non-intrusive algorithm corrects corrupted data in classification models.

Proposes MELODIC family for simultaneous binary logistic regression.

Adapts EGOP to multi-class setting and proposes a simple rough estimator.

Solving logistic regression with L1-regularization in distributed settings is an important problem. This problem arises when training dataset is very large and cannot fit the memory of a single machine. We present d-GLMNET, a new algorithm solving logistic regression with L1-regularization in the distributed settings. …