Paper finds a lower bound for estimating low-rank matrices in logistic regression.
problem Estimating low-rank coefficient matrices in logistic regression.
method Derives a minimax lower bound on the risk.
result The bound depends on matrix dimensions, rank, and sample size.
Novel bounds for logistic regression coreset construction and feature selection.
problem Efficiently summarize and reduce logistic regression inputs.
method Feature space sketching for logistic regression.
result Tight bounds for coreset construction and feature selection.
Unified framework for sparse logistic regression with nonconvex regularization.
problem Sparse logistic regression with nonconvex regularization.
method Unified framework, line search criteria for nonconvex terms.
result Effective classification and feature selection at lower computational cost.
Prevalidated ridge regression simplifies logistic regression for high-dimensional data.
problem Efficient probabilistic classification in high-dimensional data with logistic regression.
method Developed a prevalidated ridge regression model that matches logistic regression's performance but is more computationally efficient.
result Prevalidated ridge regression achieves similar classification error and log-loss to logistic regression for high-dimensional data.
Improved sketching for logistic and ℓ 1 \ell_1 ℓ 1 regression with near-linear dimensions.
problem Efficiently approximate ℓ 1 \ell_1 ℓ 1 and logistic regression problems. method New sketching techniques achieving near-linear dimensions for both problems.
result Achieved near-linear sketching dimensions for ℓ 1 \ell_1 ℓ 1 and logistic regression. Paper introduces MLRH, a probabilistic model for multilabel classification.
problem Multilabel classification challenges in various domains.
method Introduces hidden variables to relax one-hot-encoding in logistic regression.
result Probabilistic model achieves competitive performance compared to other algorithms.
Safe screening rules reduce computation time in logistic regression with ℓ 0 − ℓ 2 \ell_0-\ell_2 ℓ 0 − ℓ 2 regularization.
problem Efficiently solving logistic regression with many features and regularization.
method Screening rules based on Fenchel dual lower bounds of strong conic relaxations.
result A high percentage of features can be safely removed before solving, leading to substantial speed-up.
Improves logistic regression performance with nonconvex programming.
problem Stochastic generalized linear regression with chance constraints.
method Nonconvex programming techniques, clustering, quantile estimation.
result Over 1 to 2 percent improvement in model performance.
Paper explores connections between loss functions and consistency in binary classification and regression.
problem Consistency in binary classification and regression applications.
method Characterization of conformable loss functions and derivation of a new Huber-type loss function.
result Margin-based loss functions are equivalent to loss functions of squared standardized logistic regression residuals.
Maximum likelihood estimator performance in logistic regression analyzed.
problem Performance of maximum likelihood estimator in logistic regression.
method Sharp non-asymptotic guarantees for existence and excess logistic risk.
result Sharp guarantees for the existence and excess risk of MLE in logistic regression.
We comment on the fact that gradient ascent for logistic regression has a connection with the perceptron learning algorithm. Logistic learning is the "soft" variant of perceptron learning.
Paper introduces imprecise logistic regression for handling uncertain data.
problem Uncertainties in data prevent traditional logistic regression from being applied effectively.
method Develops imprecise logistic regression model using intervals of possible values.
result Clearly expresses epistemic uncertainty in predictions.
PIANO speeds up multinomial logistic regression solving.
problem Handling large datasets and many classes in logistic regression.
method Parallel iterative algorithm based on Majorization Minimization.
result PIANO converges to a stationary point of Multinomial and Sparse Multinomial Logistic Regression.
The l1-regularized logistic regression (or sparse logistic regression) is a widely used method for simultaneous classification and feature selection. Although many recent efforts have been devoted to its efficient implementation, its application to high dimensional data still poses significant challenges. In this paper…
A new data-oblivious sketch for logistic regression reduces data size while maintaining approximation accuracy.
problem Efficiently solving logistic regression in one pass over a data stream.
method Data-oblivious sketching approach that reduces data size to poly(μdlog n) weighted points.
result Sketching reduces data size significantly and provides approximation guarantees.
Logistic regression is a widely used method in several fields. When applying logistic regression to imbalanced data, for which majority classes dominate over minority classes, all class labels are estimated as `majority class.' In this article, we use an F-measure optimization method to improve the performance of logis…
A new algorithm approximates logistic regression probabilities efficiently.
problem Efficiently approximating probabilities in logistic regression for large datasets.
method Randomized sampling-based algorithm with leverage scores.
result Accurate approximations to estimated probabilities with smaller sample sizes.
Study shows sample complexity for logistic regression with normal covariates.
problem Estimating parameters of logistic regression with normal design.
method Analyzes sample complexity in terms of dimension and inverse temperature.
result Shows two change-points in sample complexity curve based on inverse temperature.
Logistic regression gets a new, simpler uniform bound.
problem Finding a uniform bound for logistic regression's empirical risk.
method PAC-Bayes approach with second-order expansion and Rademacher-complexity bounds.
result Provides a dimension-free uniform concentration bound.
Proposes MELODIC family for simultaneous binary logistic regression.
problem Simultaneous analysis of multiple binary response variables.
method Defines models in a reduced Euclidean space using a distance rule.
result Improves predictive accuracy through model interdependence.
Coresets are one of the central methods to facilitate the analysis of large data sets. We continue a recent line of research applying the theory of coresets to logistic regression. First, we show a negative result, namely, that no strongly sublinear sized coresets exist for logistic regression. To deal with intractable…
Solving logistic regression with L1-regularization in distributed settings is an important problem. This problem arises when training dataset is very large and cannot fit the memory of a single machine. We present d-GLMNET, a new algorithm solving logistic regression with L1-regularization in the distributed settings. …
Improved classifier for PU data using logistic regression.
problem Analysis of Positive Unlabeled data under SCAR assumption.
method Fitting misspecified logistic regression model to PU data.
result The classifier performs on par or better than competitors on real data sets.
A neural network solves logistic regression with ℓ 1 \ell_1 ℓ 1 regularization efficiently.
problem Efficiently solving logistic regression with ℓ 1 \ell_1 ℓ 1 regularization due to non-differentiability of ℓ 1 \ell_1 ℓ 1 norm. method A simple projection neural network that avoids auxiliary variables and smooth approximations.
result The neural network converges to a solution of the problem with any initial value and outperforms existing methods.
Improved logistic regression for multi-omics data improves prediction and variable selection.
problem Predicting binary class labels from multi-omics datasets with varying characteristics.
method Two-step penalized logistic regression with separate variable selection for each data layer.
result Our approach selects more relevant predictors and achieves comparable prediction performance.
In this work, we study the use of logistic regression in manufacturing failures detection. As a data set for the analysis, we used the data from Kaggle competition Bosch Production Line Performance. We considered the use of machine learning, linear and Bayesian models. For machine learning approach, we analyzed XGBoost…
We introduce Graph-Sparse Logistic Regression, a new algorithm for classification for the case in which the support should be sparse but connected on a graph. We val- idate this algorithm against synthetic data and benchmark it against L1-regularized Logistic Regression. We then explore our technique in the bioinformat…
Paper proposes a faster federated learning method for logistic regression.
problem Federated learning for logistic regression with disjoint feature sets.
method Quasi-Newton method under additively homomorphic encryption.
result Significant reduction in communication rounds with minimal additional cost.
Improved multiclass logistic regression with lower computational complexity.
problem High computational complexity in existing methods for multiclass logistic regression.
method Developed a new algorithm that achieves a lower computational complexity.
result Achieved a regret of O ( log ( B n ) ) O(\log(Bn)) O ( log ( B n )) with computational complexity O ( n 1.5 ) O(n^{1.5}) O ( n 1.5 ) . Efficient logistic regression for aggregated data reduces computation time.
problem Inference for logistic regression models is computationally expensive for large datasets.
method Adapted symbolic data analysis to summarise predictor variables into histograms and use composite likelihoods.
result The method achieves comparable classification rates to full data analysis but at lower computational cost.
Decision trees and logistic regression are one of the most popular and well-known machine learning algorithms, frequently used to solve a variety of real-world problems. Stability of learning algorithms is a powerful tool to analyze their performance and sensitivity and subsequently allow researchers to draw reliable c…
Paper proposes a new method for estimating conditional densities using logistic regressions.
problem Estimating conditional densities for complex distributions.
method Parametric conditional density estimation via weighted logistic regressions.
result Maximum likelihood estimates can be obtained efficiently via a block-wise alternating maximization scheme and local case-control sampling.
Machine learning improves joint default assessment by capturing non-linear dependencies.
problem Capturing non-linear dependencies among covariates for accurate joint default assessment.
method Application of machine learning techniques to credit card dataset, comparing with logistic regression.
result Machine learning outperforms logistic regression in assessing portfolio riskiness.
Early stopping improves logistic regression's calibration and consistency in high dimensions.
problem Improving the statistical performance of gradient descent in overparameterized logistic regression.
method Investigates the effects of early stopping on gradient descent in logistic regression.
result Early-stopped gradient descent is well-calibrated and statistically consistent, while asymptotic gradient descent is not.
We propose a new algorithm called PLUTO for building logistic regression trees to binary response data. PLUTO can capture the nonlinear and interaction patterns in messy data by recursively partitioning the sample space. It fits a simple or a multiple linear logistic regression model in each partition. PLUTO employs th…
Large stepsizes can accelerate gradient descent for logistic regression.
problem Optimizing logistic regression with large stepsizes.
method Gradient descent with large stepsize for ℓ 2 \ell_2 ℓ 2 -regularized logistic regression. result Large stepsizes can achieve O ~ ( κ ) \widetilde{\mathcal{O}}(\sqrtκ) O ( κ ) convergence, improving over O ~ ( κ ) \widetilde{\mathcal{O}}(\sqrtκ) O ( κ ) from classical theory. Paper introduces data-dependent SSP for private linear and logistic regression.
problem Private linear and logistic regression with better performance.
method Data-dependent sufficient statistic perturbation (SSP) for linear and logistic regression.
result Data-dependent SSP outperforms state-of-the-art methods for linear and logistic regression.
A successful approach to structured learning is to write the learning objective as a joint function of linear parameters and inference messages, and iterate between updates to each. This paper observes that if the inference problem is "smoothed" through the addition of entropy terms, for fixed messages, the learning ob…
Multi-task learning has shown to significantly enhance the performance of multiple related learning tasks in a variety of situations. We present the fused logistic regression, a sparse multi-task learning approach for binary classification. Specifically, we introduce sparsity inducing penalties over parameter differenc…
SLOE speeds up logistic regression in high dimensions with accurate signal strength estimation.
problem Poor performance of logistic regression in high-dimensional settings.
method SLOE reparameterizes the signal strength for faster and more accurate estimation.
result SLOE provides a fast and accurate method for dimensionality correction in logistic regression.
Improved VB algorithm for high-dimensional logistic regression with theoretical guarantees.
problem Sparse high-dimensional logistic regression model selection.
method Spike and slab variational Bayes approximation.
result Optimal convergence rates in ℓ 2 \ell_2 ℓ 2 and prediction loss for sparse truths. Efficiently selects important variables in high-dimensional logistic regression.
problem Variable selection in high-dimensional logistic regression with binary responses.
method Developed a variational empirical Bayes approach for efficient model space marginal distribution.
result The variational approximation inherits strong selection consistency from the posterior distribution.
Sparse multinomial logistic regression for multiclass classification with feature selection.
problem High-dimensional multiclass classification with a focus on sparse models.
method Penalized maximum likelihood with complexity penalty, feature selection using group Lasso and Slope classifiers.
result Achievement of minimax order in both small and large number of classes regimes.
The paper proposes a new method for comparing logistic regression models across different populations.
problem Comparing logistic regression models across sub-populations can lead to misleading results.
method Develops a cascading set of equivalence tests for logistic regression models, addressing coding, predictions, and overall accuracy.
result Equivalence testing incentivizes accurate inference and avoids perverse incentives from significance tests.
A fast method for Lasso and Logistic Lasso problems.
problem Solving Lasso and Logistic Lasso regression problems efficiently.
method Iterative active set approach using solver updates.
result 31.41 times faster on average for compressed sensing.
Enhances KLR for indefinite kernels with L 1 L_1 L 1 -norm regularization.
problem Classifying with indefinite kernels captures more domain-specific information.
method Introduces L 1 L_1 L 1 -norm regularization to induce sparsity and a proximal linearized algorithm. result Superior performance in accuracy and sparsity on multiple datasets.
FOLKLORE algorithm speeds up online multiclass logistic regression.
problem Efficiently solving online multiclass logistic regression without high computational cost.
method Developed FOLKLORE algorithm with improved runtime and regret bound.
result First practical algorithm for online multiclass logistic regression.
Adversarial training makes logistic regression weight loss landscapes sharper.
problem Understanding why adversarial training sharpens the weight loss landscape in logistic regression.
method Theoretical analysis of linear logistic regression model with L2 norm constraints, and experiments on ResNet18.
result Adversarial training sharpens the weight loss landscape in linear logistic regression models.