Paper finds a lower bound for estimating low-rank matrices in logistic regression.
problem Estimating low-rank coefficient matrices in logistic regression.
method Derives a minimax lower bound on the risk.
result The bound depends on matrix dimensions, rank, and sample size.
Optimizes link prediction using matrix logistic regression.
problem Predicting links in networks with limited data.
method Formulated as matrix logistic regression, analyzed in high dimensions, combinatorial estimator with penalized maximum likelihood.
result Achieves minimax rate for Frobenius-norm risk, cannot be computed efficiently.
New method uses nuclear and ℓ1 penalties for matrix regression, improving brain disorder detection.
problem Modeling high-dimensional matrix predictors with binary responses.
method Convex optimization with ADMM for low-rank and sparse structures.
result Effective in identifying brain disorder-related connectivity patterns.
This paper analyzes stability of decision trees and logistic regression.
problem Stability of decision trees and logistic regression is analyzed to understand their performance and sensitivity.
method Two stability notions (hypothesis and pointwise hypothesis stability) are derived for decision trees and logistic regression. The stability of decision trees depends on the number of leaves, while for logistic regression, it depends on the smallest eigenvalue of the Hessian matrix. Upper bounds on generalization error are constructed.
result Logistic regression is not a stable learning algorithm.
A new hybrid Newton algorithm improves convergence in logistic regression.
problem Solving large-scale binary classification problems efficiently.
method Proposes a hybrid stochastic Newton algorithm with two weighted components in the Hessian matrix estimation.
result Proves almost sure convergence to the true parameter of logistic regression.
Improves logistic regression performance with nonconvex programming.
problem Stochastic generalized linear regression with chance constraints.
method Nonconvex programming techniques, clustering, quantile estimation.
result Over 1 to 2 percent improvement in model performance.
A new algorithm approximates logistic regression probabilities efficiently.
problem Efficiently approximating probabilities in logistic regression for large datasets.
method Randomized sampling-based algorithm with leverage scores.
result Accurate approximations to estimated probabilities with smaller sample sizes.
Efficiently preserves privacy in logistic regression for IoT data.
problem Balancing data privacy and utility in collaborative learning.
method Matrix encryption approach for secure multi-party computation.
result Proposes a privacy-preserving logistic regression model with fast convergence.
New research finds many coreset methods for logistic regression are not better than simple sampling.
problem Evaluation of coreset methods for reducing data size in logistic regression.
method Comparison of multiple coreset and optimal subsampling methods for logistic regression.
result Many coreset methods do not outperform simple uniform subsampling.
Study high-dimensional logistic regression with missing data, providing exact error characterizations.
problem High-dimensional logistic regression with missing or corrupted covariates.
method Exact characterizations of prediction and estimation errors under independence and moment conditions.
result Characterizations are universal and hold for various imputation strategies.
The likelihood ratio test in high-dimensional logistic regression is not a chi-square, but a rescaled one.
problem The incorrect chi-square approximation in high-dimensional logistic regression.
method Proving the rescaled chi-square distribution and solving nonlinear equations.
result The likelihood ratio test is a rescaled chi-square, not a standard chi-square.
Method completes mixed matrix from complex surveys with heterogeneous missingness.
problem Recovering a mixed dataframe matrix from complex survey sampling with different missingness patterns.
method Two-stage procedure: logistic regression for missingness modeling, and weighted log-likelihood maximization with low-rank constraint.
result The proposed method achieves sublinear convergence and shows superior performance compared to existing methods.
This paper proposes robust matrix variate regression models with rank constraints and vector regularization.
problem High dimensional and noisy matrix-valued predictors in regression models.
method Rank constraint, vector regularization, alternating projected gradient descent algorithm.
result The proposed method achieves the minimax rate of estimation errors.
Transformers can approximate Newton's method for logistic regression.
problem Implementing higher order optimization methods in Transformers.
method Linear attention Transformers with ReLU layers approximating second order optimization algorithms.
result Transformers can implement a single step of Newton's iteration for matrix inversion.
New algorithm samples matrix rows proportional to their ℓ_p norm in a turnstile data stream.
problem Sampling rows of a dynamic matrix efficiently in a turnstile data stream.
method Develops a novel algorithm for sampling rows proportional to their ℓ_p norm in a turnstile data stream, returning sampled row indexes and approximated sampling probabilities.
result Achieves (1+ε) approximation for logistic regression in a turnstile data stream with polynomial sketch size. A method for logistic regression inference using both internal and external data.
problem Inability to estimate intercept and marginal case proportion in case-control logistic regression.
method Empirical likelihood approach integrating internal and external data.
result Intercept parameter becomes identifiable with external information, and all parameters are estimable consistently.
A multilevel framework speeds up sparse optimization for inverse covariance estimation and logistic regression.
problem Sparse optimization problems in machine learning and signal processing.
method Multilevel framework exploiting sparseness of solutions.
result Efficiently solves l1 regularized optimization problems for inverse covariance estimation and logistic regression.
The paper improves matrix completion with auxiliary covariates using LS estimation.
problem Matrix completion with noisy data and auxiliary covariates.
method Iterative least squares estimation with statistical properties derived.
result Asymptotic normal distributions of estimators for low-rank matrix and coefficient matrix.
Logistic regression connects to perceptron learning via gradient ascent.
problem No specific problem stated; focuses on connection between algorithms.
method Gradient ascent for logistic regression compared to perceptron learning.
result Gradient ascent for logistic regression is a soft variant of perceptron learning.
Improved robustness in multivariate regression and classification with DRO under Wasserstein metric.
problem Outliers in covariates and responses.
method Distributionally Robust Optimization (DRO) with Wasserstein metric ambiguity set and regularization.
result Significant improvement in predictive error and robustness.
Novel bounds for logistic regression coreset construction and feature selection.
problem Efficiently summarize and reduce logistic regression inputs.
method Feature space sketching for logistic regression.
result Tight bounds for coreset construction and feature selection.
Unified framework for sparse logistic regression with nonconvex regularization.
problem Sparse logistic regression with nonconvex regularization.
method Unified framework, line search criteria for nonconvex terms.
result Effective classification and feature selection at lower computational cost.
Logistic regression models with n observations and q linearly-independent covariates are shown to have Fisher information volumes which are bounded below by πq and above by (qn)πq. This is proved with a novel generalization of the classical theorems of Pythagoras and de Gua, which is of independent …
Improves logistic regression performance on imbalanced data.
problem Imbalanced data leads to all labels being estimated as majority class.
method Uses F-measure optimization to estimate relative density ratio and approximate relative F-measure.
result Proposed method improves logistic regression performance on imbalanced data.
Graph-Sparse Logistic Regression for sparse and connected support classification.
problem Sparse and connected support classification problems.
method Introduces Graph-Sparse Logistic Regression algorithm.
result Validated and benchmarked against L1-regularized Logistic Regression.
New method estimates missingness probabilities for MNAR matrix completion.
problem Bias in matrix completion due to missing not at random data.
method Estimate missingness probabilities using nuclear norm structure.
result Improved matrix completion accuracy without auxiliary information.
Prevalidated ridge regression simplifies logistic regression for high-dimensional data.
problem Efficient probabilistic classification in high-dimensional data with logistic regression.
method Developed a prevalidated ridge regression model that matches logistic regression's performance but is more computationally efficient.
result Prevalidated ridge regression achieves similar classification error and log-loss to logistic regression for high-dimensional data.
Improved sketching for logistic and ℓ1 regression with near-linear dimensions.
problem Efficiently approximate ℓ1 and logistic regression problems. method New sketching techniques achieving near-linear dimensions for both problems.
result Achieved near-linear sketching dimensions for ℓ1 and logistic regression. Supervised NMF improves ICU mortality risk prediction.
problem Predicting ICU mortality risk from complex, temporal data.
method Adopted Subgraph Augmented Nonnegative Matrix Factorization (SANMF) and integrated logistic regression loss function.
result Supervised SANMF outperformed conventional methods in ICU mortality risk prediction.
New method creates coresets for logistic regression, even for hard data.
problem Creating efficient coresets for logistic regression analysis.
method Introduce a complexity measure μ(X) and a sensitivity sampling scheme for bounded μ(X) complexity.
result First provably sublinear (1±ε)-coreset for logistic regression.
New method for multiclass classification reduces error bounds.
problem High-dimensional multiclass classification with sparse coefficients.
method Sparse multinomial logistic regression with convex penalties.
result Plug-in classifiers achieve minimax generalization error bounds.
Paper introduces MLRH, a probabilistic model for multilabel classification.
problem Multilabel classification challenges in various domains.
method Introduces hidden variables to relax one-hot-encoding in logistic regression.
result Probabilistic model achieves competitive performance compared to other algorithms.
Safe screening rules reduce computation time in logistic regression with ℓ0−ℓ2 regularization.
problem Efficiently solving logistic regression with many features and regularization.
method Screening rules based on Fenchel dual lower bounds of strong conic relaxations.
result A high percentage of features can be safely removed before solving, leading to substantial speed-up.
Study uses machine learning, linear, and Bayesian models for logistic regression in manufacturing failures detection.
problem Manufacturing failures detection using logistic regression models.
method Machine learning (XGBoost), linear, and Bayesian approaches for logistic regression.
result Bayesian approach provides statistical distribution for model parameters, useful for probabilistic analysis.
Paper explores connections between loss functions and consistency in binary classification and regression.
problem Consistency in binary classification and regression applications.
method Characterization of conformable loss functions and derivation of a new Huber-type loss function.
result Margin-based loss functions are equivalent to loss functions of squared standardized logistic regression residuals.
Maximum likelihood estimator performance in logistic regression analyzed.
problem Performance of maximum likelihood estimator in logistic regression.
method Sharp non-asymptotic guarantees for existence and excess logistic risk.
result Sharp guarantees for the existence and excess risk of MLE in logistic regression.
Paper introduces imprecise logistic regression for handling uncertain data.
problem Uncertainties in data prevent traditional logistic regression from being applied effectively.
method Develops imprecise logistic regression model using intervals of possible values.
result Clearly expresses epistemic uncertainty in predictions.
PIANO speeds up multinomial logistic regression solving.
problem Handling large datasets and many classes in logistic regression.
method Parallel iterative algorithm based on Majorization Minimization.
result PIANO converges to a stationary point of Multinomial and Sparse Multinomial Logistic Regression.
The l1-regularized logistic regression (or sparse logistic regression) is a widely used method for simultaneous classification and feature selection. Although many recent efforts have been devoted to its efficient implementation, its application to high dimensional data still poses significant challenges. In this paper…
A new data-oblivious sketch for logistic regression reduces data size while maintaining approximation accuracy.
problem Efficiently solving logistic regression in one pass over a data stream.
method Data-oblivious sketching approach that reduces data size to poly(μdlog n) weighted points.
result Sketching reduces data size significantly and provides approximation guarantees.
New method calibrates logistic regression tuning parameters.
problem Difficulty in calibrating tuning parameters in logistic regression.
method Simple tests along the tuning parameter path.
result Optimal guarantees for feature selection.
An ensemble approach learns vector-weighted formulae for RLR.
problem Learning Relational Logistic Regression (RLR) with vector-weighted features.
method Functional-gradient boosting methods for probabilistic logic models.
result Our approach outperforms other methods for learning RLR.
Paper compares SVM and Bayesian Logistic Regression for breast cancer diagnosis.
problem Improving breast cancer diagnosis accuracy using machine learning.
method Used SVM, Bayesian Logistic Regression, and K-Nearest-Neighbors for classification.
result SVM outperformed other classifiers, closely matched by Bayesian Logistic Regression.
Study shows sample complexity for logistic regression with normal covariates.
problem Estimating parameters of logistic regression with normal design.
method Analyzes sample complexity in terms of dimension and inverse temperature.
result Shows two change-points in sample complexity curve based on inverse temperature.
Logistic regression gets a new, simpler uniform bound.
problem Finding a uniform bound for logistic regression's empirical risk.
method PAC-Bayes approach with second-order expansion and Rademacher-complexity bounds.
result Provides a dimension-free uniform concentration bound.
To improve accuracy and speed of regressions and classifications, we present a data-based prediction method, Random Bits Regression (RBR). This method first generates a large number of random binary intermediate/derived features based on the original input matrix, and then performs regularized linear/logistic regressio…
Proposes a method to identify relevant genes in autism-related diseases using auxiliary information.
problem Identifying relevant genes in autism-related diseases from diverse data sources.
method Uses logistic regression to filter irrelevant genes and clusters relevant genes into cohesive groups using adjacency matrix.
result Superior performance and robustness in finite samples observed in simulation studies.
Researchers find a class/cross-class structure in deep learning spectra.
problem Understanding the structure of deep learning classifiers.
method Empirical spectral analysis of deepnet classifiers.
result The cross-class structure explains various features in deepnet spectra.