arXiv research

Classification is the most important process in data analysis. However, due to the inherent non-convex and non-smooth structure of the zero-one loss function of the classification model, various convex surrogate loss functions such as hinge loss, squared hinge loss, logistic loss, and exponential loss are introduced. T…

Paper explores connections between loss functions and consistency in binary classification and regression.

SGD converges globally to logistic loss minima for two-layer nets.

A successful approach to structured learning is to write the learning objective as a joint function of linear parameters and inference messages, and iterate between updates to each. This paper observes that if the inference problem is "smoothed" through the addition of entropy terms, for fixed messages, the learning ob…

Multi-class classification methods based on both labeled and unlabeled functional data sets are discussed. We present a semi-supervised logistic model for classification in the context of functional data analysis. Unknown parameters in our proposed model are estimated by regularization with the help of EM algorithm. A …

Paper proposes a new method for estimating conditional densities using logistic regressions.

Novel oracle-type inequality for logistic loss in DNNs achieves sharp convergence rates.

We analyze the optimization landscape of α-loss in logistic models.

PSLR classifies functional data with scalar covariates using path signatures.

Logistic regression gets a new, simpler uniform bound.

The article describe the model, derivation, and implementation of variational Bayesian inference for linear and logistic regression, both with and without automatic relevance determination. It has the dual function of acting as a tutorial for the derivation of variational Bayesian inference for simple models, as well a…

Introduces Fitzpatrick losses, tighter than Fenchel-Young losses.

We consider the problem of learning Relational Logistic Regression (RLR). Unlike standard logistic regression, the features of RLRs are first-order formulae with associated weight vectors instead of scalar weights. We turn the problem of learning RLR to learning these vector-weighted formulae and develop a learning alg…

Improved variational inference for logistic regression and classification.

The paper proves neural networks' consistency and optimal convergence rates for various function classes.

Introduces Soft-SVM for binary classification bridging logistic and SVM.

In this work we propose to fit a sparse logistic regression model by a weakly convex regularized nonconvex optimization problem. The idea is based on the finding that a weakly convex function as an approximation of the $\ell_0$ pseudo norm is able to better induce sparsity than the commonly used $\ell_1$ norm. For a cl…

This work proposes the Bregman-Tweedie classification model and analyzes the domain structure of the extended exponential function, an extension of the classic generalized exponential function with additional scaling parameter, and related high-level mathematical structures, such as the Bregman-Tweedie loss function an…

This paper concerns a method of selecting a subset of features for a sequential logit model. Tanaka and Nakagawa (2014) proposed a mixed integer quadratic optimization formulation for solving the problem based on a quadratic approximation of the logistic loss function. However, since there is a significant gap between …

Paper explains learning property of logistic and softmax losses for balanced and imbalanced class data.

Improved multiclass logistic regression with lower computational complexity.

New algorithms reduce regret in neural logistic bandits.

A combined model integrates latent factor and logistic regression for citation network analysis.

Efficient RL algorithm for multinomial logistic MDPs with provable guarantees.

SLOE speeds up logistic regression in high dimensions with accurate signal strength estimation.

New algorithm reduces regret for logistic bandits without $κ$ dependency.

This paper provides an overview of activation functions in neural networks.

Proposes a new loss function for robust learning.

The article compares neural networks and logistic regression for credit scoring and introduces a new probability calibration technique.

Machine learning improves joint default assessment by capturing non-linear dependencies.

Proposes methods to estimate posterior probability and propensity score functions without assuming constant propensity score.

The paper reveals the hidden costs of digitizing commodity money and proposes a new stable-coin system.

New framework optimizes classification trees with logistic loss and $\ell_1$ regularization.

A new learning method using hyperbolic geometry for class labels.

We revisit logistic regression and its nonlinear extensions, including multilayer feedforward neural networks, by showing that these classifiers can be viewed as converting input or higher-level features into Dempster-Shafer mass functions and aggregating them by Dempster's rule of combination. The probabilistic output…

Estimates non-parametric logistic model using case-control data and external summary info.

This work improves regret minimization for logistic bandits by reducing dependence on a large constant.

Consider semi-supervised learning for classification, where both labeled and unlabeled data are available for training. The goal is to exploit both datasets to achieve higher prediction accuracy than just using labeled data alone. We develop a semi-supervised logistic learning method based on exponential tilt mixture m…

We present $α$-loss, $α\in [1,\infty]$, a tunable loss function for binary classification that bridges log-loss ($α=1$) and $0$-$1$ loss ($α= \infty$). We prove that $α$-loss has an equivalent margin-based form and is classification-calibrated, two desirable properties for a good surrogate loss function for the ideal y…

This article addresses the problem of classification method based on both labeled and unlabeled data, where we assume that a density function for labeled data is different from that for unlabeled data. We propose a semi-supervised logistic regression model for classification problem along with the technique of covariat…

Following the publication of an attack on genome-wide association studies (GWAS) data proposed by Homer et al., considerable attention has been given to developing methods for releasing GWAS data in a privacy-preserving way. Here, we develop an end-to-end differentially private method for solving regression problems wi…

This paper proposes a distributionally robust approach to logistic regression. We use the Wasserstein distance to construct a ball in the space of probability distributions centered at the uniform distribution on the training samples. If the radius of this ball is chosen judiciously, we can guarantee that it contains t…

Unified surrogate loss framework for multi-label learning with strong consistency guarantees.

The paper establishes convergence rates for MoE models in classification problems.

Flexible Bayesian approach for generalized linear models, especially for sparse logistic regression.

New loss functions based on f-divergences improve language model performance.

In this work, we consider a manufactory process which can be described by a multiple-instance logistic regression model. In order to compute the maximum likelihood estimation of the unknown coefficient, an expectation-maximization algorithm is proposed, and the proposed modeling approach can be extended to identify the…

The study examines how class imbalance impacts logistic regression models in low-default credit portfolios.