Sharp bounds on uniform generalization errors in binary linear classification.
problem Understanding the uniform generalization errors in binary linear classification.
method Isoperimetric arguments, Poincaré and log-Sobolev inequalities for joint distributions.
result Sharp concentration bounds on uniform generalization errors, almost sure convergence in broad settings.
Regularized linear regression improves binary classification performance, especially with ridge and ℓ1 regularization.
problem Improving binary classification accuracy with noisy labels.
method Systematic study of regularization strengths on linear classifiers trained on noisy binary classification data.
result Ridge regression consistently improves classification error, while ℓ1 regularization can induce sparsity and ℓ∞ regularization can concentrate weights to two values. The paper analyzes how overparameterized models can generalize well in multiclass classification.
problem Generalization in multiclass classification with overparameterized models.
method Survival/contamination analysis framework adapted for multiclass classification.
result Multiclass classification can generalize well even with many classes, unlike regression tasks.
New test for SGD in binary classification reduces computation time.
problem Determining optimal stopping for SGD in binary classification.
method Proposes a new, simple, computationally inexpensive termination criterion for SGD.
result Termination criterion reduces expected misclassification probability.
Batch normalization biases linear models towards uniform margins, improving performance in binary classification.
problem Understanding the implicit bias of batch normalization in linear models and neural networks.
method Analyzing gradient descent convergence on linear models and two-layer CNNs with batch normalization.
result Gradient descent with batch normalization in linear models converges to a uniform margin classifier with an exponential convergence rate.
For the problem of binary linear classification and feature selection, we propose algorithmic approaches to classifier design based on the generalized approximate message passing (GAMP) algorithm, recently proposed in the context of compressive sensing. We are particularly motivated by problems where the number of feat…
Sharp bounds on binary model inference performance.
problem High-dimensional inference in binary models.
method Convex empirical risk minimization, sharp asymptotics, optimal performance bounds.
result Sharp predictions and optimal performance bounds for binary models.
Paper proposes GEG to enhance fairness in binary and multi-class classification.
problem Fairness in multi-class classification tasks is under-explored.
method Formulates multi-objective problem between effectiveness and fairness constraints, proposes GEG algorithm.
result GEG improves fairness up to 92% and decreases accuracy up to 14%.
Study shows squared loss is robust for binary classification with noisy features.
problem Learning binary classifiers with noisy features.
method Sy-De and Asy-In attribute noise models, analyzing l0−1 and lsq losses. result Squared loss is robust in Sy-De model but not in Asy-In model.
Model reveals double descent in binary linear classification.
problem Investigating classification error in high-dimensional binary linear classification.
method Gradient descent on logistic loss, maximum-likelihood, max-margin (SVM) solutions, and convex Gaussian min-max theorem.
result Double descent phenomenon observed in classification error for varying overparameterization ratio.
New algorithm optimizes AUC in binary classification and changepoint detection.
problem Difficult to optimize AUC in binary classification and changepoint detection.
method Proposes efficient path-following algorithms for choosing optimal learning rate.
result Proposed line search algorithm computes complete AUM/AUC representation.
Paper improves DP-ERM for binary linear classification with large-margin subsets.
problem Differentially private binary linear classification with large-margin subsets.
method Efficient (ε,δ)-DP algorithm with empirical zero-one risk bound. result Improved empirical zero-one risk bound for binary linear classification.
We show linear XOR classification is possible and propose equality separation for anomaly detection.
problem Linearly separating XOR data.
method Equality separation, adapting SVM objective for data within/outside margin.
result Equality separation can detect both seen and unseen anomalies.
Proposes a method to classify binary data from multiple unlabeled datasets.
problem High annotation costs in training classifiers from weakly supervised data.
method Introduces surrogate set classification (SSC) to predict data origin from multiple unlabeled datasets, then uses this to train a binary classifier.
result Demonstrates superior performance compared to existing methods.
We introduce Supersparse Linear Integer Models (SLIM) as a tool to create scoring systems for binary classification. We derive theoretical bounds on the true risk of SLIM scoring systems, and present experimental results to show that SLIM scoring systems are accurate, sparse, and interpretable classification models.
Paper introduces fair GLMs with convex penalty for equalizing GLM outcomes.
problem Achieving fairness in GLMs for practical use.
method Two fairness criteria based on GLM outcomes/log-likelihoods, achieved via a convex penalty on linear components.
result The fair GLM estimator is efficient and can handle various response variables.
Paper introduces r-DEP classifier for binary classification tasks.
problem No natural ordering for feature patterns in practical situations.
method Introduces reduced dilation-erosion (r-DEP) classifier using multi-valued mathematical morphology.
result r-DEP classifiers outperform traditional SVCs in balanced accuracy.
Improves binary classification from positive data with skewed confidence.
problem Skewed confidence in positive data affects the performance of Pconf classifiers.
method Parameterized model of skewed confidence and hyperparameter selection.
result Proposed method effectively cancels out the negative impact of skewed confidence.
This thesis formalizes metric selection for machine learning applications.
problem Selecting the right performance metric for machine learning models.
method Metric Elicitation framework using pairwise preference feedback.
result Novel strategies for eliciting various classification metrics robust to noise.
Study shows fine-tuned linear models outperform pretrained ones in transfer learning.
problem Transfer learning and fine-tuning in linear models for regression and binary classification.
method Stochastic gradient descent on pretrained linear models with small target data sets.
result Fine-tuned models outperform pretrained ones under certain conditions.
Quantum machine learning model for binary classification.
problem Efficiency in high-dimensional binary classification tasks.
method Quantum-classical hybrid algorithm and quantum computer for inference.
result Quantum discriminator achieves 99% accuracy on Iris dataset.
Given a binary prediction problem, which performance metric should the classifier optimize? We address this question by formalizing the problem of Metric Elicitation. The goal of metric elicitation is to discover the performance metric of a practitioner, which reflects her innate rewards (costs) for correct (incorrect)…
Grokking occurs in simple binary logistic classification near linear separability and noise.
problem Delayed generalization in binary logistic classification.
method Analytical and empirical insights into gradient descent dynamics near critical points.
result Logistic regression exhibits grokking when training data is nearly linearly separable from the origin with strong noise.
The paper develops bounds for predictive values in binary classification.
problem Lack of confidence intervals for positive and negative predictive values.
method Bi-criterion framework and distribution-free large deviation and uniform convergence bounds.
result New bounds for predictive values without relying on concentration inequalities.
Our paper examines binary linear classification under Gaussian mixtures, revealing conditions for optimal performance.
problem Understanding the conditions for optimal performance of binary linear classifiers under Gaussian mixtures.
method We study max-margin SVM and min-norm interpolating classifiers, deriving bounds and conditions for optimal performance.
result Interpolating estimators achieve asymptotically optimal performance under certain conditions, emphasizing the role of SNR and covariance.
01 loss is robust to outliers and adversarial attacks, outperforming convex losses.
problem Improving robustness in binary classification models.
method Developed stochastic coordinate descent algorithms for 01 loss classifiers and neural networks.
result 01 loss classifiers are more robust to adversarial attacks compared to convex losses.
Paper proposes methods to improve SVM classifiers in noisy data scenarios.
problem Improving SVM classifiers when training data contains label noise.
method Mixed Integer Linear and Non Linear models with relabeling and clustering.
result Effective methods improve SVM performance in noisy data scenarios.
Paper uses Sugeno integral for binary classification with ordinal data.
problem Learning from ordinal data in machine learning.
method Proposes a method using the Sugeno integral for binary classification, transforming feature values into local evaluations and tuning a threshold.
result Generalizes approach to k-maxitive capacities to control classifier flexibility and prevent overfitting. Introduces Soft-SVM for binary classification bridging logistic and SVM.
problem Data separability issues in binary classification.
method Soft-SVM regression using convex relaxation of hinge loss with softness and class-separation parameters.
result Soft-SVM performs well in classification and prediction errors.
Compressive Sensing (CS) theory asserts that sparse signal reconstruction is possible from a small number of linear measurements. Although CS enables low-cost linear sampling, it requires non-linear and costly reconstruction. Recent literature works show that compressive image classification is possible in CS domain wi…
Linear classifiers can resist adversarial attacks on Gaussian data.
problem Adversarial attacks on high-dimensional data.
method Adversarial training of linear classifiers on Gaussian data.
result Linear classifiers can resist adversarial attacks on Gaussian data.
Classification with a large number of classes is a key problem in machine learning and corresponds to many real-world applications like tagging of images or textual documents in social networks. If one-vs-all methods usually reach top performance in this context, these approaches suffer from a high inference complexity…
The paper explores how to select data points for optimal learning performance.
problem Optimizing data selection for empirical risk minimizers.
method Fixing a learning rule and focusing on optimizing the training data selection.
result Achieving performance comparable to training on the entire population with a small subset of data points.
The paper optimizes hyperplanes for binary classification in high-dimensional data with latent Gaussian mixtures.
problem Binary classification in high-dimensional data with latent Gaussian mixtures.
method Generalized least squares estimator for estimating the direction of the optimal separating hyperplane. Simple correction for intercept estimation.
result The procedure is minimax optimal in many scenarios and can retain the interpolation property.
Study calibrates high-dimensional binary classifiers using angle between estimator and true weights.
problem Calibrating high-dimensional binary classifiers with provable properties.
method Interpolates with a chance classifier to construct well-calibrated predictor based on angle between estimator and true weights.
result Angular calibration approach is provably well-calibrated in high dimensions, minimizing Bregman divergence.
A new algorithm finds a separating hyperplane with fewer updates.
problem Finding a separating hyperplane with minimal updates.
method Optimistic Perceptron algorithm.
result The Optimistic Perceptron finds a separating hyperplane with no more than $ rac{1}{γ}$ updates.
A new algorithm reduces time complexity for binary time series classification.
problem High time complexity of ensemble shapelet transform limits its application.
method Introduces short isometric shapelet transform with two strategies: fixed shapelet length and single linear classifier.
result Demonstrates superior performance and reduced time complexity.
This research sets limits on how complex multi-class learning problems can be.
problem Understanding the complexity of multi-class classification problems.
method Established upper bounds on Natarajan dimensions for specific function classes.
result Upper bounds on Natarajan dimensions for multi-class decision trees, random forests, and neural networks.
This paper uses MIO to select features for kernel SVM classification.
problem Feature selection for kernel SVM classification.
method Mixed-integer optimization (MIO) for feature subset selection.
result The MIO approach can often outperform linear-SVM-based methods in prediction performance.
This study uses neural networks to solve interpolation problems with sparse, infinitely wide layers.
problem Exact data interpolation using sparse, infinitely wide neural networks.
method Atomic norm framework to derive convex hulls and equivalent convex formulations.
result Simple characterizations of convex hulls for different constraints on network weights and biases.
Recent studies in the literature have paid much attention to the sparsity in linear classification tasks. One motivation of imposing sparsity assumption on the linear discriminant direction is to rule out the noninformative features, making hardly contribution to the classification problem. Most of those work were focu…
Ridge regression shows different behaviors in binary classification with noisy labels.
problem Binary classification with noisy labels and anisotropic cluster distributions.
method Investigation of ridge regression behavior in overparameterized settings with label noise.
result Ridge regression exhibits qualitatively different behavior based on the scale of cluster mean vectors and covariance matrices.
The paper provides theoretical guarantees for neural network-based anomaly detection.
problem Theoretical guarantees for unsupervised neural network-based anomaly detection.
method Casting anomaly detection as a binary classification problem, establishing non-asymptotic upper bounds and convergence rates.
result The convergence rate on the excess risk matches the minimax optimal rate.
Probabilistic learning for binary classification with categorical variables.
problem Binary classification with categorical covariates.
method Probabilistic analysis and two algorithms for learning boolean functions.
result Effective learning of boolean functions from binary data.
Analyzes double descent in binary classification models with different losses.
problem Understanding the double descent phenomenon in binary classification models.
method Analytic study of gradient descent with logistic and square losses on binary linear classification models.
result The double descent phenomenon persists but with differences compared to logistic loss.
We propose a method for maximizing a partial area under a receiver operating characteristic (ROC) curve (pAUC) for binary classification tasks. In binary classification tasks, accuracy is the most commonly used as a measure of classifier performance. In some applications such as anomaly detection and diagnostic testing…
Risk bounds for Classification and Regression Trees (CART, Breiman et. al. 1984) classifiers are obtained under a margin condition in the binary supervised classification framework. These risk bounds are obtained conditionally on the construction of the maximal deep binary tree and permit to prove that the linear penal…
Improved text classification performance through conformal transformations of kernels.
problem Text document categorization in high-dimensional spaces.
method Introduced new Gaussian Cosine kernel and two conformal transformations.
result Conformal transformations significantly improve kernel performance, especially for sub-optimal kernels.