The paper analyzes how overparameterized models can generalize well in multiclass classification.
problem Generalization in multiclass classification with overparameterized models.
method Survival/contamination analysis framework adapted for multiclass classification.
result Multiclass classification can generalize well even with many classes, unlike regression tasks.
Improved mistake bound for group linear separable cases in online multiclass linear classification.
problem Improving mistake bounds for online multiclass linear classification under group linear separable conditions.
method Refined group weak linear separability condition and rational kernel approach.
result Achieved a mistake bound of K ⋅ 2 i l d e O ( 1 / γ log L ) ) K\cdot 2^{ ilde{O}(\sqrt{1/γ}\log L)}) K ⋅ 2 i l d e O ( 1/ γ l o g L ) ) under group weak linear separable condition. New method for multiclass classification reduces error bounds.
problem High-dimensional multiclass classification with sparse coefficients.
method Sparse multinomial logistic regression with convex penalties.
result Plug-in classifiers achieve minimax generalization error bounds.
Study shows SQ hardness for multiclass linear classification with random noise.
problem Complexity of multiclass linear classification with random noise.
method Proves super-polynomial SQ lower bounds for MLC with RCN.
result Super-polynomial SQ lower bounds for MLC with RCN.
Paper analyzes multiclass classification with high-dimensional data.
problem Understanding statistical properties and behavior of multiclass classification algorithms.
method Asymptotic analysis of linear multiclass classification.
result Test error varies significantly across different training algorithms and data distributions.
One-bit quantization and sparsification improve multiclass classification with strong regularization.
problem Overfitting mislabeled data in multiclass classification.
method Linear regression with regularization and one-bit quantization/sparsification.
result Sparse and one-bit solutions perform almost as well as the optimal solution with f ( ⋅ ) = ∥ ⋅ ∥ 2 2 f(\cdot) = \|\cdot\|_2^2 f ( ⋅ ) = ∥ ⋅ ∥ 2 2 . Study resolves conjecture on overparameterized linear models' generalization.
problem Asymptotic generalization of multiclass classification with overparameterized models.
method Gaussian covariates bi-level model, Hanson-Wright inequality variant.
result Min-norm interpolating classifier can be suboptimal compared to noninterpolating classifiers.
New algorithms compute robustness bounds for multiclass classification models.
problem Computing robustness of deep learning models in multiclass classification.
method Optimal transport and linear programming/entropic regularization.
result Tractable algorithms for computing robustness bounds.
Gaptron algorithm reduces mistakes in online multiclass classification.
problem Online multiclass classification with limited information.
method Randomized first-order algorithm exploiting the gap between zero-one loss and surrogate losses.
result First linear time algorithm with O ( K T ) O(K\sqrt{T}) O ( K T ) expected regret. This thesis formalizes metric selection for machine learning applications.
problem Selecting the right performance metric for machine learning models.
method Metric Elicitation framework using pairwise preference feedback.
result Novel strategies for eliciting various classification metrics robust to noise.
New research shows SVM and related methods can overfit without harm in multiclass classification.
problem Understanding benign overfitting in multiclass classification.
method Analyzing three training algorithms: ERM with cross-entropy, least-squares, and one-vs-all SVM.
result All three algorithms can lead to classifiers that interpolate training data and have equal accuracy under high overparameterization.
Study expands multiclass classification models with new rates and partial concept classes.
problem Multiclass classification with a bounded number of labels under various conditions.
method Extends traditional PAC model to distribution-dependent and data-dependent learning rates, characterizes optimal rates for universal and partial concept classes.
result Characterizes three types of learning rates (exponential, linear, arbitrarily slow) for fixed distributions and complexity measures for partial concept classes.
Unified binary and multiclass margin-based classification methods.
problem No consensus on multiclass loss functions analogous to binary margin loss.
method Showed multiclass loss functions can be expressed in relative margin form.
result Extended classification-calibration result to multiclass.
Study optimal rates for multiclass classification, resolving open questions.
problem Optimal rates for multiclass classification with any label space.
method Establishes optimal rates for all hypothesis classes, defining new tree structures.
result Optimal rates for multiclass classification with no infinite DSL trees.
Unified approach to multiclass classification using Gabriel graphs.
problem Improving multiclass classification accuracy and efficiency.
method Integrates Gabriel graphs for binary and multiclass classification, proposing new activation functions and support edge neurons.
result Experimental results show superior performance compared to previous GG-based classifiers.
Proposes a new method to improve multiclass probability calibration.
problem Uncalibrated class probabilities in multiclass classifiers leading to over-confidence.
method Dirichlet calibration method applicable to any model class, derived from Dirichlet distributions.
result Improved probabilistic predictions across various datasets and classifiers.
Enhanced word embeddings boost multiclass text classification accuracy.
problem Improving multiclass text classification accuracy using pre-trained embeddings.
method Proposed word-class embeddings (WCEs) to enhance pre-trained word embeddings.
result WCEs significantly improve multiclass text classification accuracy.
In this paper, we present a novel approach to construct multiclass classifiers by means of arrangements of hyperplanes. We propose different mixed integer (linear and non linear) programming formulations for the problem using extensions of widely used measures for misclassifying observations where the \textit{kernel tr…
New research determines the optimal sample complexity for multiclass and list learning.
problem Determining the optimal sample complexity for multiclass classification.
method Algebraic characterization of multiclass hypothesis classes in terms of their DS dimension.
result Proves a longstanding conjecture and determines the optimal dependence of sample complexity on DS dimension.
Study shows exponential error reduction in multiclass classification without bias-variance trade-off.
problem Multiclass classification with margin conditions.
method Analysis of classification error under hard-margin conditions.
result Exponential decrease in classification error without bias-variance trade-off.
Establishes a condition for multiclass classification-calibration of Gamma-Phi losses.
problem Ensuring classification-calibration of multiclass Gamma-Phi losses.
method Develops a general sufficient condition for classification-calibration of Gamma-Phi losses.
result Proves the first family of nonconvex multiclass surrogate losses for which classification-calibration has been fully justified.
New reformulations for multiclass classification problems using optimal transport.
problem Adversarial multiclass classification problems.
method Multimarginal optimal transport formulation.
result Reveals geometric structure and extends binary classification results.
Paper studies multiclass classifiers from binary classifiers, proving methods and demonstrating advantages.
problem Constructing efficient multiclass classifiers from binary ones.
method Two methods: one vs. all and hierarchical classification, with a new leverage-hierarchical method introduced.
result Proves upper bounds and exact formulas for multiclass regret in terms of binary regrets.
This work extends implicit bias analysis to multiclass classification using a new loss framework.
problem The implicit bias of gradient descent on multiclass data without explicit regularization.
method Employing the PERM framework to introduce a multiclass extension of the exponential tail property.
result Extended implicit bias result to multiclass classification using a new loss framework.
FOLKLORE algorithm speeds up online multiclass logistic regression.
problem Efficiently solving online multiclass logistic regression without high computational cost.
method Developed FOLKLORE algorithm with improved runtime and regret bound.
result First practical algorithm for online multiclass logistic regression.
Algorithm minimizes risk for multiclass classification of stochastic diffusion paths.
problem Multiclass classification of stochastic diffusion paths with distinct drift functions.
method Empirical risk minimization using L2 risk.
result Achieves fast rates of convergence under margin assumption.
Paper proposes online algorithms for multiclass classification with partial labels.
problem Classifying data with partial labels.
method Avg Perceptron, Max Perceptron, Avg Pegasos, Max Pegasos algorithms.
result Mistake bounds for Avg Perceptron and regret bound for Avg Pegasos.
We study the problem of efficient online multiclass linear classification with bandit feedback, where all examples belong to one of K K K classes and lie in the d d d -dimensional Euclidean space. Previous works have left open the challenge of designing efficient algorithms with finite mistake bounds when the data is linear…
Generalized additive models (GAMs) are favored in many regression and binary classification problems because they are able to fit complex, nonlinear functions while still remaining interpretable. In the first part of this paper, we generalize a state-of-the-art GAM learning algorithm based on boosted trees to the multi…
Study online multiclass classification under bandit feedback, extending previous results.
problem Online multiclass classification with bandit feedback, focusing on label space unboundedness.
method Extend Daniely and Helbertal's results, show necessity and sufficiency of Bandit Littlestone dimension for learnability.
result Sequential uniform convergence is necessary but not sufficient for bandit online learnability.
We develop efficient algorithms to train ℓ 1 \ell_1 ℓ 1 -regularized linear classifiers with large dimensionality d d d of the feature space, number of classes k k k , and sample size n n n . Our focus is on a special class of losses that includes, in particular, the multiclass hinge and logistic losses. Our approach combines several…
Proposes MCLLO for assessing and recalibrating multiclass probability predictions.
problem Limited multicategory recalibration methods for assessing and comparing model calibration.
method MCLLO recalibration method that assesses calibration without model access and is easy to interpret.
result MCLLO outperforms other methods in simulations and real-world case studies.
Extends boosting to multiclass online agnostic classification.
problem Online multiclass classification with weak learners.
method Reduces multiclass online agnostic boosting to online convex optimization.
result First boosting algorithm for online agnostic multiclass classification.
Study proves existence of robust classifiers in multiclass adversarial training.
problem Proves existence of robust classifiers in multiclass adversarial training.
method Three models of adversarial training in multiclass classification, proving existence of Borel measurable robust classifiers.
result Proves existence of Borel measurable robust classifiers in each model.
We propose a robust adversarial prediction framework for general multiclass classification. Our method seeks predictive distributions that robustly optimize non-convex and non-continuous multiclass loss metrics against the worst-case conditional label distributions (the adversarial distributions) that (approximately) m…
New algorithm improves multiclass classification regret bound.
problem Improving multiclass classification with bandit feedback.
method Developed a new bandit classification algorithm.
result Guaranteed regret of O ~ ( ∣ H ∣ + T ) \smash{\widetilde{O}(|H|+\sqrt{T})} O ( ∣ H ∣ + T ) , improving over classical algorithms. In this paper, we tackle the question of discovering an effective set of spatial filters to solve hyperspectral classification problems. Instead of fixing a priori the filters and their parameters using expert knowledge, we let the model find them within random draws in the (possibly infinite) space of possible filters…
New binary classification techniques help multiclass classification by aggregating proper learners.
problem Multiclass classification faces a properness barrier that prevents optimal learning by proper learners.
method Aggregations of proper binary learners, generalized to multiclass settings, achieve optimal sample complexity.
result Optimal binary learners can achieve sample complexity $O\left(\frac{d_G + \ln(1 / δ)}ε
ight)$ for classes with finite Graph dimension d G d_G d G . Develops a functional mix-of-experts model for multiclass classification.
problem Multiclass classification with univariate functional predictors.
method Functional mix-of-experts model with regularization and sparsity constraints.
result Regularized maximum likelihood estimation yields interpretable coefficient functions.
Binary and multiclass epilepsy detection methods using EEG features.
problem Epilepsy diagnosis from EEG data.
method Feature extraction from power spectrum, spectrogram, and bispectrogram; eight machine learning algorithms used.
result Random forest and backpropagation algorithms achieved highest accuracy for binary and multiclass classification.
Develops NPMC method for noisy labels, improving multiclass classification accuracy.
problem Asymmetric misclassification costs and label noise in multiclass classification.
method Empirical likelihood approach using exponential tilting density ratio model.
result Root n consistent and asymptotically normal estimators for clean labels and noise mechanism.
A Fourier-based learning algorithm for multiclass classification.
problem Highly nonlinear multiclass classification problems.
method Smoothing technique with low-pass filters to calculate probability distributions.
result Probabilistic explanation for classification without kernel functions.
A robust multiclass SVM tackles imbalanced data uncertainty.
problem Imbalanced multiclass classification with uncertain labels.
method Wasserstein distributionally robust optimization for multiclass SVM.
result Our model outperforms state-of-the-art OVA models in imbalanced settings.
Improved multiclass classification with class-weighted nearest neighbors.
problem Multiclass classification with large or imbalanced classes.
method Class-weighted k-nearest neighbors algorithm, derived bounds on accuracy and risk.
result Optimized classification metrics like F1 score or Matthew's Correlation Coefficient.
Unified SVM framework tackles multiclass and multilabel classification.
problem Extending SVM to handle multiclass and multilabel problems.
method Unified framework with class-specific weight vectors and penalizing patterns close to an origin.
result Unified framework achieves competitive performance for multiclass and multilabel classification.
Paper tackles noisy bandit feedback for multiclass classification.
problem Learning multiclass classifier with corrupted feedback.
method Proposes an unbiased estimator technique to estimate noise rates and an end-to-end framework.
result Algorithm achieves mistake bounds of O ( T ) O(\sqrt{T}) O ( T ) in high noise and O ( T i c e f r a c 23 ) O(T^{
icefrac{2}{3}}) O ( T i ce f r a c 2 3 ) in worst case. Study builds a classifier for diffusions with unknown diffusion but known drifts.
problem Multiclass classification of S.D.E. paths with unknown diffusion coefficient.
method Plug-in classifier using nonparametric estimators of drift and diffusion functions.
result Consistent classification procedure with rate of convergence under different assumptions.
This work analyzes label embedding for large multiclass classification problems.
problem Label embedding for large multiclass classification problems.
method Analysis of label embedding in extreme multiclass classification, presenting an excess risk bound and showing a trade-off between computational and statistical efficiency.
result The statistical penalty for label embedding vanishes with sufficiently low coherence under the Massart noise condition.