arXiv research

Paper introduces MRCs that minimize worst-case 0-1 loss, providing tight performance guarantees.

MRCs minimize worst-case expected 0-1 loss and provide performance guarantees.

New solver for MKL-SVM with 0/1 loss function.

Paper introduces MKL-$L_{0/1}$-SVM for SVM with $(0, 1)$ loss.

Deep neural networks (DNNs) have great expressive power, which can even memorize samples with wrong labels. It is vitally important to reiterate robustness and generalization in DNNs against label corruption. To this end, this paper studies the 0-1 loss, which has a monotonic relationship with an empirical adversary (r…

A new SVM classifier using $L_{0/1}$ soft-margin loss for improved performance.

This research analyzes the consistency of convex and nonconvex surrogate losses for adversarially robust classification.

Study shows squared loss is robust for binary classification with noisy features.

ICE algorithm solves exact 0-1 loss linear classification problem efficiently.

Characterizes learnability of forgiving 0-1 loss functions in multiclass settings.

We present $α$-loss, $α\in [1,\infty]$, a tunable loss function for binary classification that bridges log-loss ($α=1$) and $0$-$1$ loss ($α= \infty$). We prove that $α$-loss has an equivalent margin-based form and is classification-calibrated, two desirable properties for a good surrogate loss function for the ideal y…

In classification, the de facto method for aggregating individual losses is the average loss. When the actual metric of interest is 0-1 loss, it is common to minimize the average surrogate loss for some well-behaved (e.g. convex) surrogate. Recently, several other aggregate losses such as the maximal loss and average t…

Active learning is a type of sequential design for supervised machine learning, in which the learning algorithm sequentially requests the labels of selected instances from a large pool of unlabeled data points. The objective is to produce a classifier of relatively low risk, as measured under the 0-1 loss, ideally usin…

This paper improves SAM by reformulating it as a bilevel optimization problem.

Paper proves L2 regression can learn k-juntas without distributional assumptions.

Conventional techniques for supervised classification constrain the classification rules considered and use surrogate losses for classification 0-1 loss. Favored families of classification rules are those that enjoy parametric representations suitable for surrogate loss minimization, and low complexity properties suita…

Paper unifies bias and variance models for classification.

Classification and regression tasks in overparameterized models show different generalization properties.

The paper sets lower bounds for adversarial robustness in multiclass classification.

Develops gradient boosting for multi-label classification.

Given a task of predicting $Y$ from $X$, a loss function $L$, and a set of probability distributions $Γ$ on $(X,Y)$, what is the optimal decision rule minimizing the worst-case expected loss over $Γ$? In this paper, we address this question by introducing a generalization of the principle of maximum entropy. Applying t…

New method improves transductive learning predictions with multiplicative oracle inequalities.

New criterion assesses cluster separability for validation.

Deep neural networks outperform traditional methods in high-dimensional classification.

We analyze the optimization landscape of α-loss in logistic models.

The study establishes SQ lower bounds for learning halfspaces and ReLUs under Gaussian marginals.

SGLB boosts machine learning with Langevin diffusion for multimodal loss functions.

We provide novel theoretical insights on structured prediction in the context of efficient convex surrogate loss minimization with consistency guarantees. For any task loss, we construct a convex surrogate that can be optimized via stochastic gradient descent and we prove tight bounds on the so-called "calibration func…

New loss function calibrates WW-hinge loss for multiclass SVM.

This paper quantifies and mitigates a bias in the Hayashi-Yoshida estimator causing data loss.

Optimal domain adaptation model using Fisher's Linear Discriminant.

This research improves PAC-Bayesian bounds for classification tasks using convexified loss.

New PAC-Bayes bounds for unbounded loss functions.

We introduce a tunable loss function called $α$-loss, parameterized by $α\in (0,\infty]$, which interpolates between the exponential loss ($α= 1/2$), the log-loss ($α= 1$), and the 0-1 loss ($α= \infty$), for the machine learning setting of classification. Theoretically, we illustrate a fundamental connection between $…

We study online aggregation of the predictions of experts, and first show new second-order regret bounds in the standard setting, which are obtained via a version of the Prod algorithm (and also a version of the polynomially weighted average algorithm) with multiple learning rates. These bounds are in terms of excess l…

Boosting with tempered exponential measures improves AdaBoost's convergence rate.

Decentralized algorithm for online optimization with long-term constraints.

We prove that, under low noise assumptions, the support vector machine with $N\ll m$ random features (RFSVM) can achieve the learning rate faster than $O(1/\sqrt{m})$ on a training set with $m$ samples when an optimized feature map is used. Our work extends the previous fast rate analysis of random features method from…

The majority of traditional classification ru les minimizing the expected probability of error (0-1 loss) are inappropriate if the class probability distributions are ill-defined or impossible to estimate. We argue that in such cases class domains should be used instead of class distributions or densities to construct …

Extends graph similarity theory to improve MPNNs' generalization abilities.

Proposes a hierarchical curriculum loss to improve model accuracy and interpretability.

The paper proves neural networks' consistency and optimal convergence rates for various function classes.

A new method learns proper multiclass losses and probabilities.

Denoising autoencoders (DAEs) are powerful deep learning models used for feature extraction, data generation and network pre-training. DAEs consist of an encoder and decoder which may be trained simultaneously to minimise a loss (function) between an input and the reconstruction of a corrupted version of the input. The…

This paper investigates Pareto optimal (PO, for short) insurance contracts in a behavioral finance framework, in which the insured evaluates contracts by the rank-dependent utility (RDU) theory and the insurer by the expected value premium principle. The incentive compatibility constraint is taken into account, so the …

We study consistency properties of surrogate loss functions for general multiclass learning problems, defined by a general multiclass loss matrix. We extend the notion of classification calibration, which has been studied for binary and multiclass 0-1 classification problems (and for certain other specific learning pro…

Deep neural networks achieve optimal classification rates in high dimensions.

In this paper we refine the process of computing calibration functions for a number of multiclass classification surrogate losses. Calibration functions are a powerful tool for easily converting bounds for the surrogate risk (which can be computed through well-known methods) into bounds for the true risk, the probabili…