arXiv research

Label smoothing improves model performance even with noisy labels.

Label smoothing improves generalization by controlling generalization loss.

The generalization and learning speed of a multi-class neural network can often be significantly improved by using soft targets that are a weighted average of the hard targets and the uniform distribution over labels. Smoothing the labels in this way prevents the network from becoming over-confident and label smoothing…

Pairwise Label Smoothing improves deep model generalization by reducing overconfidence.

Paper introduces SLS to improve label smoothing regularization.

Improved OOD detection using label smoothing and k-NN density estimates.

Improves confidence calibration in neural networks by smoothing labels based on class similarity.

We study Label-Smoothing as a means for improving adversarial robustness of supervised deep-learning models. After establishing a thorough and unified framework, we propose several variations to this general method: adversarial, Boltzmann and second-best Label-Smoothing methods, and we explain how to construct your own…

Enhances deep networks robustness with data mollification and label smoothing.

In this article, we mathematically study several GAN related topics, including Inception score, label smoothing, gradient vanishing and the -log(D(x)) alternative. --- An advanced version is included in arXiv:1703.02000 "Activation Maximization Generative Adversarial Nets". Please refer Section 6 in 1703.02000 for deta…

The study examines label smoothing to improve confidence calibration in fine-tuned LLMs.

Study analyzes label smoothing in deep learning optimization.

Self-distillation improves model performance by increasing teacher diversity and smoothing predictions.

GS-B$^3$SE improves label shift estimation by smoothing priors on a graph.

Label smoothing improves model robustness against misspecification.

Knowledge graphs capture structured information and relations between a set of entities or items. As such knowledge graphs represent an attractive source of information that could help improve recommender systems. However, existing approaches in this domain rely on manual feature engineering and do not allow for an end…

Proposes a data augmentation method to improve multi-label learning performance.

Method reweights instances and classes to improve robustness in noisy data.

Practically, we are often in the dilemma that the labeled data at hand are inadequate to train a reliable classifier, and more seriously, some of these labeled data may be mistakenly labeled due to the various human factors. Therefore, this paper proposes a novel semi-supervised learning paradigm that can handle both l…

New method makes machine learning models robust to label flipping attacks.

The recently proposed self-ensembling methods have achieved promising results in deep semi-supervised learning, which penalize inconsistent predictions of unlabeled data under different perturbations. However, they only consider adding perturbations to each single data point, while ignoring the connections between data…

Label assignment problems with large state spaces are important tasks especially in computer vision. Often the pairwise interaction (or smoothness prior) between labels assigned at adjacent nodes (or pixels) can be described as a function of the label difference. Exact inference in such labeling tasks is still difficul…

Proposes sigmoidF1 loss for multilabel classification, improving performance metrics.

This work proposes a novel method for semi-supervised learning from partially labeled massive network-structured datasets, i.e., big data over networks. We model the underlying hypothesis, which relates data points to labels, as a graph signal, defined over some graph (network) structure intrinsic to the dataset. Follo…

Learning with Label Proportions (LLP) is the problem of recovering the underlying true labels given a dataset when the data is presented in the form of bags. This paradigm is particularly suitable in contexts where providing individual labels is expensive and label aggregates are more easily obtained. In the healthcare…

Logit regularization induces logit clustering, affecting classifier performance.

IDS improves RLHF by smoothing reward data, enhancing model performance.

Simple method improves deep classifier accuracy under noisy labels.

Adversarial training is one of the strongest defenses against adversarial attacks, but it requires adversarial examples to be generated for every mini-batch during optimization. The expense of producing these examples during training often precludes adversarial training from use on complex image datasets. In this study…

A new unsupervised contrastive learning framework improves time series representation learning.

New method calibrates multi-class predictions efficiently without sacrificing accuracy.

Recently, a variety of regularization techniques have been widely applied in deep neural networks, such as dropout, batch normalization, data augmentation, and so on. These methods mainly focus on the regularization of weight parameters to prevent overfitting effectively. In addition, label regularization techniques su…

CCVAE captures label characteristics in VAEs for better representation learning.

PML-LFC improves PML by estimating label confidence from both feature and label spaces.

In applications of supervised learning applied to medical image segmentation, the need for large amounts of labeled data typically goes unquestioned. In particular, in the case of brain anatomy segmentation, hundreds or thousands of weakly-labeled volumes are often used as training data. In this paper, we first observe…

Let $M$ be a smooth closed orientable surface, and let $F$ be the space of Morse functions on $M$ such that at least $χ(M)+1$ critical points of each function of $F$ are labeled by different labels (enumerated). Endow the space $F$ with $C^\infty$-topology. We prove the homotopy equivalence $F\sim R\times{\widetilde{\c…

We propose a supervised anomaly detection method for data with inexact anomaly labels, where each label, which is assigned to a set of instances, indicates that at least one instance in the set is anomalous. Although many anomaly detection methods have been proposed, they cannot handle inexact anomaly labels. To measur…

CutMix training technique improves spatial locality in Vision Transformers.

SGD handles label noise with bounds improving over SGLD.

Noisy labels often occur in vision datasets, especially when they are obtained from crowdsourcing or Web scraping. We propose a new regularization method, which enables learning robust classifiers in presence of noisy data. To achieve this goal, we propose a new adversarial regularization scheme based on the Wasserstei…

We propose a new regularization method based on virtual adversarial loss: a new measure of local smoothness of the conditional label distribution given input. Virtual adversarial loss is defined as the robustness of the conditional label distribution around each input data point against local perturbation. Unlike adver…

AUC-spec optimizes graph-based SSL for complex label distributions.

Unified surrogate loss framework for multi-label learning with strong consistency guarantees.

We describe an adversarial learning approach to constrain convolutional neural network training for image registration, replacing heuristic smoothness measures of displacement fields often used in these tasks. Using minimally-invasive prostate cancer intervention as an example application, we demonstrate the feasibilit…

We apply the network Lasso to classify partially labeled data points which are characterized by high-dimensional feature vectors. In order to learn an accurate classifier from limited amounts of labeled data, we borrow statistical strength, via an intrinsic network structure, across the dataset. The resulting logistic …

In this paper we study the existence of solutions for a class of non-linear differential equation on compact Riemannian manifolds. We establish a lower and upper solutions' method to show the existence of a smooth positive solution for the equation (EQ1) \begin{equation} \label{E4} Δu \ + \ a(x)u \ = \ f(x)F(u) \ + \ h…

Private algorithms adapt from public to private domains with minimal labeled data.

Label noise is emerging as a pressing issue in sound event classification. This arises as we move towards larger datasets that are difficult to annotate manually, but it is even more severe if datasets are collected automatically from online repositories, where labels are inferred through automated heuristics applied t…