Adaptive label regularization improves neural network performance.
problem Improving deep neural network performance through label regularization.
method Proposes an adaptive label regularization method that updates label representation online.
result Significant improvement in image recognition and text classification tasks.
Contrastive regularization improves semi-supervised learning by better propagating confident pseudo-labels.
problem Consistency regularization's limitation in high performance and efficiency.
method Proposes contrastive regularization to update model features, pushing confident labels into unlabeled samples.
result Improves semi-supervised learning tasks with fewer training iterations and robust performance.
Multi-label text classification is a popular machine learning task where each document is assigned with multiple relevant labels. This task is challenging due to high dimensional features and correlated labels. Multi-label text classifiers need to be carefully regularized to prevent the severe over-fitting in the high …
Paper proposes a new model using consistency regularization for learning from label proportions.
problem Learning from label proportions with weak labels on bags of instances.
method Consistency regularization applied to semi-supervised learning.
result LLP with consistency regularization achieves superior performance.
Noisy labels often occur in vision datasets, especially when they are obtained from crowdsourcing or Web scraping. We propose a new regularization method, which enables learning robust classifiers in presence of noisy data. To achieve this goal, we propose a new adversarial regularization scheme based on the Wasserstei…
Improved diffusion map enhances manifold regularization for semi-supervised learning.
problem Limited performance of manifold regularization models in capturing global structure.
method Enhanced diffusion map with improved label propagation function.
result Proposed method improves manifold regularization model's performance.
Improved fine-tuning with regularization and robustness for noisy labels.
problem Fine-tuning pre-trained models on small datasets can lead to overfitting and memorization.
method PAC-Bayes generalization bound analysis, layer-wise regularization, self-label-correction, label-reweighting.
result Improves performance by 1.76% on average for image classification tasks and 0.75% for few-shot classification.
Label noise in SGD helps converge to flatter minima.
problem Improving generalization in overparametrized models.
method Analyzes SGD with label noise, showing convergence to regularized minima.
result SGD with label noise converges to flatter minima, improving generalization.
Paper introduces SLS to improve label smoothing regularization.
problem Improve generalization of neural networks by softening labels.
method Structural Label Smoothing (SLS) to mitigate bias in Bayes error rate.
result Empirical validation shows SLS outperforms traditional label smoothing.
A fast method for discrete OT with group-sparse regularization for class label preservation.
problem Efficiently measuring the distance between two discrete distributions with class labels.
method Fast discrete OT with group-sparse regularizers using gradient-based algorithms.
result Up to 8.6 times faster than original method without degrading accuracy.
Consistency regularization improves robustness to noisy labels.
problem Improving model robustness to noisy labels in machine learning.
method Empirical study of consistency regularization on noisy datasets.
result Consistency regularization improves model robustness to label noise.
Soft labeling impacts OOD detection in neural networks.
problem Impact of soft labeling on OOD detection in deep neural networks.
method Empirical analysis of how soft labeling affects OOD detection performance.
result Soft labeling can either improve or deteriorate OOD detection performance.
This work compares regularization and constrained inference for label constraints in machine learning.
problem Improving model performance with label constraints in machine learning.
method Comparison of regularization and constrained inference strategies.
result Constrained inference reduces population risk by correcting model violations, while regularization narrows the generalization gap but introduces bias.
This paper proposes a method to select relevant features for multi-label learning.
problem Feature selection in multi-label learning to retain important information with minimal features.
method Random manifold sampling and joint sparse regularization to solve multicollinearity and obtain sparse feature sets.
result The proposed method outperforms other methods in selecting relevant features for multi-label learning.
An important problem in multi-label classification is to capture label patterns or underlying structures that have an impact on such patterns. This paper addresses one such problem, namely how to exploit hierarchical structures over labels. We present a novel method to learn vector representations of a label space give…
No regularization needed for InLDL, achieving efficient and effective model.
problem InLDL struggles with performance degradation due to missing degrees.
method Proposes a model that uses label distribution as a prior, implicitly regularizing the learning process.
result Achieves competitive performance without explicit regularization.
TCR improves DNN robustness to noisy labels with minimal overhead.
problem Training on noisy labeled datasets degrades DNN generalization.
method TCR combines original labels and previous epoch predictions for regularization.
result TCR consistently enhances DNN robustness to label noise.
PSDR improves robustness against noisy labels by penalizing KL divergence between similar inputs.
problem Robust training of DNNs in datasets with noisy labels.
method Introduces PSDR, a manifold regularizer that penalizes KL divergence between similar inputs.
result Significantly improves robustness against noisy labels on benchmark datasets.
Framework prevents deep learning models from memorizing noisy labels.
problem Deep learning models memorize noisy labels during early learning phase.
method Develops a technique that exploits early learning phase via regularization.
result Framework achieves robustness to noisy annotations on benchmarks and real-world datasets.
ALPS improves neural network robustness and generalization.
problem Challenges in designing effective regularization schemes for adversarial robustness.
method Adversarial Labelling of Perturbed Samples (ALPS) using synthetic samples and min-max formulation.
result ALPS achieves state-of-the-art regularization performance and adversarial robustness.
This work improves deep learning from noisy crowdsourced labels.
problem Learning label correction and neural classifier from noisy crowdsourced data.
method Coupled Cross-Entropy Minimization (CCEM) with identifiability and regularization.
result The CCEM criterion correctly identifies annotators' confusion and neural classifier under realistic conditions.
Statistical methods protecting sensitive information or the identity of the data owner have become critical to ensure privacy of individuals as well as of organizations. This paper investigates anonymization methods based on representation learning and deep neural networks, and motivated by novel information theoretica…
Simple method improves deep learning with noisy labels.
problem Deep learning's overfitting to noisy labels.
method Adds a variance regularization term to penalize neural network's Jacobian norm.
result Achieves state-of-the-art performance with high noise tolerance.
JoCoR improves deep learning with noisy labels by reducing network diversity.
problem Learning with noisy labels in deep learning.
method JoCoR uses two networks to make predictions, calculates a joint loss with Co-Regularization, and updates both networks simultaneously.
result JoCoR outperforms state-of-the-art approaches in learning with noisy labels.
RegMixMatch optimizes Mixup for semi-supervised learning by integrating high- and low-confidence samples.
problem Mixup degrades SSL performance by compromising artificial labels purity.
method RegMixMatch integrates high- and low-confidence samples, uses class-aware Mixup, and mitigates confirmation bias.
result RegMixMatch achieves state-of-the-art performance in SSL benchmarks.
We propose Regularized Learning under Label shifts (RLLS), a principled and a practical domain-adaptation algorithm to correct for shifts in the label distribution between a source and a target domain. We first estimate importance weights using labeled source data and unlabeled target data, and then train a classifier …
Pairwise Label Smoothing improves deep model generalization by reducing overconfidence.
problem Improving deep model generalization through regularization.
method PLS smooths labels for pairs of samples, learning distribution mass during training.
result PLS significantly outperforms LS and baseline models, reducing up to 30% classification error.
New method improves deep learning models robustness to label noise.
problem Improving deep learning models' robustness to corrupted labels.
method Sparse over-parameterization and implicit regularization.
result State-of-the-art test accuracy against label noise on various datasets.
Increasing variance of losses improves learning with noisy labels.
problem Learning with noisy labels and the need to penalize variance of losses.
method Designing regularizers based on the label noise transition matrix to increase variance of losses.
result Increasing variance of losses significantly improves generalization ability.
Paper proposes a robust deep graph-based classifier for noisy labels.
problem Difficulty in feature learning with noisy training labels.
method Convolutional neural networks with graph Laplacian regularization (GLR).
result Proposed method outperforms state-of-the-art classifiers on noisy datasets.
Over-parameterized deep neural networks trained by simple first-order methods are known to be able to fit any labeling of data. Such over-fitting ability hinders generalization when mislabeled training examples are present. On the other hand, simple regularization methods like early-stopping can often achieve highly no…
Simple regularization methods mimic adversarial training's robustness.
problem Expensive adversarial training for robustness.
method Label smoothing and logit squeezing.
result Achieves strong adversarial robustness without adversarial examples.
Proposes a meta-learning algorithm to improve semi-supervised learning.
problem Overfitting and limited model generalization in consistency-based semi-supervised learning.
method Introduces a learn-to-generalize regularization term and meta-gradient optimization.
result Demonstrates improved performance on various datasets compared to state-of-the-art methods.
A framework learns dynamic soft labels to improve model generalization and accuracy.
problem Models trained on one-hot labels overfit and are sensitive to noisy annotations.
method Proposes a framework where labels are treated as learnable parameters, adapting dynamically during optimization.
result Consistent gains across different datasets and architectures, improving ResNet18 by 2.1% on CIFAR100.
Logit regularization induces logit clustering, affecting classifier performance.
problem Understanding the mechanism of logit regularization in classification.
method Analysis of logit regularization in linear classification, proving logit clustering leads to Fisher's Linear Discriminant alignment.
result Logit regularization can halve critical sample complexity and induce robust generalization.
A new method classifies high-dimensional images with minimal labels using diffusion geometry.
problem Classifying high-dimensional images efficiently and accurately.
method Spatially-regularized nonlinear diffusion geometry for clustering and active learning.
result High-accuracy labelings achieved with a very small number of training labels.
Paper tackles noisy labels in deep learning networks.
problem Learning with noisy labels in deep neural networks.
method Sparse regularization strategy to approximate one-hot constraint.
result Improves performance of commonly-used loss functions in noisy labels and class imbalance.
Method estimates noise transition matrix from noisy labels without relying on unreliable class-posterior estimation.
problem Estimating noise transition matrix from noisy data.
method Total variation regularization to encourage distinguishable predicted probabilities.
result Consistent estimator of the noise transition matrix under mild assumptions.
Improves domain adaptation by aligning source and target distributions and mitigating noisy labels.
problem Improving performance on target images with different acquisition conditions.
method Combines optimal transport, MixUp regularization, and robust loss for noisy labels.
result Improves domain adaptation performance on various benchmarks and real-world problems.
Binary classification improves with a small fraction of corrupted labels.
problem Binary classification with corrupted labels.
method Established corruption as a form of regularization and computed upper bounds on estimation error.
result Corruption is beneficial only up to a small fraction of the total sample, scaling with the square root of the sample size.
Paper tackles noisy labels by compressing feature representations.
problem Learning with noisy labels leads to overfitting and poor generalization.
method Introduces compression inductive bias using Dropout and Nested Dropout.
result Compression helps in combating label noise and improving performance.
Proposes a regularization method for unsupervised domain adaptation that aligns predictions with target data's top singular vectors.
problem Domain adaptation challenges in high joint error scenarios.
method Regularizes classifier to align with unsupervised target data guided by label alignment property (LAP).
result The method improves performance in MNIST-USPS domain adaptation and cross-lingual sentiment analysis.
End-to-end deep metric learning tackles multi-label image classification.
problem Multi-label image classification problem.
method Two-way deep distance metric learning in a latent space with a reconstruction module.
result Our method outperforms state-of-the-arts on publicly available image datasets.
ERM and RERM minimize error even with malicious label corruptions.
problem Malicious label corruptions in regression problems.
method Empirical Risk Minimizers (ERM) and Regularized Empirical Risk Minimizers (RERM) under a local Bernstein condition.
result The L2-error rate is bounded by $r_N + AL |\cO|/N$ under the local Bernstein condition. The predictive performance of supervised learning algorithms depends on the quality of labels. In a typical label collection process, multiple annotators provide subjective noisy estimates of the "truth" under the influence of their varying skill-levels and biases. Blindly treating these noisy labels as the ground trut…
We formulate a principle for classification with the knowledge of the marginal distribution over the data points (unlabeled data). The principle is cast in terms of Tikhonov style regularization where the regularization penalty articulates the way in which the marginal density should constrain otherwise unrestricted co…
ε-Consistent Mixup improves semi-supervised classification accuracy.
problem Improving semi-supervised classification accuracy with limited labeled data.
method Combines Mixup's linear interpolation with consistency regularization, using an adaptive tradeoff between the two.
result ε-Consistent Mixup yields the largest gains in low label-availability scenarios.
Self-training improves generalization by fitting to reliable pseudo-labels or gradually improving the classification plane.
problem Understanding how self-training improves generalization in high-dimensional Gaussian mixtures.
method Analyzing iterative self-training on binary Gaussian mixtures in the asymptotic limit.
result ST improves generalization by fitting to reliable pseudo-labels or gradually improving the classification plane.