Soft labeling impacts OOD detection in neural networks.
problem Impact of soft labeling on OOD detection in deep neural networks.
method Empirical analysis of how soft labeling affects OOD detection performance.
result Soft labeling can either improve or deteriorate OOD detection performance.
Improves deep neural networks using soft labels through alternating minimization.
problem Improving deep neural networks training with soft labels.
method Co-Learns DNNs and soft labels via Alternating Minimization of two objectives.
result COLAM achieves improved performance on many tasks with better testing classification accuracy.
MSLG generates soft labels to improve DNN performance on noisy datasets.
problem Significant performance degradation of DNNs due to noisy labels.
method Meta-learning techniques to estimate optimal label distribution and iteratively update soft labels.
result MSLG outperforms state-of-the-art methods by a large margin on various datasets.
Proposes IFCDA framework to improve cross-domain adaptation.
problem Negative transfer and difficulty in handling category-irrelevant losses in DA.
method Importance filtered mechanism to generate filtered soft labels, combined with graph-based label propagation.
result Significantly improves performance in both Closed-Set and Open-Set DA scenarios.
Rectified decision trees improve machine learning interpretability and effectiveness.
problem Combining interpretability and effectiveness in machine learning models.
method Knowledge distillation and modified decision tree splitting criteria.
result Soft labels improve model performance and reduce model size.
A framework learns dynamic soft labels to improve model generalization and accuracy.
problem Models trained on one-hot labels overfit and are sensitive to noisy annotations.
method Proposes a framework where labels are treated as learnable parameters, adapting dynamically during optimization.
result Consistent gains across different datasets and architectures, improving ResNet18 by 2.1% on CIFAR100.
Paper estimates optimal classification error with soft labels and calibration.
problem Estimating the optimal classification error with soft labels and calibration.
method Extends previous work on soft labels to estimate Bayes error, addressing bias and corrupted labels.
result The method provides a statistically consistent estimator of the Bayes error, even with imperfectly calibrated soft labels.
Improves least squares classifiers with unlabeled data.
problem Improving supervised classifiers using unlabeled data.
method Block coordinate descent applied to two objective functions.
result Soft-label variant typically outperforms hard-label variant.
Propagates soft labels on hypergraphs using optimal transportation.
problem Semi-supervised learning on hypergraphs.
method Wasserstein barycenters and message-passing algorithm.
result Generalization error bounds for 2-Wasserstein distance.
Method generates prototypes from small datasets for efficient learning.
problem Efficiently learning from small datasets with soft labels.
method Modular method for generating soft-label prototypical lines and Hierarchical Soft-Label Prototype k-Nearest Neighbor algorithm.
result High classification accuracy with significantly fewer prototypes than classes.
Paper estimates FPR of Bayes classifier using soft labels.
problem Determining optimal classifier performance.
method Uses soft labels and denoising technique.
result Consistent and unbiased FPR estimator developed.
Less-than-one-shot learning tackles few-shot learning with minimal data.
problem Training models on very small datasets while maintaining accuracy.
method Soft-label k-Nearest Neighbors classifier and theoretical lower bounds analysis.
result Achieving learning of multiple classes with fewer than the required samples.
Proposes a robust VIB approach using soft labels and mutual info estimation.
problem Improving robustness of VIB to adversarial perturbations.
method Refines categorical class information with soft labels from a reference network, relaxes Gaussian posterior assumption.
result Significantly outperforms benchmarked models on MNIST and CIFAR-10.
New PL method improves ASR accuracy without pseudo-labels.
problem Improving ASR accuracy with limited labeled data.
method End-to-end continuous pseudo-labeling with soft-labels.
result Soft-labels can lead to model collapse, but regularization can mitigate this.
Improved dataset distillation for images and texts boosts model accuracy.
problem Reducing dataset size for faster and more energy-efficient model training.
method Simultaneous distillation of images and soft labels, extending to text datasets.
result 2-4% increase in accuracy for image classification tasks, 20% reduction in distilled samples.
A new method for Gaussian process classification using privileged information.
problem Efficiently incorporating privileged information into Gaussian process classification.
method Soft-to-Hard Label Transfer (SLT-GP) method.
result SLT-GP method achieves better performance than GPC and GPC+.
A new method improves graph-based semi-supervised classification by removing noise and mixed signs.
problem Inaccurate soft labels and noise in graph-based semi-supervised classification.
method Triple-matrix-recovery-based robust auto-weighted label propagation framework (ALP-TMR).
result Improved robustness to noise and outliers in label estimation.
Proposes ReDT for interpretable, compressed, and robust decision trees.
problem Improving interpretability and performance of decision trees.
method Knowledge distillation with soft labels and multiple cross-validation.
result ReDT achieves fewer nodes than classical decision trees while maintaining good performance and interpretability.
Proposes a new method for deep ensembles that improves accuracy and calibration.
problem Improving accuracy and calibration of deep ensembles.
method Estimates confusion matrices of ensemble members and weighs them according to their inferred performance.
result Empirically shows superiority of soft Dawid Skene over ensemble averaging.
Compressed Federated Distillation reduces communication in federated learning.
problem Communication constraints in Federated Learning.
method Compressed Federated Distillation (CFD) leverages soft labels and quantization techniques.
result Reduces communication by more than 4 orders of magnitude compared to Federated Averaging.
Study improves fair opinion aggregation by balancing voter attributes.
problem Aggregation of opinions can be biased by voter attributes.
method Combines majority voting and D&S model with fairness options.
result Effective combination of Soft D&S and fairness options for different data types.
Improved KNN data valuation method with reduced computation time.
problem Efficiently valuing individual data points in KNN models.
method Proposed a new utility function and derived its calculation for KNN classifiers/regressors, achieving similar time complexity as the original method.
result Soft-label KNN-SV outperforms the original method in mislabeled data detection.
SBA improves neural network generalization by dynamically augmenting data.
problem Lack of direct supervision in data augmentation for generalization.
method Stochastic Batch Augmentation with dynamic soft label regularization.
result SBA improves generalization and speeds up training.
EXoN creates an explainable latent space for semi-supervised learning.
problem Creating an explainable latent space for semi-supervised learning.
method EXoN combines VAE with SCI (Soft-label Consistency Interpolation) to create an explainable latent space.
result EXoN reduces the cost of investigating representation patterns on the latent space.
A game-theoretic approach for unsupervised domain adaptation.
problem Assigning labels to unlabeled instances in a target domain using labeled instances from a source domain.
method Graph transduction games (GTG) for iterative algorithm.
result Guaranteed termination to a Nash equilibrium, corresponding to consistent labeling.
Compact neural networks improve ECT macromolecule classification speed without sacrificing accuracy.
problem Efficiently classify millions of macromolecules captured by ECT for faster structural analysis.
method Knowledge distillation to compress complex teacher networks into simpler student networks.
result Compressed models reduce parameter count and prediction time while maintaining accuracy.
Improved self-distillation reduces label noise and enhances model accuracy.
problem Label noise in multi-class classification.
method Label averaging and refined partial labels.
result Single-round self-distillation achieves comparable performance to multi-round distillation.
SPEQ improves quantized neural networks by stochastic precision sharing and cosine similarity loss.
problem Improving quantized deep neural networks for edge devices.
method SPEQ combines stochastic precision sharing and cosine similarity loss for knowledge distillation.
result SPEQ outperforms existing methods in various tasks.
SelectMix improves deep learning robustness against noisy labels.
problem Deep neural networks memorize noisy labels, degrading performance.
method Confidence-guided targeted sample mixing with soft labels.
result SelectMix consistently outperforms baseline methods on noisy label datasets.
The paper analyzes knowledge distillation in wide neural networks, providing theoretical insights and practical implications.
problem Lack of theoretical understanding of knowledge distillation in wide neural networks.
method Theoretical analysis of knowledge distillation in a linearized model of a wide neural network, introducing a metric of task training difficulty.
result For a perfect teacher, a high ratio of teacher's soft labels can be beneficial. For imperfect teacher, hard labels can correct wrong predictions.
Proposes STN for HDA by aligning domain-shared classifier and subspace.
problem Learning in target domain using knowledge from heterogeneous source domain.
method End-to-end learning of domain-shared classifier and domain-invariant subspace; soft-label strategy for unlabeled data; adaptive coefficient for soft-labels.
result Significantly outperforms state-of-the-art approaches in various transfer tasks.
A new method for unsupervised domain adaptation using manifold learning.
problem Leveraging rich source domain information to target domain without labeled data.
method Discriminative Manifold Embedding and Alignment framework.
result Consistent transferability and discriminability achieved through manifold metric alignment.
Proposes a framework to improve domain adaptation without labeled data.
problem Improving adaptability and preserving intrinsic data structure in unsupervised domain adaptation.
method Discriminative Manifold Propagation framework using soft labels and manifold metric alignment.
result The method achieves better transferability and discriminability compared to existing approaches.
SoftCVI uses contrastive estimation to infer complex posteriors.
problem Estimating complex posteriors in Bayesian inference.
method Contrastive variational inference with self-generated soft labels.
result SoftCVI outperforms other variational approaches in stability and coverage.
Logistic regression can handle noisy labels effectively when labels are imperfectly assigned by multiple experts.
problem Label noise in supervised classification due to manual labelling by multiple experts.
method Using approximate posterior probabilities of class membership from multiple experts to train logistic regression models.
result Logistic regression can be robust to label noise when classification difficulty is the only source of errors.
A new method transfers adversarial robustness from teacher to student using feature distillation.
problem Adversarial robustness transfer across different models and tasks.
method Guided Adversarial Contrastive Distillation (GACD) with contrastive learning and sample reweighted estimation.
result GACD effectively transfers adversarial robustness from teacher to student, achieving comparable or better results.
Bayesian method predicts labels on large graphs using Laplacian eigenfunctions.
problem Binary classification on large graphs.
method Hierarchical Bayesian approach with truncated Laplacian regularization.
result Improved scalability for large graphs compared to untruncated Laplacian.
Improved segmentation model adaptation for new domains.
problem Reduced performance of pre-trained models on new domains.
method Calculated soft-label prototypes and predicted closest to class probabilities.
result Significant performance improvements on synthetic-to-real segmentation.
No regularization needed for InLDL, achieving efficient and effective model.
problem InLDL struggles with performance degradation due to missing degrees.
method Proposes a model that uses label distribution as a prior, implicitly regularizing the learning process.
result Achieves competitive performance without explicit regularization.
IDS improves RLHF by smoothing reward data, enhancing model performance.
problem Reward model performance degrades and overoptimization hinders true objective.
method Iterative Data Smoothing (IDS) updates model and data labels during each epoch.
result IDS outperforms traditional methods in RLHF.
Spanning attack improves black-box attacks with unlabeled data.
problem Query inefficiency in black-box attacks due to high input space dimensionality.
method Proposes spanning attack by constraining adversarial perturbations in a low-dimensional subspace via an auxiliary unlabeled dataset.
result Significantly improves query efficiency of black-box attacks.
This work improves regression performance by using distributional losses, finding better optimization leads to improved generalization.
problem Improving regression performance in reinforcement learning.
method Introduced a novel distributional regression loss and investigated its effects on optimization and generalization.
result The novel distributional regression loss leads to improved prediction accuracy and better optimization.
Proposes DFDG for robust domain generalization without source domain labels.
problem Robustness of deep learning models in real-world applications where train and test distributions differ.
method Model-agnostic, class-aware alignment of class relationships through saliency maps.
result Competitive performance on time series sensor and image classification datasets.
In this paper we propose a simple yet powerful method for learning representations in supervised learning scenarios where each original input datapoint is described by a set of vectors and their associated outputs may be given by soft labels indicating, for example, class probabilities. We represent an input datapoint …
OMADA improves uncertainty calibration for deep networks.
problem Challenges in reliably quantifying uncertainty for modern deep networks.
method On-Manifold Adversarial Data Augmentation (OMADA) using latent space adversarial attacks.
result OMADA consistently yields more accurate and better calibrated classifiers than baseline models.
Estimates hybrid dynamical systems with polynomial expansions and Markovian switching.
problem Identifying hybrid dynamical systems with nonlinear autoregressive exogenous (NARX) components and Markovian switching.
method Probabilistic framework using Expectation Maximization for parameter estimation, including submodel coefficients, hidden state values, and transition probabilities. Disentangles mode classification and NARX regression tasks. Uses soft-labels and coordinate descent approach for parameter fitting.
result Demonstrated on a SMNARX problem with three nonlinear sub-models, achieving parsimonious models through l1-norm bridge estimation and hard-thresholding.
New SGD covering technique yields dimension-independent generalization bounds.
problem Generalization of stochastic gradient descent in non-convex, non-smooth settings.
method Localized ε-covers for SGD trajectories, showing dimension-independent complexity.
result Generalization error upper bounded by O ( ( log n log ( n P ) ) / n ) O(\sqrt{(\log n\log(nP))/n}) O ( ( log n log ( n P )) / n ) . Voting ensemble of robust models improves robustness.
problem Improving robustness of defensive models against adversarial attacks.
method Hard-label voting ensemble of pretrained robust models.
result Voting ensemble can boost robust error over individual models.