Study shows semi-supervised learning can be more robust with fewer labeled examples.
problem Learning robust predictors in semi-supervised PAC model with minimal labeled data.
method Characterizes the minimal labeled and unlabeled data required for robust learning.
result Proves nearly matching upper and lower bounds on labeled sample complexity.
Self-supervision provides effective representations for downstream tasks without requiring labels. However, existing approaches lag behind fully supervised training and are often not thought beneficial beyond obviating or reducing the need for annotations. We find that self-supervision can benefit robustness in a varie…
AAT separates robust and non-robust features without supervision.
problem Adversarial vulnerability and accuracy reduction in machine learning models.
method Adversarial Asymmetric Training (AAT) algorithm.
result Preserves accuracy and achieves better disentanglement than previous methods.
Paper proposes a method to train robust neural networks without labeled data.
problem Training robust neural networks without class labels.
method Adversarial contrastive learning framework using unlabeled data.
result Robust Contrastive Learning (RoCL) achieves comparable robust accuracy to supervised methods and significantly improved robustness.
Self-supervision improves GCNs' generalizability and robustness.
problem Improving graph convolutional networks' performance.
method Three mechanisms of self-supervision, multi-task learning, and graph adversarial training.
result Self-supervision enhances GCNs' robustness and generalizability.
Self-supervised learning performs better than supervised learning on imbalanced datasets.
problem The performance gap between balanced and imbalanced pre-training with self-supervised learning is smaller than with supervised learning.
method Systematic investigation of self-supervised learning under dataset imbalance, including experiments and theoretical analyses.
result Self-supervised representations are more robust to class imbalance than supervised representations.
LOT improves adversarial robustness by training 1-Lipschitz convolution layers.
problem Improving adversarial robustness of deep neural networks.
method LOT: Layer-wise Orthogonal Training for 1-Lipschitz convolution layers.
result LOT significantly enhances certified robustness of Lipschitz-bounded models.
FROST speeds up and stabilizes one-shot semi-supervised learning.
problem Slow training and sensitivity to labeled data choices in semi-supervised learning.
method Combines semi-supervised learning with a one-stage, single network self-training approach.
result FROST trains up to 10x faster and is more robust to labeled data choices.
Efficiently builds diverse sub-model ensembles for robust self-supervised learning.
problem Challenges in diversity and efficiency of deep ensembles for self-supervised representation learning.
method Ensemble of independent sub-networks with a new loss function for diversity.
result Significantly improves prediction reliability and model calibration.
RoPAWS improves semi-supervised learning on uncurated data.
problem Efficiency of semi-supervised learning with real-world unlabeled data.
method Reinterprets PAWS as a generative classifier and calibrates predictions using densities of labeled and unlabeled data.
result Significant improvement in performance for uncurated data.
Doubly robust self-training improves semi-supervised learning by balancing labeled and pseudo-labeled data.
problem Improving semi-supervised learning performance with limited labeled data.
method Introduces doubly robust self-training, a method that combines labeled and pseudo-labeled data to balance between labeled-only and pseudo-labeled-only training.
result Demonstrates superior performance of doubly robust self-training on ImageNet and nuScenes datasets.
RIO uses rotation-equivariance to train robust inertial odometry models.
problem Training robust inertial odometry models with limited labeled data.
method Rotation-equivariance as self-supervisor, adaptive Test-Time Training (TTT).
result RIO-trained models achieve on-par performance with full-labeled data models.
DSL uses supervised learning to optimize portfolios, improving stability and performance.
problem Optimizing robust portfolios in financial markets.
method DSL reframes portfolio construction as a supervised learning problem, using cross-entropy loss and optimizing Sharpe or Sortino ratios. Deep Ensemble methods are employed to reduce variance.
result DSL outperforms traditional and machine learning methods, achieving higher median returns and more stable risk-adjusted performance.
SNAP improves robust computation by emphasizing trustworthy items and downweighting outliers.
problem Improving robustness in computation, especially in high-dimensional settings.
method SNAP assigns weights based on mutual agreement, suppressing outlier contributions.
result SNAP ensures outliers contribute negligibly to computations, even in high-dimensional settings.
TrustNet robustly learns noise patterns from trusted data to improve weakly-supervised classification.
problem Robustness to label noise in weakly-supervised learning.
method TrustNet learns noise patterns from trusted data, then trains a robust classifier using these patterns.
result TrustNet outperforms state-of-the-art methods in robustness to various noise patterns.
Consistency regularization improves robustness to noisy labels.
problem Improving model robustness to noisy labels in machine learning.
method Empirical study of consistency regularization on noisy datasets.
result Consistency regularization improves model robustness to label noise.
A simple self-supervised model for tensor RPCA using deep unfolding.
problem Tensor robust principal component analysis (RPCA) challenges in practical applications.
method Deep unfolding with only four hyperparameters.
result Competitive or superior performance compared to supervised methods, even in data-starved scenarios.
Self-supervised learning excels in topic modeling by being less model-specific.
problem How self-supervised learning discovers useful representations in topic models.
method Applying self-supervised learning objectives to topic model-generated data.
result Self-supervised learning objectives can recover useful posterior information for topic models, outperforming misspecified models.
This work improves adversarial robustness in sparse coding models.
problem The gap between theoretical models and practical deployment in adversarial robustness.
method Combining a sparsity-promoting encoder with a linear classifier, and providing a robustness certificate.
result Bounding the robust risk and providing a robustness certificate for end-to-end classification.
The effectiveness of supervised learning techniques has made them ubiquitous in research and practice. In high-dimensional settings, supervised learning commonly relies on dimensionality reduction to improve performance and identify the most important factors in predicting outcomes. However, the economic importance of …
Develops robust and efficient SS estimators for treatment effects.
problem Estimating treatment effects in semi-supervised settings with limited labeled data.
method A family of SS estimators using labeled and unlabeled data, ensuring robustness and efficiency.
result Root-n consistency and asymptotic normality of SS estimators under correct specification of propensity score and nuisance functions.
Unsupervised learning representations generalize better than supervised learning under distribution shifts.
problem Robustness of unsupervised representations to distribution shift.
method Extensive evaluation on synthetic and realistic datasets, including controllable domain generalization datasets.
result Unsupervised representations learned from SSL and AE generalize better than supervised learning under various distribution shifts.
A robust loss for anomaly mitigation and unsupervised contamination classification
problem Detecting and mitigating contamination in supervised and unsupervised settings
method Neural Bayesian Anomaly Mitigation (NBAM)
result Recovering the structure of contamination and identifying label-flip pairs
We propose a novel objective function for learning robust deep representations of data based on information theory. Data is projected into a feature-vector space such that the mutual information of all subsets of features relative to the supervising signal is maximized. This objective function gives rise to robust repr…
Graph Random Neural Network improves semi-supervised learning on graphs.
problem Over-smoothing, non-robustness, and weak-generalization in GNNs with few labeled nodes.
method Random propagation strategy and consistency regularization.
result Significantly outperforms state-of-the-art GNN baselines on semi-supervised node classification.
Paper introduces robust learning methods using coordinate gradient descent.
problem Supervised learning with corrupted features and labels.
method Coordinate gradient descent combined with robust estimators of partial derivatives.
result Robust learning methods with nearly identical numerical complexity to non-robust ones.
Paper proposes S2ConvSCN for robust subspace clustering and classification.
problem Insufficient handling of nonlinear manifolds, data corruptions, and out-of-sample data.
method Self-supervised convolutional subspace clustering network (S2ConvSCN) with FC layer, CIM for robustness, and BD regularization. result Robust S2ConvSCN outperforms baseline on unseen data. Improved quantile estimation using semi-supervised data.
problem Quantile estimation in high-dimensional settings with limited labeled data.
method Proposes semi-supervised estimators using a flexible imputation strategy and debiasing step.
result Improved estimation accuracy compared to supervised methods, robust to misspecification.
We propose a novel method for semi-supervised learning (SSL) based on data-driven distributionally robust optimization (DRO) using optimal transport metrics. Our proposed method enhances generalization error by using the unlabeled data to restrict the support of the worst case distribution in our DRO formulation. We en…
The paper improves self-training in semi-supervised learning by selecting more robust pseudo-labeled data.
problem Improving the reliability of pseudo-labeled data selection in self-training for semi-supervised learning.
method Proposes a multi-objective utility function to select pseudo-labeled data that maximizes reliability, considering model selection, accumulation of errors, and covariate shift uncertainties.
result Robustness towards model choice can lead to substantial accuracy gains in self-training.
New algorithm robustly trains deep neural networks under corrupted supervision.
problem Training deep neural networks with corrupted supervision data.
method Unified framework for classification and regression, focusing on collective impact of data points on average gradient.
result Achieves strong guarantees without assuming the type of corruption, robust under various types of corruption.
New method improves transfer and robustness of supervised contrastive learning.
problem Class collapse in supervised contrastive learning leads to poor representation quality.
method Adding a weighted class-conditional InfoNCE loss and a class-conditional autoencoder.
result Improves transfer and robustness on 5 standard datasets and 3 worst-group robustness datasets.
Improves SSL with doubly robust estimation of unlabeled class distribution.
problem Limited labeled data and long-tailed class distributions in unlabeled data.
method Explicitly estimate unlabeled class distribution using doubly robust estimator.
result Improves performance of SSL methods on unlabeled data.
MEC improves efficiency and robustness in semi-supervised inference.
problem Efficient inference with limited labeled data and robust uncertainty quantification.
method Machine-Learning-Assisted Generalized Entropy Calibration (MEC) using cross-fitted, calibration-weighted PPI.
result MEC achieves semiparametric efficiency bounds under weaker assumptions and provides near-nominal coverage.
The paper improves semi-supervised learning using f-divergences and α-Rényi divergences.
problem Improving semi-supervised learning with noisy pseudo-labels.
method Inspired by f-divergences and α-Rényi divergences, the paper develops new empirical risk functions and regularization techniques. result The new methods show better performance than traditional self-training methods, especially in noisy pseudo-label scenarios.
Study on clustering in high dimensions with anisotropic Gaussian mixtures, showing interpolation can be optimal and robust.
problem Clustering in high-dimensional anisotropic Gaussian mixtures.
method Derive minimax bounds, analyze ℓ2-regularized classifiers, and investigate interpolation's robustness. result Interpolating solutions can be optimal and robust under certain conditions.
Paper improves robustness of ABI with unlabeled data.
problem Robustness issues in ABI for out-of-simulation data.
method Semi-supervised approach using unlabeled data and self-consistency losses.
result Semi-supervised ABI with unlabeled data improves robustness significantly.
TopoGeoScore selects robust checkpoints using only source-domain representations.
problem Selecting robust checkpoints without target-domain labels or samples.
method Constructs class-conditional mutual k-nearest-neighbour graphs and extracts three interpretable signals.
result Source representations contain measurable global-local-topological evidence of robustness.
Deep nearest neighbors outperform self-supervised methods in anomaly detection.
problem Anomaly detection using self-supervised deep methods.
method Simple nearest-neighbor approach on Imagenet pretrained features.
result Nearest-neighbor method outperforms self-supervised methods in accuracy, few shot generalization, training time, and noise robustness.
Different types of training data have led to numerous schemes for supervised classification. Current learning techniques are tailored to one specific scheme and cannot handle general ensembles of training data. This paper presents a unifying framework for supervised classification with general ensembles of training dat…
ORAT improves model robustness against outliers and adversarial attacks.
problem Challenges of training data like outliers and adversarial samples.
method Bi-level optimization with robust rank-based loss function.
result ORAT achieves theoretical consistency and uniform convergence rates.
Framework uses unlabeled out-of-domain data to improve semi-supervised classification.
problem Improving generalization in semi-supervised classification problems.
method Combines Distributionally Robust Optimization (DRO) with self-supervised training.
result Significant improvement in generalization error compared to ERM.
Distributionally Robust Supervised Learning (DRSL) is necessary for building reliable machine learning systems. When machine learning is deployed in the real world, its performance can be significantly degraded because test data may follow a different distribution from training data. DRSL with f-divergences explicitly …
New framework improves model robustness by focusing on stable relations across environments.
problem Standard supervised learning fails under data distribution shift.
method Gradient-based learning framework derived from the principle of independent causal mechanisms (ICM).
result Models generalize well to unseen scenarios, ignoring unstable relations.
RAVEN improves weak-to-strong generalization under distribution shifts.
problem Weak models fail to supervise strong models effectively under distribution shifts.
method RAVEN dynamically learns optimal combinations of weak models and strong model parameters.
result RAVEN outperforms existing methods by over 30% on out-of-distribution tasks.
Paper tackles active labeling for partial supervision.
problem Accessing stochastic gradients with partial supervision.
method Streaming technique to minimize generalization error.
result Proves minimization of generalization error ratio.
S2MAM improves semi-supervised learning by selecting relevant variables and updating similarity metrics.
problem Joint learning from labeled and unlabeled data with geometric structure.
method Bilevel optimization scheme for automatic variable selection and similarity matrix update.
result The proposed S2MAM achieves robust and interpretable predictions.
Semi-supervised learning is an important and active topic of research in pattern recognition. For classification using linear discriminant analysis specifically, several semi-supervised variants have been proposed. Using any one of these methods is not guaranteed to outperform the supervised classifier which does not t…