Paper tackles online adaptation to changing label distributions.
problem Adapting machine learning models to changing label distributions in real-world settings.
method Leverages novel analysis to show estimation of expected test loss is possible without true labels. Proposes adaptation algorithms inspired by classical online learning techniques.
result Empirically verified that OGD is particularly effective and robust to various label shift scenarios.
In this paper, we consider a novel machine learning problem, that is, learning a classifier from noisy label distributions. In this problem, each instance with a feature vector belongs to at least one group. Then, instead of the true label of each instance, we observe the label distribution of the instances associated …
AdapTable adapts tabular models to shifts without source data, improving HELOC performance.
problem Distribution shifts in tabular data threaten model performance.
method Shift-aware uncertainty calibrator and label distribution handler.
result Up to 16% improvement on HELOC dataset.
Enhances labels from unlabeled data using sample correlations.
problem Lack of label distributions in real-world applications.
method Proposes LESC and gLESC methods to enhance label distributions.
result Improves performance of label enhancement through sample correlations.
Unsupervised domain adaptation is a promising way to generalize deep models to novel domains. However, the current literature assumes that the label distribution is domain-invariant and only aligns the feature distributions or vice versa. In this work, we explore the more realistic task of Class-imbalanced Domain Adapt…
No regularization needed for InLDL, achieving efficient and effective model.
problem InLDL struggles with performance degradation due to missing degrees.
method Proposes a model that uses label distribution as a prior, implicitly regularizing the learning process.
result Achieves competitive performance without explicit regularization.
Enhances LDL by integrating distance and directional information for more robust label feature representation.
problem Lack of robust label feature representation in LDL tasks, especially with label ambiguity.
method Introduces Structural Anchor Points (SAPs) to capture inter-cluster interactions and a novel LSFs construction strategy, LIFT-SAP.
result Improves LDL performance by 15% on average across 15 real-world datasets.
Method converts age labels into distributions to improve speaker age estimation.
problem Label ambiguity in age labels makes precise speaker age estimation challenging.
method Converts age labels into label distributions and uses label distribution learning.
result Our method outperforms baseline methods by reducing MAE by 10% on a real-world dataset.
Unified approach to non-standard classification tasks.
problem Non-standard classification tasks like semi-supervised, positive-unlabelled, multi-positive-unlabelled and noisy-label learning.
method Probabilistic, unified approach training a classifier to predict label-distributions, then inferring class-distributions.
result Unified model for various non-standard classification tasks.
Proposes a new approach to MSDA by introducing latent covariate shift to handle varying label distributions.
problem Challenges of conventional MSDA approaches in real-world settings where label distributions vary across domains.
method Introduces latent covariate shift (LCS) and a causal generative model with latent noises, latent content variable, and latent style variable.
result Identifies latent content variable up to block identifiability, enabling more nuanced label distribution recovery.
Paper tackles multi-label learning by improving SVR for positive semidefinite metrics.
problem Learning positive semidefinite metrics for multi-label and label distribution learning.
method Proposes two methods to overcome SVR's limitation in learning positive semidefinite metrics.
result Demonstrates new methods achieve favorable performance in multi-label and label distribution learning.
Paper proposes a new method to aggregate multiple sources with different label distributions.
problem Aggregating from multiple target-shifted sources with different label distributions.
method Unified framework to select relevant sources for domain adaptation with limited label, unsupervised, and label partial unsupervised scenarios.
result Empirical results significantly outperform baselines.
Paper detects common subtrees with identical labels in trees.
problem Finding common subtrees with identical label distribution in tree data.
method Developed an algorithm for tree isomorphism and a new compression scheme for trees.
result The method efficiently finds and compresses common subtrees with identical labels.
New method pools labels from similar data items to improve learning from small samples.
problem Learning from small, human-annotated samples with potential disagreement among annotators.
method Proposes neighborhood-based pooling for sharing labels across similar data items.
result Improves learning from small, noisy samples by pooling labels from similar items.
A bidirectional loss function improves label distribution learning and enhancement.
problem Challenges in label distribution learning and label enhancement.
method Bidirectional loss function to address dimensional gap and label enhancement.
result The bidirectional loss function improves the accuracy of label distribution learning and enhancement.
New approach improves domain adaptation with label shift assumptions.
problem Improving domain adaptation when label distributions differ between source and target domains.
method Proposes generalized label shift (GLS) and modifies three DA algorithms (JAN, DANN, CDAN) to handle label distribution mismatches. result Modified DA algorithms outperform base versions, especially with large label distribution mismatches.
AUC-spec optimizes graph-based SSL for complex label distributions.
problem Training accurate models with scarce labeled data and abundant unlabeled data.
method Computes a low-dimensional representation that maximizes class separation via AUC optimization.
result AUC-spec achieves competitive results on synthetic and real-world datasets.
Balanced Activation improves object detection performance on long-tailed datasets.
problem Mismatch between training and testing label distributions in object detection.
method Introduces Balanced Activation (Balanced Softmax and Balanced Sigmoid) to address label distribution shift.
result Balanced Activation provides ~3% gain in mAP on LVIS-1.0 compared to state-of-the-art methods.
Paper tackles long-tailed labels in classification problems.
problem Imbalanced or long-tailed label distribution in real-world classification problems.
method Logit adjustment applied post-hoc or during training to encourage a large relative margin between rare and dominant labels.
result Unified and generalised techniques for coping with long-tailed labels, improving generalisation and performance.
The paper introduces a method to quantify uncertainty in neural networks without parametric assumptions.
problem Uncertainty quantification for neural network predictions.
method Nonparametric estimation of conditional label distribution using Nadaraya-Watson kernel.
result The method effectively disentangles aleatoric and epistemic uncertainties.
Under covariate shift, training (source) data and testing (target) data differ in input space distribution, but share the same conditional label distribution. This poses a challenging machine learning task. Robust Bias-Aware (RBA) prediction provides the conditional label distribution that is robust to the worstcase lo…
Class-imbalance is an inherent characteristic of multi-label data which affects the prediction accuracy of most multi-label learning methods. One efficient strategy to deal with this problem is to employ resampling techniques before training the classifier. Existing multilabel sampling methods alleviate the (global) im…
Algorithm performance in supervised learning is a combination of memorization, generalization, and luck. By estimating how much information an algorithm can memorize from a dataset, we can set a lower bound on the amount of performance due to other factors such as generalization and luck. With this goal in mind, we int…
Deep learning algorithms can fare poorly when the training dataset suffers from heavy class-imbalance but the testing criterion requires good generalization on less frequent classes. We design two novel methods to improve performance in such scenarios. First, we propose a theoretically-principled label-distribution-awa…
Label noise in adversarial training leads to robust overfitting, explained and mitigated.
problem Label noise in adversarial training causes robust overfitting.
method Proposed a method to automatically calibrate labels.
result Consistent performance improvements across various models and datasets.
We propose a new regularization method based on virtual adversarial loss: a new measure of local smoothness of the conditional label distribution given input. Virtual adversarial loss is defined as the robustness of the conditional label distribution around each input data point against local perturbation. Unlike adver…
Unified view of label shift estimation methods.
problem Label distribution changes but class-conditional distributions remain the same.
method Unified view of two approaches: BBSE and MLLS.
result Unified framework and theoretical characterization of MLLS.
This research tackles imbalanced continual learning with a new sampling strategy.
problem Long-tailed distribution in multi-label datasets.
method Partitioning Reservoir Sampling (PRS) for balanced knowledge of head and tail classes.
result The proposed PRS strategy maintains a balanced knowledge of both head and tail classes.
Domain adaptation is an important technique to alleviate performance degradation caused by domain shift, e.g., when training and test data come from different domains. Most existing deep adaptation methods focus on reducing domain shift by matching marginal feature distributions through deep transformations on the inpu…
Paper tackles dynamic label shift in online learning, achieving optimal performance.
problem Adapting to changing class marginals in online supervised and unsupervised learning.
method Develops novel algorithms reducing adaptation to online regression, achieving optimal dynamic regret.
result Achieves superior performance in various online label shift scenarios.
SIG model identifies invariant variables for MSDA with fewer domain constraints.
problem Challenges in enforcing minimal changes across domains for MSDA.
method Subspace identification theory and variational inference.
result SIG model outperforms existing techniques on various benchmark datasets.
Classical clustering algorithms typically either lack an underlying probability framework to make them predictive or focus on parameter estimation rather than defining and minimizing a notion of error. Recent work addresses these issues by developing a probabilistic framework based on the theory of random labeled point…
Paper quantifies label shift robustly.
problem Quantifying label shift in datasets.
method Robust estimators of label distribution.
result Maximum Likelihood Estimator is a robust estimator.
Deep Convolutional Neural Networks (DCNN) has shown excellent performance in a variety of machine learning tasks. This manuscript presents Deep Convolutional Neural Fields (DeepCNF), a combination of DCNN with Conditional Random Field (CRF), for sequence labeling with highly imbalanced label distribution. The widely-us…
The paper proposes a method to reliably select design algorithms for machine learning-guided design tasks.
problem Choosing the right design algorithm for machine learning-guided design tasks.
method Combining designs' predicted property values with held-out labeled data to reliably forecast characteristics of the label distributions produced by different design algorithms.
result The method is guaranteed to return design algorithms that yield successful label distributions.
Paper tackles online label shift in real-world applications.
problem Adapting to changing label distributions in online learning.
method Formulated an unbiased risk estimator and proposed online ensemble algorithms.
result Achieved optimal dynamic regret, indicating adaptability to label shift.
New framework optimizes label shift adaptation using aligned distribution mixture.
problem Label shift where source and target label distributions differ.
method Aligned Distribution Mixture (ADM) framework, incorporating insights from generalization theory.
result The ADM framework improves four typical label shift methods and introduces a one-step approach.
New truthful calibration errors improve model ranking in multiclass prediction.
problem Non-truthful calibration errors can mislead model comparisons.
method Introduced perfectly truthful calibration errors for multiclass predictions.
result Truthful calibration errors preserve decision-theoretic dominance and stabilize model rankings.
A new method for batch prediction sets in classification problems.
problem Constructing reliable prediction sets for multiple unlabeled examples.
method Proposes a uniformly more powerful approach to batch prediction sets using specific combinations of conformal p-values.
result The proposed method provides narrower prediction sets compared to the Bonferroni correction.
We study the stochastic block model with two communities where vertices contain side information in the form of a vertex label. These vertex labels may have arbitrary label distributions, depending on the community memberships. We analyze a linearized version of the popular belief propagation algorithm. We show that th…
This paper investigates semi-supervised hashing methods using variational autoencoders.
problem Semantic hashing with scarce labels.
method Two semi-supervised approaches: joint modeling and pairwise loss.
result The pairwise approach can improve hash quality with many labeled points but degrades with few labels.
Cross-domain sentiment analysis is currently a hot topic in the research and engineering areas. One of the most popular frameworks in this field is the domain-invariant representation learning (DIRL) paradigm, which aims to learn a distribution-invariant feature representation across domains. However, in this work, we …
Domain adaptation addresses the common problem when the target distribution generating our test data drifts from the source (training) distribution. While absent assumptions, domain adaptation is impossible, strict conditions, e.g. covariate or label shift, enable principled algorithms. Recently-proposed domain-adversa…
We present novel graph kernels for graphs with node and edge labels that have ordered neighborhoods, i.e. when neighbor nodes follow an order. Graphs with ordered neighborhoods are a natural data representation for evolving graphs where edges are created over time, which induces an order. Combining convolutional subgra…
We describe Information Forests, an approach to classification that generalizes Random Forests by replacing the splitting criterion of non-leaf nodes from a discriminative one -- based on the entropy of the label distribution -- to a generative one -- based on maximizing the information divergence between the class-con…
We propose a robust adversarial prediction framework for general multiclass classification. Our method seeks predictive distributions that robustly optimize non-convex and non-continuous multiclass loss metrics against the worst-case conditional label distributions (the adversarial distributions) that (approximately) m…
Wasserstein Dropout improves uncertainty estimation in neural networks.
problem Estimating neural uncertainties for safe machine learning.
method A purely non-parametric approach using dropout-based sub-network distributions and Wasserstein distance.
result Wasserstein Dropout outperforms state-of-the-art methods in uncertainty estimation.
We might hope that when faced with unexpected inputs, well-designed software systems would fire off warnings. Machine learning (ML) systems, however, which depend strongly on properties of their inputs (e.g. the i.i.d. assumption), tend to fail silently. This paper explores the problem of building ML systems that fail …