Paper tackles long-tailed labels in classification problems.
problem Imbalanced or long-tailed label distribution in real-world classification problems.
method Logit adjustment applied post-hoc or during training to encourage a large relative margin between rare and dominant labels.
result Unified and generalised techniques for coping with long-tailed labels, improving generalisation and performance.
This research tackles imbalanced continual learning with a new sampling strategy.
problem Long-tailed distribution in multi-label datasets.
method Partitioning Reservoir Sampling (PRS) for balanced knowledge of head and tail classes.
result The proposed PRS strategy maintains a balanced knowledge of both head and tail classes.
Tail-GNNs improve protein function prediction using relational reinforcement.
problem Predicting hierarchical protein functions from sequence data.
method Combining Tail-GNNs with dilated convolutional networks for multi-task learning.
result Significant improvement in F_1 score for protein function prediction.
Paper proposes LAHA to improve XMTC by integrating document content and label correlation.
problem Challenges in tagging documents with most relevant labels from a large label set.
method Hybrid attention deep neural network model (LAHA) that combines multi-label self-attention and adaptive fusion strategies.
result LAHA outperforms state-of-the-art methods, especially on tail labels.
Learning requires memorizing labels, especially in long-tailed data.
problem Understanding why memorizing labels is necessary for accurate learning.
method Introduced a theoretical model for natural data distributions, showing memorization is necessary for optimal generalization error.
result Memorization of labels, even for outliers and noisy labels, is necessary for achieving close-to-optimal generalization error.
The goal in extreme multi-label classification is to learn a classifier which can assign a small subset of relevant labels to an instance from an extremely large set of target labels. Datasets in extreme classification exhibit a long tail of labels which have small number of positive training instances. In this work, w…
Improved predictions for rare labels using neural networks and ontologies.
problem Long-tailed frequency distribution in multi-label prediction problems.
method Modified neural network output layer with a Bayesian network of sigmoids leveraging ontology relationships.
result Significant improvements in per-label AUROC and average precision for less common labels.
Study shows imbalanced labels can be beneficial but not always in class-imbalanced learning.
problem Challenges in class-imbalanced learning with heavy label bias.
method Systematic investigation of semi-supervised and self-supervised approaches to leverage imbalanced labels.
result Imbalanced labels are valuable in semi-supervised learning but not always in self-supervised learning.
We analyze the local Rademacher complexity of empirical risk minimization (ERM)-based multi-label learning algorithms, and in doing so propose a new algorithm for multi-label learning. Rather than using the trace norm to regularize the multi-label predictor, we instead minimize the tail sum of the singular values of th…
Balanced Activation improves object detection performance on long-tailed datasets.
problem Mismatch between training and testing label distributions in object detection.
method Introduces Balanced Activation (Balanced Softmax and Balanced Sigmoid) to address label distribution shift.
result Balanced Activation provides ~3% gain in mAP on LVIS-1.0 compared to state-of-the-art methods.
Multi-label classification has received considerable interest in recent years. Multi-label classifiers have to address many problems including: handling large-scale datasets with many instances and a large set of labels, compensating missing label assignments in the training set, considering correlations between labels…
Improves SSL with doubly robust estimation of unlabeled class distribution.
problem Limited labeled data and long-tailed class distributions in unlabeled data.
method Explicitly estimate unlabeled class distribution using doubly robust estimator.
result Improves performance of SSL methods on unlabeled data.
New loss functions improve extreme classification with missing labels.
problem Large number of infrequent labels and missing labels in XMC.
method Derive unbiased loss functions for XMC, incorporating them into existing algorithms.
result Significant improvement in extreme classification performance (up to 20%) over existing methods.
New loss function restores importance weighting in overparameterized models.
problem Restoring importance weighting in overparameterized neural networks.
method Introduced polynomially-tailed losses to restore effects of importance weighting.
result Polynomially-tailed losses improve performance in correcting distribution shift.
Paper introduces Balanced Meta-Softmax for better long-tailed visual recognition.
problem Long-tailed distribution mismatch between training and testing data.
method Balanced Meta-Softmax, an unbiased extension of Softmax, using a Meta Sampler.
result Balanced Meta-Softmax outperforms state-of-the-art solutions on visual recognition and instance segmentation.
A variation of the preferential attachment random graph model of Barabási and Albert is defined that incorporates planted communities. The graph is built progressively, with new vertices attaching to the existing ones one-by-one. At every step, the incoming vertex is randomly assigned a label, which represents a commun…
Paper develops heavy-tailed embeddings for better text classification and augmentation.
problem Improving text classification, especially for extreme values.
method Develops heavy-tailed embeddings using multivariate extreme value theory and introduces a scale-invariant classifier.
result The classifier outperforms baselines and generates meaningful augmented text.
ELM improves neural model embeddings for long-tail learning.
problem Learning skewed label distributions in neural models.
method Enforces margins in logit space and regularizes embedding distribution.
result ELM reduces generalization gap and tightens tail class embeddings.
A new method detects changes in mixture models quickly and accurately.
problem Detecting changes in mixture models with heavy-tailed components.
method Change-point methods based on robust and quick approach.
result The method is up to 500 times faster and more accurate than existing methods.
Paper connects sampling and labeling biases in large-output spaces.
problem Efficient training in large-output spaces with label imbalance.
method Unified approach to address sampling and labeling biases.
result Different negative sampling schemes trade-off performance on dominant and rare labels.
This paper examines biases in foundation models under long-tailed data and proposes a method to mitigate parameter imbalance.
problem The bias introduced by imbalanced training data in foundation models affects long-tailed downstream tasks.
method The paper examines parameter imbalance and data imbalance, proposing a backdoor adjustment method to mitigate parameter imbalance.
result An average performance increase of about 1.67% on each dataset.
Learning shrinks hard tail, improving inference performance.
problem Improving inference performance in neural networks.
method Latent Instance Difficulty (LID) model analyzing fine-tuning of neural networks.
result Training-dependent inference scaling, with βexteff growing with sample size before saturating. New conformal prediction methods for long-tailed classification problems.
problem Rare classes are systematically omitted in existing conformal prediction methods.
method Introduced a new conformal score function and a new interpolation procedure.
result Smoothly trade off set size and class-conditional coverage.
FLAIR dataset for federated learning benchmarks.
problem Lack of suitable federated learning datasets.
method Curated large-scale annotated image dataset for multi-label classification.
result FLAIR captures real-world federated learning challenges.
A new method estimates parameters in heavy-tailed corrupted regression with unknown covariance and heterogeneous noise.
problem Estimating parameters in regression with heavy-tailed errors and unknown covariance.
method Near-optimal computationally tractable estimator based on power method and Multiplicative Weight Update algorithm.
result The estimator achieves the optimal statistical rate and breakdown-point under near-optimal sample size.
New method uses label-weighted conformal prediction for macro-coverage guarantees in classification.
problem Finding a balance between class-conditional and marginal coverage in long-tailed datasets.
method Label-weighted conformal prediction for macro-coverage guarantees.
result Validated prediction sets with macro-coverage guarantees on large-scale image datasets.
Extreme multi-label classification (XMC) refers to supervised multi-label learning involving hundreds of thousand or even millions of labels. In this paper, we develop a suite of algorithms, called Bonsai, which generalizes the notion of label representation in XMC, and partitions the labels in the representation space…
Deep neural network predicts semantic labels for source code.
problem Difficulty in labeling and understanding new programming languages and functionalities.
method Language-agnostic deep convolutional neural network trained on Stack Overflow code snippets.
result Mean area under ROC of 0.957 and top-1 accuracy of 86.6% on Github code documents.
The objective in extreme multi-label learning is to train a classifier that can automatically tag a novel data point with the most relevant subset of labels from an extremely large label set. Embedding based approaches make training and prediction tractable by assuming that the training label matrix is low-rank and hen…
Ridge regression shows different behaviors in binary classification with noisy labels.
problem Binary classification with noisy labels and anisotropic cluster distributions.
method Investigation of ridge regression behavior in overparameterized settings with label noise.
result Ridge regression exhibits qualitatively different behavior based on the scale of cluster mean vectors and covariance matrices.
Paper proposes a novel method to assess treatment effect estimators using cross-validation.
problem Lack of ground truth to objectively assess treatment effect estimators in RCTs.
method Cross-validation-like methodology combining noisy difference-of-means estimate and aggregation across RCTs.
result Aggressive downweighting or truncation of large values reduces variance and improves treatment effect estimation.
Bayesian framework estimates label shift for improved classifier performance.
problem Label shift in supervised learning leading to degraded classifier performance.
method Bayesian framework with dynamic Dirichlet priors and online EM algorithms.
result Significant improvements in classifier accuracy over state-of-the-art methods.
New experiments show deep networks benefit from memorizing rare data points.
problem Understanding why neural networks perform well on training data.
method Estimating influence and memorization values of training examples.
result Deep networks benefit from memorizing rare data points for good performance.
End-to-end anomaly detection framework using labeled anomalies.
problem Limited deep learning for anomaly detection and inefficiency of existing methods.
method Deviation learning neural network with labeled anomalies and prior probability.
result Significantly better anomaly scoring than state-of-the-art methods.
The paper tackles extrapolation in extreme regions of regression problems.
problem Extrapolation on the tails of covariates in continuous regression problems.
method Statistical regression on a subsample of furthest observations, focusing on their angular components, using multivariate regular variation theory.
result Quantifies predictive performance on tail regions in terms of excess risk, presenting it as a finite sample risk bound with a bias-variance decomposition.
ERM and RERM minimize error even with malicious label corruptions.
problem Malicious label corruptions in regression problems.
method Empirical Risk Minimizers (ERM) and Regularized Empirical Risk Minimizers (RERM) under a local Bernstein condition.
result The L2-error rate is bounded by $r_N + AL |\cO|/N$ under the local Bernstein condition. FedReLa: A novel data-level approach for imbalanced federated learning
problem Improving accuracy of federated learning models under class imbalance and data heterogeneity
method Re-labeling samples with a feature-dependent label re-allocator
result Significant improvements in accuracy for minority classes and overall accuracy on stepwise-imbalanced and long-tailed datasets
ZDP detects drift in large language models without labels, proving key theorems and metrics.
problem Detecting drift in large language models without task labels or output evaluations.
method Zero-Direction Probing (ZDP) framework based on null directions of transformer activations, proving theoretical guarantees.
result Proves the Variance--Leak Theorem, Fisher Null-Conservation, Rank--Leak bound, and logarithmic-regret guarantee.
RED-2400 is a public benchmark of trading events from a Solana exchange, labeled by algorithmic rejection.
problem Analyzing algorithmically-rejected trading events for insights into market dynamics.
method Public dataset of 6,660 algorithmically-rejected trading events, linked to post-rejection price and liquidity trajectories.
result First window of a planned series of datasets extending the time horizon and enabling regime-stratified analysis.
New method models fat-tailed distributions with anisotropic tail-adaptive flows.
problem Gaussian-based variational inference fails to accurately capture tail decay in fat-tailed distributions.
method Improved theory on tails of flows, developed anisotropic tail-adaptive flows (ATAF).
result ATAF models tail-anisotropy, outperforming prior work on synthetic and real-world targets.
New measures capture tail dependence and non-exchangeability in financial data.
problem Underestimation of tail dependence and inability to capture non-exchangeable tail dependence.
method Tail copulas and novel tail dependence measures (MTCM, ATCM) are proposed.
result Captures non-exchangeable tail dependence and provides analytical forms for various copulas.
The paper examines how heavy-tailed risks behave under Gaussian copula models.
problem Understanding tail risk probabilities with heavy-tailed marginal risks and Gaussian dependence.
method Modeling heavy-tailed risks using regular variation and analyzing tail probabilities under Gaussian copula.
result The rate of decay of tail set probabilities varies with the type of tail sets and Gaussian correlation matrix.
Adaptive regularization tackles heteroskedastic and imbalanced datasets in deep learning.
problem Heteroskedastic and imbalanced datasets challenge deep learning due to varying label uncertainty and long-tailed label distributions.
method Data-dependent adaptive regularization that applies stronger regularization to higher-uncertainty, lower-density regions.
result Significant improvement in noise-robust deep learning over other methods on benchmark tasks.
Study characterizes and mitigates imbalances in neurosymbolic learning.
problem Characterizing and mitigating class-specific risks in neural classifiers.
method Theoretical analysis and practical techniques including estimating marginal gold labels and mitigating imbalances at training and testing time.
result Learning imbalances can be greatly impacted by the symbolic component σ, unlike in supervised and weakly supervised learning.
New tail dependence measures for stock indices.
problem Measuring tail dependence between financial variables.
method Introducing a new stochastic order and studying monotone tail dependence measures.
result Advantage of new tail dependence measures over classical ones.
Study tail behavior of sum of heavy-tailed risks with copulas.
problem Analyzing the tail behavior of sums of heavy-tailed risks with dependence modeled by copulas.
method Modeling dependence with copulas and analyzing tail asymptotics of sums of heavy-tailed risks.
result Obtained asymptotic expansions for Value-at-Risk of aggregate risk.
Paper tackles open set domain adaptation by detecting unknown classes.
problem Adapting to target domains with unknown classes when label spaces partially overlap.
method Instance-level reweighting strategy combined with Extreme Value Theory for unknown class detection.
result Proposed method outperforms state-of-the-art models on conventional datasets.
Paper provides tail bounds for stochastic mirror descent in heavy-tailed noise.
problem Optimizing convex and Lipschitz functions with heavy-tailed noise.
method Develops tail bounds for optimization error of Stochastic Mirror Descent.
result Tail bounds extend to heavier-tailed noise regimes without diameter constraints.