Distillation affects some classes more than others, impacting fairness and bias.
problem Distillation affects some classes more than others, impacting fairness and bias.
method Examined class-wise accuracy and fairness metrics (DPD, EOD) on models trained with different datasets.
result Increasing the distillation temperature improves the distilled student model's fairness and individual fairness.
ABROCA assesses algorithmic bias, revealing skewed distributions that inflate results.
problem Detecting nuanced performance differences in classifier fairness.
method Study of ABROCA metric's statistical properties under various conditions.
result ABROCA distributions are skewed, inflating results by chance in imbalanced classes.
New algorithm offers costless model selection in contextual bandits.
problem Minimizing cumulative regret in stochastic contextual bandits.
method Gradually increasing class complexity and adapting to the simplest class with dominant estimation variance.
result Costless model selection is feasible under certain conditions, providing improved regret guarantees.
Unified framework suppresses model bias in semi-supervised learning with decoupled sampling control.
problem Class imbalance in semi-supervised learning, especially with distributional mismatches.
method Unified framework SC-SSL with decoupled sampling control, explicit expansion capability, and adaptive sampling probabilities.
result Consistent and state-of-the-art performance across various benchmark datasets and distribution settings.
Mitigates confirmation bias in SSL by adjusting pseudo labels dynamically.
problem Confirmation bias in semi-supervised learning leads to errors in pseudo labels.
method TaMatch framework adjusts scaling ratio to debias pseudo labels and dynamically adjusts target distribution.
result TaMatch significantly outperforms existing methods in SSL tasks.
New proof links initial class bias to DNN trainability, challenging traditional understanding.
problem Understanding the initial class bias in DNNs and its impact on trainability.
method Theoretical proof linking initial class bias to mean field theories of DNNs.
result Efficient learning is connected to a network's prejudice towards a specific class, contradicting traditional understanding.
Contrastive learning struggles with class collapse and feature suppression, revealing bias towards simpler solutions.
problem Contrastive learning struggles with class collapse and feature suppression, especially in supervised and unsupervised settings.
method Unified theoretical framework to determine which features are learnt by CL, revealing bias towards simpler solutions.
result Bias towards simpler solutions is a key factor in class collapse and feature suppression.
New method quantifies inductive bias for machine learning tasks.
problem Quantifying the amount of inductive bias in machine learning models.
method Estimates inductive bias by modeling loss distribution of random hypotheses.
result Higher dimensional tasks require greater inductive bias.
ConQUR tackles delusional bias in deep Q-learning, improving performance in Atari games.
problem Delusional bias in deep Q-learning.
method Efficient methods to mitigate delusional bias by training Q-approximators with consistent labels and a search framework.
result Improves performance in Atari games, sometimes dramatically.
Decision trees can be biased towards minority class, contrary to belief.
problem Bias in decision trees towards minority class in imbalanced datasets.
method Critical evaluation of past literature, specific conditions analysis, tree-fitting adjustments, and post-hoc calibration methods.
result Decision trees can be biased towards minority class under specific conditions, not always towards majority.
Assessing the fairness of a decision making system with respect to a protected class, such as gender or race, is challenging when class membership labels are unavailable. Probabilistic models for predicting the protected class based on observable proxies, such as surname and geolocation for race, are sometimes used to …
We study the phenomenon of bias amplification in classifiers, wherein a machine learning model learns to predict classes with a greater disparity than the underlying ground truth. We demonstrate that bias amplification can arise via an inductive bias in gradient descent methods that results in the overestimation of the…
Recommendation systems have been integrated into the majority of large online systems to filter and rank information according to user profiles. It thus influences the way users interact with the system and, as a consequence, bias the evaluation of the performance of a recommendation algorithm computed using historical…
Matrix SMD converges to unique solution minimizing Bregman divergence.
problem High-dimensional multi-output classification and matrix completion problems.
method Stochastic Mirror Descent with matrix parameters and matrix mirror functions.
result Matrix SMD converges exponentially to the unique solution minimizing Bregman divergence.
In real-world classification problems, the class balance in the training dataset does not necessarily reflect that of the test dataset, which can cause significant estimation bias. If the class ratio of the test dataset is known, instance re-weighting or resampling allows systematical bias correction. However, learning…
New findings show DNC is not optimal for deep models, revealing a low-rank bias.
problem Theoretical limitations of DNC in non-linear models and multi-class classification.
method Analysis of non-linear models of arbitrary depth in multi-class classification.
result DNC stops being optimal for DUFM when going beyond two layers or two classes, due to a low-rank bias.
Untrained neural networks can unfairly assign predictions to the same class.
problem Biasing effects in neural networks during initial training phases.
method Theoretical analysis of deep neural networks, focusing on Initial Guessing Bias (IGB).
result Model structure and preprocessing methods influence IGB.
Study reveals class disparities in balanced datasets through spectral imbalance.
problem Class disparities in balanced datasets are overlooked despite model performance gaps.
method Developed a theoretical framework and studied 11 encoders to diagnose spectral imbalance.
result Identified spectral imbalance as a source of class disparities in balanced datasets.
This report examines the Pinned AUC metric introduced and highlights some of its limitations. Pinned AUC provides a threshold-agnostic measure of unintended bias in a classification model, inspired by the ROC-AUC metric. However, as we highlight in this report, there are ways that the metric can obscure different kinds…
Gradient descent learns ReLU functions with non-zero bias efficiently.
problem Learning ReLU functions with non-zero bias under Gaussian distributions.
method Gradient descent starting from random initialization.
result Gradient descent achieves near-optimal error with high probability.
Machine learning algorithms can misrepresent training data, study finds.
problem Misrepresentation of training data in machine learning algorithms.
method Demonstrated through underestimation of training data due to irreducible error, regularization, and class imbalance.
result Careful management of synthetic counterfactuals can mitigate underestimation bias.
Improved sampling from complex distributions with reduced bias.
problem Reducing bias in high-dimensional sampling algorithms.
method Hierarchical entropy analysis to weaken assumptions and expand scope.
result Bias reduction in low-dimensional marginals scales with lower dimension, not full dimension.
Bayesian adaptive designs can be biased by active learning, especially with misspecified models.
problem Active learning bias in Bayesian adaptive experimental designs.
method Analysis of linear and preference learning models, empirical testing.
result Model misspecification and noise influence active learning bias in Bayesian designs.
Mirror flow in shallow neural networks shows similar implicit bias to gradient flow, with key differences in curvature penalties.
problem Analyzing implicit bias in shallow neural networks with mirror flow.
method Characterization through variational problems and scaled potentials.
result Mirror flow with scaled potentials induces a rich class of biases not captured by RKHS norms.
Meta-learning reduces training overhead for communication systems.
problem Inefficiency in machine learning due to frequent retraining when system configuration changes.
method Meta-learning selects a suitable inductive bias from related tasks, reducing training data and time requirements.
result Meta-learning can reduce training overhead for communication systems.
New method reduces bias in estimating causal effects from discretized variables.
problem Bias in estimating causal effects from discretized continuous variables.
method Proposes a bias-reduced functional that evaluates outcome regression at within-bin conditional means.
result Demonstrates substantial bias reduction and near-nominal confidence interval coverage.
Algorithm corrects bias in classification data.
problem Underrepresentation and intersectional bias in classification data.
method Estimate group-wise drop-out rates with small unbiased data, construct reweighting scheme, and present algorithm.
result Efficiently approximate loss of any hypothesis on true distribution.
Active learning method balances bias and variance under class imbalance.
problem Active learning under label shift when class proportions differ.
method Mediated Active Learning under Label Shift (MALLS) using a 'medial distribution'.
result MALLS reduces asymptotic sample complexity under arbitrary label shift.
In this paper, we study the accuracy of values aggregated over classes predicted by a classification algorithm. The problem is that the resulting aggregates (e.g., sums of a variable) are known to be biased. The bias can be large even for highly accurate classification algorithms, in particular when dealing with class-…
This work analyzes the maximum-margin bias in quasi-homogeneous neural networks.
problem Analyzing the maximum-margin bias in quasi-homogeneous neural networks.
method Geometric analysis of gradient dynamics for quasi-homogeneous models.
result Gradient flow implicitly favors a subset of parameters, leading to asymmetric norm minimization.
Study reveals how neural network architectures bias their learning based on feature directions.
problem Understanding how neural network architectures bias their learning based on feature directions.
method Defined neural anisotropy directions (NADs) to encapsulate the directional inductive bias of architectures and provided an efficient method to identify them.
result NADs characterize the features used by CNNs to discriminate between different classes for the CIFAR-10 dataset.
Excessive reuse of holdout data can lead to overfitting. However, there is little concrete evidence of significant overfitting due to holdout reuse in popular multiclass benchmarks today. Known results show that, in the worst-case, revealing the accuracy of k k k adaptively chosen classifiers on a data set of size n n n al…
Paper debiases multiple word embedding biases simultaneously.
problem Reduction of multiple biases in word embeddings.
method Joint multiclass debiasing approach using Word Embeddings Association Test (WEAT).
result Demonstrates reduction or complete elimination of bias in word embeddings.
Study improves statistical power for detecting algorithmic bias in educational data.
problem Challenges in measuring algorithmic bias using ABROCA due to skewed distribution.
method Investigates ABROCA's distributional properties and proposes nonparametric randomization tests.
result ABROCA-based bias assessments are underpowered in typical EDM sample sizes.
Paper connects sampling and labeling biases in large-output spaces.
problem Efficient training in large-output spaces with label imbalance.
method Unified approach to address sampling and labeling biases.
result Different negative sampling schemes trade-off performance on dominant and rare labels.
Understanding the inductive bias of neural networks is critical to explaining their ability to generalise. Here, for one of the simplest neural networks -- a single-layer perceptron with n input neurons, one output neuron, and no threshold bias term -- we prove that upon random initialisation of weights, the a priori p…
A major source of risk in project management is inaccurate forecasts of project costs, demand, and other impacts. The paper presents a promising new approach to mitigating such risk, based on theories of decision making under uncertainty which won the 2002 Nobel prize in economics. First, the paper documents inaccuracy…
This work extends implicit bias analysis to multiclass classification using a new loss framework.
problem The implicit bias of gradient descent on multiclass data without explicit regularization.
method Employing the PERM framework to introduce a multiclass extension of the exponential tail property.
result Extended implicit bias result to multiclass classification using a new loss framework.
New method removes pseudo-label bias for unsupervised domain adaptation.
problem Class imbalance and distribution shift between domains.
method Implicit class-conditioned domain alignment without explicit pseudo-label optimization.
result Effective in handling within-domain class imbalance and between-domain class distribution shift.
Proposes a method to learn policies from offline data with reduced bias.
problem Learning policies from offline data with reduced bias and complexity constraints.
method Cross-fitted debiasing device for policy learning from offline data.
result Achieves N \sqrt N N regret for complex policy classes with a product-of-errors nuisance remainder. Meta-learning improves performance in stochastic linear bandits.
problem Selecting a learning algorithm that performs well across multiple bandit tasks.
method Regularized OFUL algorithm with a bias vector, estimating bias within the learning-to-learn setting.
result Meta-learning strategies improve performance when the number of tasks grows and task variance is small.
Visualizes classification results with class maps.
problem Label bias in classification predictions.
method Class maps reflecting probability, distance, and mislabeling likelihood.
result Insight into classification results and data structure.
Theory and methods to mitigate omitted variable bias in causal machine learning.
problem Mitigating omitted variable bias in causal machine learning models.
method Developed a general theory and flexible statistical inference methods for bounding and testing the magnitude of omitted variable bias.
result Simple plausibility judgments can bound the magnitude of omitted variable bias in complex, nonlinear models.
The paper analyzes optimal implicit bias in linear regression for over-parameterized models.
problem Finding the best generalization performance in over-parameterized linear regression.
method Asymptotic analysis of generalization performance for convex functions/potentials.
result Optimal implicit bias that achieves the best generalization error under certain conditions.
Exactly solvable model reveals how data geometry influences ML bias.
problem How data geometry affects machine learning bias.
method High-dimensional data imbalance model, statistical physics tools.
result Exact predictions for fairness metrics and mitigation strategies.
Study shows imbalanced labels can be beneficial but not always in class-imbalanced learning.
problem Challenges in class-imbalanced learning with heavy label bias.
method Systematic investigation of semi-supervised and self-supervised approaches to leverage imbalanced labels.
result Imbalanced labels are valuable in semi-supervised learning but not always in self-supervised learning.
Visualizes classification accuracy and label bias in neural nets and trees.
problem Identifying mislabeled cases in neural net and tree-based classifications.
method Silhouette plots and quasi residual plots of PAC (probability of alternative class).
result Comparison of different classifications using silhouette width and PAC plots.
A number of applications (e.g., AI bot tournaments, sports, peer grading, crowdsourcing) use pairwise comparison data and the Bradley-Terry-Luce (BTL) model to evaluate a given collection of items (e.g., bots, teams, students, search results). Past work has shown that under the BTL model, the widely-used maximum-likeli…