New models can't beat existing ones, so debiasing methods only slightly reduce needed labels.
problem Limiting scalability in model evaluation due to self-preferencing biases.
method Study of debiasing methods using a few high-quality labels to reduce model judgments.
result Debiasing methods can't decrease required ground truth labels by more than half when the judge is no more accurate than the model.