Crowdsourcing infers ground truth from multiple annotators, verified for supervised learning.
problem Obtaining universally valid ground truth for supervised learning is challenging and costly.
method Gather multiple annotations from diverse individuals, verify and aggregate for training classifiers.
result Inferred ground truth improves classifier performance in sensitive tasks like mitosis detection.
New method trains deep denoisers without ground truth data.
problem Training deep denoisers with high-quality ground truth images is often impractical.
method Uses Stein's unbiased risk estimator (SURE) for training deep neural networks with only noisy images.
result Trained deep denoisers perform similarly to those trained with ground truth images.
A new score measures data reliability without ground truth.
problem Assessing reliability of datasets without access to ground truth.
method Define ground-truth-based orderings and propose Gram determinant score.
result Gram determinant score effectively captures data quality across diverse observation processes.
RealCause provides a realistic benchmark for causal inference.
problem Lack of a reliable benchmark for comparing causal effect estimators.
method Flexible generative models to create a benchmark that is both ground-truth and realistic.
result Evaluation of over 1500 causal estimators provides evidence for choosing hyperparameters using predictive metrics.
Paper introduces metrics to evaluate missing data imputation without ground truth.
problem Handling missing data in time series without ground truth.
method Introduces Wasserstein distance (WD) and Jensen-Shannon divergence (JSD) as metrics to evaluate imputation quality.
result WD and JSD are effective metrics for assessing missing data imputation quality.
Paper proposes a method to recover accurate labels from partially valid data in multi-label learning.
problem Tackles noisy supervision in multi-label learning with partially valid labels.
method Develops a two-stage method that estimates label enrichment and ground-truth confidences.
result Demonstrates improved performance over state-of-the-art PML methods.
Ranking a set of objects involves establishing an order allowing for comparisons between any pair of objects in the set. Oftentimes, due to the unavailability of a ground truth of ranked orders, researchers resort to obtaining judgments from multiple annotators followed by inferring the ground truth based on the collec…
Deep learning for Venus images uses high-res hyperspectral data to simulate ground truth.
problem Lack of accurate ground truth data for training deep neural networks in remote sensing.
method Unmixing high-resolution hyperspectral images to simulate ground truth for training a CNN.
result The model can classify mid-resolution Venus images successfully.
Estimates classifier errors without ground truth using algebraic geometry.
problem Lack of ground truth in real-world production systems.
method Non-parametric estimation using algebraic geometry to solve the self-assessment problem.
result Accuracy estimators are better than one part in a hundred.
Generative model combines multi-dimensional annotations for more accurate ground truth estimation.
problem Inaccurate ground truth estimation from naive annotators' multi-dimensional annotations.
method Proposes a joint multi-dimensional model for global and time-series annotation fusion using Expectation-Maximization algorithm.
result More accurate ground truth estimates through joint modeling of multiple dimensions.
Self-supervised methods learn from noisy data alone, useful for imaging problems.
problem Inferring signals from noisy and incomplete observations.
method Learning a solver from measurement data alone, without ground-truth references.
result Self-supervised methods can learn meaningful estimates from noisy data.
Paper proposes metrics to evaluate AI explanations without ground truth.
problem Challenges in evaluating neural network explanations without ground truth.
method Designs four metrics to evaluate explanation results.
result New insights into neural network interpretation methods.
New method falsifies causal discovery results without ground truth.
problem Evaluation of causal discovery algorithms without ground truth data.
method Detects incompatibilities between causal graphs learned on different subsets of variables.
result Detection of incompatibilities can falsify wrongly inferred causal relations.
URC improves model accuracy without ground truth for new data.
problem Improving model accuracy with new, unseen data without ground truth.
method URC observes and corrects model predictions when training data is not representative of field data.
result URC corrects model bias without retraining or ground truth for some subpopulations.
SAMPLR optimizes for ground truth in aleatoric parameters to avoid curriculum-induced covariate shift.
problem Curriculum learning shifts training distribution, leading to suboptimal policies in aleatoric settings.
method SAMPLR optimizes ground-truth utility function, avoiding curriculum-induced covariate shift.
result SAMPLR preserves optimality under ground-truth distribution, promoting robustness across various environments.
Study corrects misaligned cadaster maps using noisy supervision.
problem Correcting misaligned cadaster maps with noisy supervision data.
method Iterative training rounds to refine ground truth annotations.
result Reduces noise in cadaster map alignment datasets.
Study measures malware detection metrics without ground truth.
problem Accurate measurement of security metrics in the absence of ground truth.
method Statistical estimation methods for five malware detection metrics.
result Characterized and improved statistical estimators for accuracy.
Tests for classifier independence without ground truth labels.
problem Validation of classifier independence without ground truth labels.
method Exact solution for independent binary classifiers using algebraic geometry.
result Self-consistent test for classifier independence without ground truth labels.
Improved DLG extracts accurate labels from gradients, overcoming DLG's convergence issues.
problem Private training data leakage from shared gradients in distributed learning systems.
method Proposes iDLG, a simple approach to synthesize accurate labels from gradients.
result iDLG reliably extracts ground-truth labels from gradients, unlike DLG.
Study reveals issues with neural autoregressive models and proposes mode recovery cost.
problem Unreasonable affinity of neural autoregressive models to short and long sequences.
method Investigates modes of ground-truth, empirical, and decoding-induced distributions via mode recovery cost.
result Mode recovery cost varies depending on ground-truth distribution and impacts decoding-induced distribution.
Paper estimates regression errors without known ground truth.
problem Estimating regression errors when ground truth is unknown.
method Developed a framework to estimate generalization error.
result Framework robustly detects concept drift in real-world datasets.
FUSE improves verification quality without ground truth labels.
problem Verification of model outputs using imperfect judges and reward models.
method Ensembling verifiers without ground truth labels using spectral algorithms.
result FUSE matches or improves upon semi-supervised alternatives in test-time scaling experiments.
Proposes a method for explaining tabular data using copulas.
problem Lack of ground truth for explainability in complex datasets.
method Uses copulas to specify statistical properties and build intuition.
result Demonstrates improved explainability on logistic regression and correlation use cases.
LLMs fail to match statistical ground truth despite stable run-to-run performance.
problem LLMs lack validation against statistical ground truth in automated scientific workflows.
method Introduced a behavioral evaluation framework for LLMs, separating four decision-making dimensions.
result LLMs can exhibit near-perfect stability but diverge from statistical ground truth.
A method for inferring ground-truth signals from degraded sensor data.
problem Inferring ground-truth signals from multiple degraded sensor signals.
method Iterative correction of degraded signals using a Bayesian multi-sensor data fusion method.
result The method effectively infers ground-truth signals from noisy and degraded sensor data.
The paper analyzes how noise affects distances in high-dimensional data and when they remain useful.
problem Noise corrupts distances in high-dimensional data, making them unreliable for identifying true nearest and farthest neighbors.
method The paper uses asymptotic probabilistic expressions to characterize noise effects and decomposes data into ground truth and noise components.
result Under certain conditions, empirical neighborhood relations remain truthful even when distance concentration occurs.
New method estimates model performance bounds without ground truth labels.
problem Evaluation of weakly supervised models without direct access to ground truth labels.
method Formulates model evaluation as a partial identification problem and uses Fréchet bounds for performance estimation.
result Derives accurate and computationally efficient bounds for key metrics like accuracy, precision, recall, and F1-score.
Preconditioned SGD accelerates convergence for ill-conditioned huge-scale matrix completion.
problem Recovering a low-rank matrix from incomplete data with high condition number.
method Preconditioned Stochastic Gradient Descent (SGD) for huge-scale online optimization.
result Preconditioned SGD converges to ε-accuracy in O(log(1/ε)) iterations, compared to O(κlog(1/ε)) for unpreconditioned SGD.
Noise affects the effectiveness of interpolating models, especially those with strong inductive biases.
problem The impact of noise on interpolating models with strong inductive biases.
method Analyzing linear and classification models with sparse ground truths, proving fast rates for interpolators.
result Strong inductive biases can lead to faster but noisier interpolators, contrary to intuition.
Error correction for correlated regressors using compressed sensing.
problem Measuring errors in regression tasks with sparse correlations.
method Compressed sensing applied to error signals, with ℓ1-minimization for uniqueness. result Correct solution recovery in settings of error correction.
QUACKIE creates a new benchmark for NLP interpretability.
problem Evaluating NLP interpretability methods is challenging due to biased ground truths.
method Formulated a custom classification task from question-answering datasets, generating unbiased ground truths.
result Demonstrated the effectiveness of current interpretability methods on the new benchmark.
Improved generalization with iterative self-distillation using weighted ground-truth targets.
problem Improving generalization accuracy in neural networks.
method Iterative kernel regression with weighted ground-truth targets and ℓ2 regularization. result Closed-form solution for optimal weighting parameter and efficient estimation.
Unsupervised clustering can reproduce categorization systems if features and metrics are correctly selected.
problem Reproducing expert-provided categorization systems using unsupervised clustering.
method Investigated using toy datasets and real-world fund categorization. Used appropriate feature selection and a supervised Random Forest-based distance metric.
result Unsupervised clustering can reproduce ground truth classes if features and metrics are correctly selected.
The paper analyzes GNNs with one hidden layer, proving their generalizability and convergence rate.
problem Theoretical guarantee on generalizability of GNNs with one hidden layer.
method Tensor initialization and accelerated gradient descent.
result The proposed learning algorithm converges to the ground-truth GNN model for regression and to a model close to the ground-truth for binary classification.
PoGDiff improves diffusion models on imbalanced datasets.
problem Diffusion models struggle with imbalanced text-image pairs.
method PoGDiff replaces ground-truth with a Product of Gaussians.
result PoGDiff enhances generation accuracy and quality on imbalanced datasets.
New research shows the maximum ℓ1-margin classifier doesn't adapt to sparse ground truths.
problem Understanding the limitations of the maximum ℓ1-margin classifier in high-dimensional settings.
method Analyzing convergence and prediction error rates of the maximum ℓ1-margin classifier.
result Proves tight upper and lower bounds for prediction error, showing benign overfitting.
New method accounts for uncertainty in medical AI evaluations.
problem Uncertainty in ground truth affects AI model performance estimates.
method Statistical aggregation approach to infer probabilities of medical conditions.
result Performance estimates are significantly lower when uncertainty is accounted for.
A new method finds DAG models without ground truth.
problem Finding DAG models without ground truth.
method Markov Checker test and Cross-Algorithm Frugality Search (CAFS).
result Models can be chosen without ground truth.
Researchers formalize PD and PFI to relate them to data generating process.
problem Lack of theory linking PD and PFI to data generating process.
method Formalize PD and PFI as estimators of ground truth estimands, account for model variance with learner-PD and learner-PFI.
result PD and PFI estimates deviate from ground truth due to statistical biases, model variance, and Monte Carlo approximation errors.
New method separates music vocals from accompaniment without labeled data.
problem Separating music sources without isolated recordings.
method Bootstrapping deep model using primitive auditory cues.
result Trained deep model separates vocals from accompaniment in unlabeled music.
Enhances clustering quality evaluation in noisy data.
problem Reliable clustering quality assessment in noisy Gaussian mixtures.
method Feature Importance Rescaling (FIR) method.
result FIR improves correlation between cluster validity indices and ground truth.
Study robustness of conformal prediction to label noise in regression and classification.
problem Robustness of conformal prediction to label noise in regression and classification.
method Characterized robustness of conformal prediction for both regression and classification problems, extending theory to control general loss functions.
result Conformal prediction and risk-controlling techniques can achieve conservative risk over clean ground truth labels with noisy labels.
Paper proposes using unlabeled data for fair decision-making.
problem Bias in decision-making algorithms due to biased labels and selective labeling.
method Variational autoencoder for learning unbiased data representations from both labeled and unlabeled data.
result Method learns fair and stable decision policies with high utility.
Systematic approach to generating diverse 3D scenes and images for training and benchmarking.
problem Training and benchmarking machine learning algorithms for scene understanding.
method Stochastic grammar-based pipeline for generating and rendering photorealistic 3D scenes with detailed ground truth.
result Improves performance in scene understanding tasks and provides controllable benchmarks for model diagnostics.
Agent-based model compares different COVID-19 testing policies and their effectiveness.
problem Understanding how different testing policies reveal the true number of infected cases.
method Developed an agent-based simulation framework in Python to model various testing policies and interventions.
result Contact Tracing consistently captures more positive cases than Random Symptomatic Testing, and LBT performs similarly.
VB-Score evaluates AI systems without ground truth, revealing robustness.
problem Evaluating AI systems without ground truth labels, especially for entity-centric tasks.
method VB-Score uses variance-bounded evaluation, constraint relaxation, and Monte Carlo sampling.
result VB-Score reveals robustness differences not seen by conventional frameworks.
VAEs struggle with low-dimensional data; this paper shows they can learn the correct manifold.
problem Training VAEs on low-dimensional data.
method Two-stage training algorithm based on conjecture of 0 variance generator.
result VAEs can learn generators with support equal to the ground truth manifold.
System separates sounds from mixtures without ground truth info.
problem Sound separation from multi-channel mixtures without labeled data.
method Deep clustering on multi-channel mixtures, projecting bins to spatially correlated clusters.
result Performance matches ground truth separation using only multi-channel mixtures.