Universally valid ground truth is almost impossible to obtain or would come at a very high cost. For supervised learning without universally valid ground truth, a recommended approach is applying crowdsourcing: Gathering a large data set annotated by multiple individuals of varying possibly expertise levels and inferri…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “ground truth”
Paper proposes a method to recover accurate labels from partially valid data in multi-label learning.
problem Tackles noisy supervision in multi-label learning with partially valid labels.
method Develops a two-stage method that estimates label enrichment and ground-truth confidences.
result Demonstrates improved performance over state-of-the-art PML methods.
Tests for classifier independence without ground truth labels.
problem Validation of classifier independence without ground truth labels.
method Exact solution for independent binary classifiers using algebraic geometry.
result Self-consistent test for classifier independence without ground truth labels.
RealCause provides a realistic benchmark for causal inference.
problem Lack of a reliable benchmark for comparing causal effect estimators.
method Flexible generative models to create a benchmark that is both ground-truth and realistic.
result Evaluation of over 1500 causal estimators provides evidence for choosing hyperparameters using predictive metrics.
A new score measures data reliability without ground truth.
problem Assessing reliability of datasets without access to ground truth.
method Define ground-truth-based orderings and propose Gram determinant score.
result Gram determinant score effectively captures data quality across diverse observation processes.
Recently developed deep-learning-based denoisers often outperform state-of-the-art conventional denoisers such as the BM3D. They are typically trained to minimize the mean squared error (MSE) between the output image of a deep neural network (DNN) and a ground truth image. Thus, it is important for deep-learning-based …
Algebraic Ground Truth Inference: Non-Parametric Estimation of Sample Errors by AI Algorithmsstat.ML
Estimates classifier errors without ground truth using algebraic geometry.
problem Lack of ground truth in real-world production systems.
method Non-parametric estimation using algebraic geometry to solve the self-assessment problem.
result Accuracy estimators are better than one part in a hundred.
Ranking a set of objects involves establishing an order allowing for comparisons between any pair of objects in the set. Oftentimes, due to the unavailability of a ground truth of ranked orders, researchers resort to obtaining judgments from multiple annotators followed by inferring the ground truth based on the collec…
Paper introduces metrics to evaluate missing data imputation without ground truth.
problem Handling missing data in time series without ground truth.
method Introduces Wasserstein distance (WD) and Jensen-Shannon divergence (JSD) as metrics to evaluate imputation quality.
result WD and JSD are effective metrics for assessing missing data imputation quality.
QUACKIE creates a new benchmark for NLP interpretability.
problem Evaluating NLP interpretability methods is challenging due to biased ground truths.
method Formulated a custom classification task from question-answering datasets, generating unbiased ground truths.
result Demonstrated the effectiveness of current interpretability methods on the new benchmark.
SAMPLR optimizes for ground truth in aleatoric parameters to avoid curriculum-induced covariate shift.
problem Curriculum learning shifts training distribution, leading to suboptimal policies in aleatoric settings.
method SAMPLR optimizes ground-truth utility function, avoiding curriculum-induced covariate shift.
result SAMPLR preserves optimality under ground-truth distribution, promoting robustness across various environments.
Unsupervised clustering can reproduce categorization systems if features and metrics are correctly selected.
problem Reproducing expert-provided categorization systems using unsupervised clustering.
method Investigated using toy datasets and real-world fund categorization. Used appropriate feature selection and a supervised Random Forest-based distance metric.
result Unsupervised clustering can reproduce ground truth classes if features and metrics are correctly selected.
This paper proposes a set of criteria to evaluate the objectiveness of explanation methods of neural networks, which is crucial for the development of explainable AI, but it also presents significant challenges. The core challenge is that people usually cannot obtain ground-truth explanations of the neural network. To …
New method accounts for uncertainty in medical AI evaluations.
problem Uncertainty in ground truth affects AI model performance estimates.
method Statistical aggregation approach to infer probabilities of medical conditions.
result Performance estimates are significantly lower when uncertainty is accounted for.
Generative model combines multi-dimensional annotations for more accurate ground truth estimation.
problem Inaccurate ground truth estimation from naive annotators' multi-dimensional annotations.
method Proposes a joint multi-dimensional model for global and time-series annotation fusion using Expectation-Maximization algorithm.
result More accurate ground truth estimates through joint modeling of multiple dimensions.
New method falsifies causal discovery results without ground truth.
problem Evaluation of causal discovery algorithms without ground truth data.
method Detects incompatibilities between causal graphs learned on different subsets of variables.
result Detection of incompatibilities can falsify wrongly inferred causal relations.
Autonomy and adaptation of machines requires that they be able to measure their own errors. We consider the advantages and limitations of such an approach when a machine has to measure the error in a regression task. How can a machine measure the error of regression sub-components when it does not have the ground truth…
Self-supervised methods learn from noisy data alone, useful for imaging problems.
problem Inferring signals from noisy and incomplete observations.
method Learning a solver from measurement data alone, without ground-truth references.
result Self-supervised methods can learn meaningful estimates from noisy data.
Preconditioned SGD accelerates convergence for ill-conditioned huge-scale matrix completion.
problem Recovering a low-rank matrix from incomplete data with high condition number.
method Preconditioned Stochastic Gradient Descent (SGD) for huge-scale online optimization.
result Preconditioned SGD converges to ε-accuracy in O(log(1/ε)) iterations, compared to O(κlog(1/ε)) for unpreconditioned SGD.
Study reveals issues with neural autoregressive models and proposes mode recovery cost.
problem Unreasonable affinity of neural autoregressive models to short and long sequences.
method Investigates modes of ground-truth, empirical, and decoding-induced distributions via mode recovery cost.
result Mode recovery cost varies depending on ground-truth distribution and impacts decoding-induced distribution.
It is widely believed that sharing gradients will not leak private training data in distributed learning systems such as Collaborative Learning and Federated Learning, etc. Recently, Zhu et al. presented an approach which shows the possibility to obtain private training data from the publicly shared gradients. In their…
New method estimates model performance bounds without ground truth labels.
problem Evaluation of weakly supervised models without direct access to ground truth labels.
method Formulates model evaluation as a partial identification problem and uses Fréchet bounds for performance estimation.
result Derives accurate and computationally efficient bounds for key metrics like accuracy, precision, recall, and F1-score.
Noise affects the effectiveness of interpolating models, especially those with strong inductive biases.
problem The impact of noise on interpolating models with strong inductive biases.
method Analyzing linear and classification models with sparse ground truths, proving fast rates for interpolators.
result Strong inductive biases can lead to faster but noisier interpolators, contrary to intuition.
In machine learning the best performance on a certain task is achieved by fully supervised methods when perfect ground truth labels are available. However, labels are often noisy, especially in remote sensing where manually curated public datasets are rare. We study the multi-modal cadaster map alignment problem for wh…
When Stability Fails: Hidden Failure Modes Of LLMS in Data-Constrained Scientific Decision-Makingcs.LG
LLMs fail to match statistical ground truth despite stable run-to-run performance.
problem LLMs lack validation against statistical ground truth in automated scientific workflows.
method Introduced a behavioral evaluation framework for LLMs, separating four decision-making dimensions.
result LLMs can exhibit near-perfect stability but diverge from statistical ground truth.
FUSE improves verification quality without ground truth labels.
problem Verification of model outputs using imperfect judges and reward models.
method Ensembling verifiers without ground truth labels using spectral algorithms.
result FUSE matches or improves upon semi-supervised alternatives in test-time scaling experiments.
Improved generalization with iterative self-distillation using weighted ground-truth targets.
problem Improving generalization accuracy in neural networks.
method Iterative kernel regression with weighted ground-truth targets and regularization.
result Closed-form solution for optimal weighting parameter and efficient estimation.
Polymarket-v1 Database tracks 1.2B trades across 1.3M markets with 100% ground-truth direction.
problem Lack of ground-truth data in prediction markets archives.
method Ground-truth archive of 1.2B trades from Polymarket's CTF Exchange.
result Ground-truth data reveals systematic errors in microstructure metrics.
New research shows the maximum ℓ1-margin classifier doesn't adapt to sparse ground truths.
problem Understanding the limitations of the maximum ℓ1-margin classifier in high-dimensional settings.
method Analyzing convergence and prediction error rates of the maximum ℓ1-margin classifier.
result Proves tight upper and lower bounds for prediction error, showing benign overfitting.
A new method finds DAG models without ground truth.
problem Finding DAG models without ground truth.
method Markov Checker test and Cross-Algorithm Frugality Search (CAFS).
result Models can be chosen without ground truth.
The paper analyzes GNNs with one hidden layer, proving their generalizability and convergence rate.
problem Theoretical guarantee on generalizability of GNNs with one hidden layer.
method Tensor initialization and accelerated gradient descent.
result The proposed learning algorithm converges to the ground-truth GNN model for regression and to a model close to the ground-truth for binary classification.
A method for inferring ground-truth signals from degraded sensor data.
problem Inferring ground-truth signals from multiple degraded sensor signals.
method Iterative correction of degraded signals using a Bayesian multi-sensor data fusion method.
result The method effectively infers ground-truth signals from noisy and degraded sensor data.
Regression analysis is a standard supervised machine learning method used to model an outcome variable in terms of a set of predictor variables. In most real-world applications we do not know the true value of the outcome variable being predicted outside the training data, i.e., the ground truth is unknown. It is hence…
Separating an audio scene such as a cocktail party into constituent, meaningful components is a core task in computer audition. Deep networks are the state-of-the-art approach. They are trained on synthetic mixtures of audio made from isolated sound source recordings so that ground truth for the separation is known. Ho…
Reanalysis datasets combining numerical physics models and limited observations to generate a synthesised estimate of variables in an Earth system, are prone to biases against ground truth. Biases identified with the NASA Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2) aerosol optic…
PoGDiff improves diffusion models on imbalanced datasets.
problem Diffusion models struggle with imbalanced text-image pairs.
method PoGDiff replaces ground-truth with a Product of Gaussians.
result PoGDiff enhances generation accuracy and quality on imbalanced datasets.
Unsupervised Recalibrationstat.ML
Unsupervised recalibration (URC) is a general way to improve the accuracy of an already trained probabilistic classification or regression model upon encountering new data while deployed in the field. URC does not require any ground truth associated with the new field data. URC merely observes the model's predictions a…
Provable Alternating Gradient Descent for Non-negative Matrix Factorization with Strong Correlationscs.LG
Non-negative matrix factorization is a basic tool for decomposing data into the feature and weight matrices under non-negativity constraints, and in practice is often solved in the alternating minimization framework. However, it is unclear whether such algorithms can recover the ground-truth feature matrix when the wei…
The paper analyzes how noise affects distances in high-dimensional data and when they remain useful.
problem Noise corrupts distances in high-dimensional data, making them unreliable for identifying true nearest and farthest neighbors.
method The paper uses asymptotic probabilistic expressions to characterize noise effects and decomposes data into ground truth and noise components.
result Under certain conditions, empirical neighborhood relations remain truthful even when distance concentration occurs.
Proposes a method for explaining tabular data using copulas.
problem Lack of ground truth for explainability in complex datasets.
method Uses copulas to specify statistical properties and build intuition.
result Demonstrates improved explainability on logistic regression and correlation use cases.
Neural Machine Translation (NMT) generates target words sequentially in the way of predicting the next word conditioned on the context words. At training time, it predicts with the ground truth words as context while at inference it has to generate the entire sequence from scratch. This discrepancy of the fed context l…
We give a new algorithm for learning a two-layer neural network under a general class of input distributions. Assuming there is a ground-truth two-layer network where are weight matrices, represents noise, and the number of neurons in the hidden layer is no larger than the input or outp…
The paper validates a method for recovering over-parameterized matrices and images from noisy measurements.
problem Recovering a low-rank matrix from noisy measurements when the rank is unknown.
method Using gradient descent with small random initialization on a nonconvex objective function built from a rank-overspecified factored representation of the matrix variable.
result Gradient descent iterations converge to the ground-truth matrix under certain conditions and can be stopped efficiently to detect a nearly optimal estimator.
Study robustness of conformal prediction to label noise in regression and classification.
problem Robustness of conformal prediction to label noise in regression and classification.
method Characterized robustness of conformal prediction for both regression and classification problems, extending theory to control general loss functions.
result Conformal prediction and risk-controlling techniques can achieve conservative risk over clean ground truth labels with noisy labels.
TOO optimizes stochastic epidemiological models by finding both parameter settings and random seeds.
problem Calibrating stochastic epidemiological models to match empirical observations.
method Gaussian process surrogates and Thompson sampling for optimization.
result Produces actual trajectories consistent with ground truth.
The accurate measurement of security metrics is a critical research problem because an improper or inaccurate measurement process can ruin the usefulness of the metrics, no matter how well they are defined. This is a highly challenging problem particularly when the ground truth is unknown or noisy. In contrast to the w…
Trajectory or behavior prediction of traffic agents is an important component of autonomous driving and robot planning in general. It can be framed as a probabilistic future sequence generation problem and recent literature has studied the applicability of generative models in this context. The variety or Minimum over …
It has been noticed that some external CVIs exhibit a preferential bias towards a larger or smaller number of clusters which is monotonic (directly or inversely) in the number of clusters in candidate partitions. This type of bias is caused by the functional form of the CVI model. For example, the popular Rand index (R…