Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

285684112 · Jun 202019922001200920172026
48 results for ground-truth references

It has been noticed that some external CVIs exhibit a preferential bias towards a larger or smaller number of clusters which is monotonic (directly or inversely) in the number of clusters in candidate partitions. This type of bias is caused by the functional form of the CVI model. For example, the popular Rand index (R…

2016-06-17abs ↗pdf ↗

LLMs fail to match statistical ground truth despite stable run-to-run performance.

problem LLMs lack validation against statistical ground truth in automated scientific workflows.
method Introduced a behavioral evaluation framework for LLMs, separating four decision-making dimensions.
result LLMs can exhibit near-perfect stability but diverge from statistical ground truth.

A new method samples from multi-modal distributions without hyperparameter tuning.

problem Sampling from multi-modal distributions is challenging and requires tuning hyperparameters.
method Learned Reference-based Diffusion Sampler (LRDS) that learns a reference model on high-density regions and uses it to train a diffusion-based sampler.
result LRDS best exploits prior knowledge on multi-modal distributions compared to competing algorithms.

Tactile information is important for gripping, stable grasp, and in-hand manipulation, yet the complexity of tactile data prevents widespread use of such sensors. We make use of an unsupervised learning algorithm that transforms the complex tactile data into a compact, latent representation without the need to record g…

2016-06-23abs ↗pdf ↗

A new framework evaluates LLMs by considering judge reliability.

problem Evaluating LLMs without ground truth labels can lead to biased results.
method Introduces judge-specific discrimination parameters and estimates model quality and judge reliability.
result Improves agreement with human preferences and produces calibrated uncertainty quantification.

Training autoregressive models to better predict under the test metric, instead of maximizing the likelihood, has been reported to be beneficial in several use cases but brings additional complications, which prevent wider adoption. In this paper, we follow the learning-to-search approach (Daumé III et al., 2009; Leblo…

2019-12-08abs ↗pdf ↗

Effective decision making requires understanding the uncertainty inherent in a prediction. In regression, this uncertainty can be estimated by a variety of methods; however, many of these methods are laborious to tune, generate overconfident uncertainty intervals, or lack sharpness (give imprecise intervals). We addres…

2020-02-12abs ↗pdf ↗

Conformal Alignment ensures trustworthy outputs from foundation models.

problem Ensuring outputs from foundation models align with human values in high-stakes tasks.
method A framework that trains an alignment predictor using reference data to select trustworthy outputs.
result Conformal Alignment accurately identifies trustworthy outputs via lightweight training over moderate reference data.

Exposure bias refers to the train-test discrepancy that seemingly arises when an autoregressive generative model uses only ground-truth contexts at training time but generated ones at test time. We separate the contributions of the model and the learning framework to clarify the debate on consequences and review propos…

2019-10-01abs ↗pdf ↗

We introduce a novel Deep Learning framework, which quantitatively estimates image segmentation quality without the need for human inspection or labeling. We refer to this method as a Quality Assurance Network -- QANet. Specifically, given an image and a `proposed' corresponding segmentation, obtained by any method inc…

2019-04-09abs ↗pdf ↗

Optical flow refers to the visual motion observed between two consecutive images. Since the degree of freedom is typically much larger than the constraints imposed by the image observations, the straightforward formulation of optical flow as an inverse problem is ill-posed. Standard approaches to determine optical flow…

2016-11-04abs ↗pdf ↗

The rising interest in pattern recognition and data analytics has spurred the development of innovative machine learning algorithms and tools. However, as each algorithm has its strengths and limitations, one is motivated to judiciously fuse multiple algorithms in order to find the "best" performing one, for a given da…

2017-12-08abs ↗pdf ↗

With the rapid growth in the number of scientific publications, year after year, it is becoming increasingly difficult to identify quality authoritative work on a single topic. Though there is an availability of scientometric measures which promise to offer a solution to this problem, these measures are mostly quantita…

2019-10-07abs ↗pdf ↗

Universally valid ground truth is almost impossible to obtain or would come at a very high cost. For supervised learning without universally valid ground truth, a recommended approach is applying crowdsourcing: Gathering a large data set annotated by multiple individuals of varying possibly expertise levels and inferri…

2018-07-31abs ↗pdf ↗

Paper proposes a method to recover accurate labels from partially valid data in multi-label learning.

problem Tackles noisy supervision in multi-label learning with partially valid labels.
method Develops a two-stage method that estimates label enrichment and ground-truth confidences.
result Demonstrates improved performance over state-of-the-art PML methods.

Continuous monitoring of cardiac health under free living condition is crucial to provide effective care for patients undergoing post operative recovery and individuals with high cardiac risk like the elderly. Capacitive Electrocardiogram (cECG) is one such technology which allows comfortable and long term monitoring t…

2019-03-29abs ↗pdf ↗

RealCause provides a realistic benchmark for causal inference.

problem Lack of a reliable benchmark for comparing causal effect estimators.
method Flexible generative models to create a benchmark that is both ground-truth and realistic.
result Evaluation of over 1500 causal estimators provides evidence for choosing hyperparameters using predictive metrics.

Recently developed deep-learning-based denoisers often outperform state-of-the-art conventional denoisers such as the BM3D. They are typically trained to minimize the mean squared error (MSE) between the output image of a deep neural network (DNN) and a ground truth image. Thus, it is important for deep-learning-based …

2018-03-04abs ↗pdf ↗

Estimates classifier errors without ground truth using algebraic geometry.

problem Lack of ground truth in real-world production systems.
method Non-parametric estimation using algebraic geometry to solve the self-assessment problem.
result Accuracy estimators are better than one part in a hundred.

Paper introduces metrics to evaluate missing data imputation without ground truth.

problem Handling missing data in time series without ground truth.
method Introduces Wasserstein distance (WD) and Jensen-Shannon divergence (JSD) as metrics to evaluate imputation quality.
result WD and JSD are effective metrics for assessing missing data imputation quality.

QUACKIE creates a new benchmark for NLP interpretability.

problem Evaluating NLP interpretability methods is challenging due to biased ground truths.
method Formulated a custom classification task from question-answering datasets, generating unbiased ground truths.
result Demonstrated the effectiveness of current interpretability methods on the new benchmark.

Unsupervised clustering can reproduce categorization systems if features and metrics are correctly selected.

problem Reproducing expert-provided categorization systems using unsupervised clustering.
method Investigated using toy datasets and real-world fund categorization. Used appropriate feature selection and a supervised Random Forest-based distance metric.
result Unsupervised clustering can reproduce ground truth classes if features and metrics are correctly selected.

Generative model combines multi-dimensional annotations for more accurate ground truth estimation.

problem Inaccurate ground truth estimation from naive annotators' multi-dimensional annotations.
method Proposes a joint multi-dimensional model for global and time-series annotation fusion using Expectation-Maximization algorithm.
result More accurate ground truth estimates through joint modeling of multiple dimensions.

This work optimizes signal estimation for sparse MRA with collision-free signals.

problem Recovering an unknown signal from repeated observations under cyclic isometries with high noise.
method Investigates minimax optimality for collision-free signals in the MRA model.
result The minimax optimal rate of estimation is \( \sigma^2/\sqrt{n} \) for sparse MRA.

SAMPLR optimizes for ground truth in aleatoric parameters to avoid curriculum-induced covariate shift.

problem Curriculum learning shifts training distribution, leading to suboptimal policies in aleatoric settings.
method SAMPLR optimizes ground-truth utility function, avoiding curriculum-induced covariate shift.
result SAMPLR preserves optimality under ground-truth distribution, promoting robustness across various environments.

Autonomy and adaptation of machines requires that they be able to measure their own errors. We consider the advantages and limitations of such an approach when a machine has to measure the error in a regression task. How can a machine measure the error of regression sub-components when it does not have the ground truth…

2019-06-17abs ↗pdf ↗

BoostMD accelerates molecular dynamics simulations by 8x with ML force fields.

problem Long inference times of ML force fields limit practical use in molecular dynamics.
method BoostMD uses previous time-step features to predict energies and forces, reducing complexity and computational cost.
result BoostMD achieves an 8-fold speedup and accurately samples the Boltzmann distribution.

Preconditioned SGD accelerates convergence for ill-conditioned huge-scale matrix completion.

problem Recovering a low-rank matrix from incomplete data with high condition number.
method Preconditioned Stochastic Gradient Descent (SGD) for huge-scale online optimization.
result Preconditioned SGD converges to ε-accuracy in O(log(1/ε)) iterations, compared to O(κlog(1/ε)) for unpreconditioned SGD.

Study reveals issues with neural autoregressive models and proposes mode recovery cost.

problem Unreasonable affinity of neural autoregressive models to short and long sequences.
method Investigates modes of ground-truth, empirical, and decoding-induced distributions via mode recovery cost.
result Mode recovery cost varies depending on ground-truth distribution and impacts decoding-induced distribution.

It is widely believed that sharing gradients will not leak private training data in distributed learning systems such as Collaborative Learning and Federated Learning, etc. Recently, Zhu et al. presented an approach which shows the possibility to obtain private training data from the publicly shared gradients. In their…

2020-01-08abs ↗pdf ↗