arXiv research

Improves off-policy evaluation with imperfect annotations.

End-to-end framework learns from imperfect annotations directly.

New method uses imperfect LLM annotations for valid statistical inference in social science.

ActiveLab improves classifier accuracy with fewer annotations by re-labeling.

The paper tackles learning from imperfect human feedback, especially in dueling bandit problems.

Investigates a Kyle model with imperfect information and risk aversion.

Imitation learning (IL) aims to learn an optimal policy from demonstrations. However, such demonstrations are often imperfect since collecting optimal ones is costly. To effectively learn from imperfect demonstrations, we propose a novel approach that utilizes confidence scores, which describe the quality of demonstrat…

Corrects bias in LLM-as-a-judge evaluations using adaptive calibration.

We study the effect of imperfect training data labels on the performance of classification methods. In a general setting, where the probability that an observation in the training dataset is mislabelled may depend on both the feature vector and the true label, we bound the excess risk of an arbitrary classifier trained…

In this paper we investigate the Follow the Regularized Leader dynamics in sequential imperfect information games (IIG). We generalize existing results of Poincaré recurrence from normal-form games to zero-sum two-player imperfect information games and other sequential game settings. We then investigate how adapting th…

Policy gradient method proves convergence in imperfect-information games.

New method robustly discovers causal relationships from imperfect data.

Simulator imperfection, often known as model error, is ubiquitous in practical data assimilation problems. Despite the enormous efforts dedicated to addressing this problem, properly handling simulator imperfection in data assimilation remains to be a challenging task. In this work, we propose an approach to dealing wi…

Study on teaching with imperfect knowledge, showing its impact on optimal teaching sets.

There has been an increased interest in multimodal language processing including multimodal dialog, question answering, sentiment analysis, and speech recognition. However, naturally occurring multimodal data is often imperfect as a result of imperfect modalities, missing entries or noise corruption. To address these c…

JPS improves joint policies for multi-agent collaboration in imperfect information games.

Algorithm learns NE in imperfect information games with imperfect feedback.

Model financial markets using open quantum systems to understand market imperfections.

Study shows annotation instrument design affects model performance in hate speech detection.

The study examines how verifier imperfections impact test-time scaling techniques.

Study learns optimal strategies in imperfect information games with self-play.

Researchers develop methods for causal inference with imperfect instrumental variables.

Paper solves learning imperfect-information games with fewer episodes.

We study pricing and superhedging strategies for game options in an imperfect market with default. We extend the results obtained by Kifer in \cite{Kifer} in the case of a perfect market model to the case of an imperfect market with default, when the imperfections are taken into account via the nonlinearity of the weal…

AURA: Adaptive Uncertainty-aware Refinement for LLM-as-a-Judge Auditing

Recently, network lasso has drawn many attentions due to its remarkable performance on simultaneous clustering and optimization. However, it usually suffers from the imperfect data (noise, missing values etc), and yields sub-optimal solutions. The reason is that it finds the similar instances according to their feature…

Active learning selects both observations and annotation precision for Gaussian Processes.

One of the problems on the way to successful implementation of neural networks is the quality of annotation. For instance, different annotators can annotate images in a different way and very often their decisions do not match exactly and in extreme cases are even mutually exclusive which results in noisy annotations a…

Bayesian framework mixes imperfect models for improved predictions.

Selective planning with imperfect models reduces harmful effects of model inadequacy.

Generative model combines multi-dimensional annotations for more accurate ground truth estimation.

RAD improves robustness to domain annotation noise without explicit domain annotations.

Survey on AL strategies for cost-effective annotation in classification.

Accurate annotation of medical image is the crucial step for image AI clinical application. However, annotating medical image will incur a great deal of annotation effort and expense due to its high complexity and needing experienced doctors. To alleviate annotation cost, some active learning methods are proposed. But …

DREAM learns optimal strategies in imperfect games without needing a simulator.

Verification helps prevent model collapse when training on synthesized data.

Study on liquidity and market efficiency in auction games with imperfect information.

This paper explores how imperfect reward models can improve online RLHF.

PTBCC improves accuracy in multi-class annotation aggregation by learning from prototype confusion matrices.

Bayesian methods improve text annotation quality.

New algorithm uses imperfect advice to improve online bipartite matching performance.

Paper proposes an efficient method for bounding box annotation in object detection.

Human annotations serve an important role in computational models where the target constructs under study are hidden, such as dimensions of affect. This is especially relevant in machine learning, where subjective labels derived from related observable signals (e.g., audio, video, text) are needed to support model trai…

A new framework improves VaR recalibration by balancing reliance on imperfect volatility proxies.

Annotation guidelines used to guide the annotation of training and evaluation datasets can have a considerable impact on the quality of machine learning models. In this study, we explore the effects of annotation guidelines on the quality of app feature extraction models. As a main result, we propose several changes to…

A popular approach for large scale data annotation tasks is crowdsourcing, wherein each data point is labeled by multiple noisy annotators. We consider the problem of inferring ground truth from noisy ordinal labels obtained from multiple annotators of varying and unknown expertise levels. Annotation models for ordinal…

Crowdlab uses classifiers to estimate consensus labels and annotator quality.

Combining LETKF and RC improves chaotic system prediction from noisy, sparse data.