Study on deep neural networks for reward modeling with pairwise comparison data.
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Trend · papers per month
RL-LOW algorithm achieves exponential simple regret in offline RLHF with pairwise comparisons.
This paper tackles bandit optimization with a new pairwise comparison oracle for unknown strongly concave functions.
We study the problem of ranking from crowdsourced pairwise comparisons. Answers to pairwise tasks are known to be affected by the position of items on the screen, however, previous models for aggregation of pairwise comparisons do not focus on modeling such kind of biases. We introduce a new aggregation model factorBT …
In this study, a pairwise comparison matrix is generalized to the case when coefficients create Lie group , non necessarily abelian. A necessary and sufficient criterion for pairwise comparisons matrices to be consistent is provided. Basic criteria for finding a nearest consistent pairwise comparisons matrix (extend…
We propose a statistical model to understand people's perception of their carbon footprint. Driven by the observation that few people think of CO2 impact in absolute terms, we design a system to probe people's perception from simple pairwise comparisons of the relative carbon footprint of their actions. The formulation…
Develops a statistical framework to measure uncertainty in model rankings based on human preferences.
This paper examines the problem of ranking a collection of objects using pairwise comparisons (rankings of two objects). In general, the ranking of objects can be identified by standard sorting methods using pairwise comparisons. We are interested in natural situations in which relationships among the o…
Recent work by Locatello et al. (2018) has shown that an inductive bias is required to disentangle factors of interest in Variational Autoencoder (VAE). Motivated by a real-world problem, we propose a setting where such bias is introduced by providing pairwise ordinal comparisons between instances, based on the desired…
Paper establishes statistical inference for pairwise comparison models.
Rank regression from pairwise comparisons requires many comparisons to accurately learn model parameters.
Optimal privacy-preserving ranking from noisy comparisons.
A faster algorithm for ranking from pairwise comparisons.
New model for pairwise comparisons without stochastic transitivity.
Paper proposes using pairwise feature comparisons to infer modification costs for user recourse.
Paper proposes Pcomp classification for binary classification with pairwise confidence comparisons.
This paper provides a block coordinate descent algorithm to solve unconstrained optimization problems. In our algorithm, computation of function values or gradients is not required. Instead, pairwise comparison of function values is used. Our algorithm consists of two steps; one is the direction estimate step and the o…
Given a set of pairwise comparisons, the classical ranking problem computes a single ranking that best represents the preferences of all users. In this paper, we study the problem of inferring individual preferences, arising in the context of making personalized recommendations. In particular, we assume that there are …
Breaks the hardness conjecture for batch RL with a novel tournament-based approach.
Study learns linear utility functions from comparisons, showing learnability gaps between passive and active learning.
We study the problem of interactively learning a binary classifier using noisy labeling and pairwise comparison oracles, where the comparison oracle answers which one in the given two instances is more likely to be positive. Learning from such oracles has multiple applications where obtaining direct labels is harder bu…
Active sampling algorithm improves accuracy of inferred scores from pairwise comparisons.
The paper designs tests for comparing ranked preference data and finds significant differences.
This paper optimizes the number of comparisons needed to find the best k items from pairwise comparisons.
Improved sample efficiency in preference-based RL with multiple comparisons.
In this paper, we propose an active learning algorithm and models which can gradually learn individual's preference through pairwise comparisons. The active learning scheme aims at finding individual's most preferred choice with minimized number of pairwise comparisons. The pairwise comparisons are encoded into probabi…
New model accounts for scale variation and noise in pairwise comparisons.
We describe a mathematical link between aspects of information theory, called pairwise comparisons, and discretized gauge theories. The link is made by the notion of holonomy along the edges of a simplex. This correspondance leads to open questions in both field.
Optimizes identifying top-k items from comparisons with minimal comparisons.
Unified stopping rules ensure accurate policies in contextual learning.
Comparison data arises in many important contexts, e.g. shopping, web clicks, or sports competitions. Typically we are given a dataset of comparisons and wish to train a model to make predictions about the outcome of unseen comparisons. In many cases available datasets have relatively few comparisons (e.g. there are on…
We describe a seriation algorithm for ranking a set of items given pairwise comparisons between these items. Intuitively, the algorithm assigns similar rankings to items that compare similarly with all others. It does so by constructing a similarity matrix from pairwise comparisons, using seriation methods to reorder t…
We consider sequential or active ranking of a set of n items based on noisy pairwise comparisons. Items are ranked according to the probability that a given item beats a randomly chosen item, and ranking refers to partitioning the items into sets of pre-specified sizes according to their scores. This notion of ranking …
GNNRank uses neural networks to learn global rankings from competition match data.
A common problem in machine learning is to rank a set of n items based on pairwise comparisons. Here ranking refers to partitioning the items into sets of pre-specified sizes according to their scores, which includes identification of the top-k items as the most prominent special case. The score of a given item is defi…
We consider data in the form of pairwise comparisons of n items, with the goal of precisely identifying the top k items for some value of k < n, or alternatively, recovering a ranking of all the items. We analyze the Copeland counting algorithm that ranks the items in order of the number of pairwise comparisons won, an…
Bayesian model infers strengths from noisy tennis match outcomes.
Statistical framework improves LLM chatbot ranking.
Uncoupled regression is the problem to learn a model from unlabeled data and the set of target values while the correspondence between them is unknown. Such a situation arises in predicting anonymized targets that involve sensitive information, e.g., one's annual income. Since existing methods for uncoupled regression …
Study embeds PC matrices into Grassmannian manifold for geometric interpretation.
There is a growing need for discrete choice models that account for the complex nature of human choices, escaping traditional behavioral assumptions such as the transitivity of pairwise preferences. Recently, several parametric models of intransitive comparisons have been proposed, but in all cases the maximum likeliho…
Study metric learning from limited preference comparisons, showing how low-dimensional structure can still reveal metric information.
Exact pairwise ranking is achievable but not possible under noisy comparisons.
The dueling bandit problem is a variation of the classical multi-armed bandit in which the allowable actions are noisy comparisons between pairs of arms. This paper focuses on a new approach for finding the "best" arm according to the Borda criterion using noisy comparisons. We prove that in the absence of structural a…
Statistical inference using pairwise comparison data is an effective approach to analyzing large-scale sparse networks. In this paper, we propose a general framework to model the mutual interactions in a network, which enjoys ample flexibility in terms of model parametrization. Under this setup, we show that the maximu…
The report studies ranking from pairwise comparisons in graphs, achieving optimal error bounds and proposing efficient algorithms.
New methods rank players using covariates and comparisons, outperforming existing algorithms.
Paper quantifies uncertainty in pairwise comparison models.