Study on deep neural networks for reward modeling with pairwise comparison data.
problem Reward modeling with deep neural networks in non-parametric settings.
method Established a non-asymptotic regret bound for deep reward estimators, introduced a margin-type condition.
result Improved regret bound for deep reward estimators, highlighting the importance of clear human beliefs.
Study reveals how people perceive their carbon footprint.
problem Understanding people's perception of their carbon footprint.
method Statistical model and active-learning approach to pairwise comparisons.
result Early results show promising directions for climate communication and mitigation.
RL-LOW algorithm achieves exponential simple regret in offline RLHF with pairwise comparisons.
problem Offline reinforcement learning from human feedback with pairwise comparisons.
method Proposes RL-LOW algorithm to minimize simple regret with exponential convergence.
result Achieves an exponential form of simple regret of \(\exp ( - Ω(n/H) )\).
This paper tackles bandit optimization with a new pairwise comparison oracle for unknown strongly concave functions.
problem Maximizing an unknown strongly concave function over T periods with a biased pairwise comparison oracle.
method Introduced a discretization technique and local polynomial approximation to relate the problem to linear bandits. Developed a tournament successive elimination technique to localize the discretized cell and run LinUCB algorithm on cells.
result Established optimal regret bounds and improved state-of-the-art results in operations management problems.
We study the problem of ranking from crowdsourced pairwise comparisons. Answers to pairwise tasks are known to be affected by the position of items on the screen, however, previous models for aggregation of pairwise comparisons do not focus on modeling such kind of biases. We introduce a new aggregation model factorBT …
In this study, a pairwise comparison matrix is generalized to the case when coefficients create Lie group G, non necessarily abelian. A necessary and sufficient criterion for pairwise comparisons matrices to be consistent is provided. Basic criteria for finding a nearest consistent pairwise comparisons matrix (extend…
Develops a statistical framework to measure uncertainty in model rankings based on human preferences.
problem Uncertainty in model rankings based on human preferences due to mismatch between human and model preferences.
method Statistical framework using pairwise comparisons by humans and models to provide rank-sets for each model.
result Rank-sets constructed using only pairwise comparisons by strong models often do not cover the true ranking of human preferences.
This paper examines the problem of ranking a collection of objects using pairwise comparisons (rankings of two objects). In general, the ranking of n objects can be identified by standard sorting methods using nlog2n pairwise comparisons. We are interested in natural situations in which relationships among the o…
Recent work by Locatello et al. (2018) has shown that an inductive bias is required to disentangle factors of interest in Variational Autoencoder (VAE). Motivated by a real-world problem, we propose a setting where such bias is introduced by providing pairwise ordinal comparisons between instances, based on the desired…
Paper establishes statistical inference for pairwise comparison models.
problem Statistical inference for pairwise comparison models when the number of subjects diverges.
method Identifies Fisher information matrix as a weighted graph Laplacian for asymptotic normality.
result Near-optimal asymptotic normality result for maximum likelihood estimator.
Rank regression from pairwise comparisons requires many comparisons to accurately learn model parameters.
problem Learning model parameters for rank regression from noisy pairwise comparisons.
method Uniform random pairwise comparisons to estimate model parameters with a given accuracy.
result Learning model parameters requires a number of comparisons proportional to dNlog3N/ε2. Optimal privacy-preserving ranking from noisy comparisons.
problem Protecting individual privacy in ranking from noisy comparisons.
method Differentially private ranking algorithms under edge and individual differential privacy.
result Achieved minimax optimal rates of convergence under privacy constraints.
A faster algorithm for ranking from pairwise comparisons.
problem Efficiently ranking individuals or objects from pairwise comparisons.
method An alternative and simpler iterative algorithm for ranking that converges faster.
result The new algorithm is over 100 times faster in some cases.
New model for pairwise comparisons without stochastic transitivity.
problem Suboptimal performance of models assuming stochastic transitivity in real-world scenarios.
method Proposes a general family of statistical models using a skew-symmetric matrix.
result Achieves minimax-rate optimality and adapts to data sparsity.
Paper proposes using pairwise feature comparisons to infer modification costs for user recourse.
problem Learning and inferring user preferences for modifying features in black-box models.
method Bradley-Terry model for inferring feature-wise costs from non-exhaustive human comparison surveys.
result Non-exhaustive human surveys can efficiently learn feature costs, enabling recourse finding.
Paper proposes Pcomp classification for binary classification with pairwise confidence comparisons.
problem Lack of pointwise labels due to privacy, confidentiality, or security reasons.
method Developed Pcomp classification, derived an unbiased risk estimator (URE), and improved it using correction functions and consistency regularization.
result Demonstrated the effectiveness of Pcomp classification methods.
This paper provides a block coordinate descent algorithm to solve unconstrained optimization problems. In our algorithm, computation of function values or gradients is not required. Instead, pairwise comparison of function values is used. Our algorithm consists of two steps; one is the direction estimate step and the o…
Given a set of pairwise comparisons, the classical ranking problem computes a single ranking that best represents the preferences of all users. In this paper, we study the problem of inferring individual preferences, arising in the context of making personalized recommendations. In particular, we assume that there are …
Breaks the hardness conjecture for batch RL with a novel tournament-based approach.
problem Sample-efficient reinforcement learning from exploratory data.
method BVFT algorithm using pairwise comparison and state-action partition.
result Solves the learning problem in a setting previously thought impossible.
Study learns linear utility functions from comparisons, showing learnability gaps between passive and active learning.
problem Learn linear utility functions from pairwise comparison queries.
method Analyzes passive and active learning settings, considering noise-free and noisy query responses.
result Efficient learnability of linear utilities in passive learning, but not for utility parameters without strong assumptions.
We study the problem of interactively learning a binary classifier using noisy labeling and pairwise comparison oracles, where the comparison oracle answers which one in the given two instances is more likely to be positive. Learning from such oracles has multiple applications where obtaining direct labels is harder bu…
Active sampling algorithm improves accuracy of inferred scores from pairwise comparisons.
problem Inference of accurate scores from time-consuming pairwise comparisons.
method Approximate message passing and expected information gain maximization.
result ASAP offers the highest accuracy of inferred scores compared to existing methods.
The paper designs tests for comparing ranked preference data and finds significant differences.
problem Comparing pairwise comparison and ranking data in various applications.
method Developed two-sample tests for pairwise comparison and ranking data, proving upper and lower bounds.
result Upper and lower bounds show tightness of the proposed tests, and significant differences in preferences were found.
This paper optimizes the number of comparisons needed to find the best k items from pairwise comparisons.
problem Finding the best k items from pairwise comparisons with limited comparisons.
method Developed algorithms for finding probably approximately correct and exact best k items under stochastic conditions.
result Upper and lower bounds on the number of comparisons for finding the best k items, with matching upper bounds for PAC best k items.
Improved sample efficiency in preference-based RL with multiple comparisons.
problem Sample inefficiency in preference-based reinforcement learning with pairwise comparisons.
method Proposes M-AUPO, an algorithm that selects multiple actions by maximizing average uncertainty within subsets.
result Achieves a suboptimality gap of $O\left( \frac{d}{T} \sqrt{ \sum_{t=1}^T \frac{1}{|S_t|}}
ight)$, improving performance with larger subsets.
In this paper, we propose an active learning algorithm and models which can gradually learn individual's preference through pairwise comparisons. The active learning scheme aims at finding individual's most preferred choice with minimized number of pairwise comparisons. The pairwise comparisons are encoded into probabi…
New model accounts for scale variation and noise in pairwise comparisons.
problem Nonreciprocal pairwise comparisons in decision analysis.
method Additive model with structured matrix and random perturbation.
result Explicit estimators and probability assessments of admissible ranking regions.
We describe a mathematical link between aspects of information theory, called pairwise comparisons, and discretized gauge theories. The link is made by the notion of holonomy along the edges of a simplex. This correspondance leads to open questions in both field.
A new model for sparse networks improves consistency of score estimators.
problem Statistical inference in large sparse networks.
method General pairwise comparison model with flexible parametrization.
result Maximum likelihood estimator is uniformly consistent under sparse conditions.
Optimizes identifying top-k items from comparisons with minimal comparisons.
problem Finding the top-k items from pairwise comparisons with a fixed error rate.
method Developed an asymptotically optimal algorithm using primal-dual procedure and adaptive comparison allocation.
result Proves the algorithm is asymptotically optimal for top-k identification.
Unified stopping rules ensure accurate policies in contextual learning.
problem Stopping data collection to ensure accurate policies in personalized decision problems.
method Developed unified stopping rules based on GLR statistics for pairwise action comparisons.
result Unified stopping rules achieve target precision with fewer samples than benchmarks.
Comparison data arises in many important contexts, e.g. shopping, web clicks, or sports competitions. Typically we are given a dataset of comparisons and wish to train a model to make predictions about the outcome of unseen comparisons. In many cases available datasets have relatively few comparisons (e.g. there are on…
We describe a seriation algorithm for ranking a set of items given pairwise comparisons between these items. Intuitively, the algorithm assigns similar rankings to items that compare similarly with all others. It does so by constructing a similarity matrix from pairwise comparisons, using seriation methods to reorder t…
We consider sequential or active ranking of a set of n items based on noisy pairwise comparisons. Items are ranked according to the probability that a given item beats a randomly chosen item, and ranking refers to partitioning the items into sets of pre-specified sizes according to their scores. This notion of ranking …
GNNRank uses neural networks to learn global rankings from competition match data.
problem Learning global rankings from pairwise comparisons in directed graphs.
method Proposes GNNRank, a trainable GNN-based framework with digraph embedding and new objectives.
result GNNRank achieves competitive and superior performance compared to baselines.
A common problem in machine learning is to rank a set of n items based on pairwise comparisons. Here ranking refers to partitioning the items into sets of pre-specified sizes according to their scores, which includes identification of the top-k items as the most prominent special case. The score of a given item is defi…
We consider data in the form of pairwise comparisons of n items, with the goal of precisely identifying the top k items for some value of k < n, or alternatively, recovering a ranking of all the items. We analyze the Copeland counting algorithm that ranks the items in order of the number of pairwise comparisons won, an…
Bayesian model infers strengths from noisy tennis match outcomes.
problem Ranking tennis players from match outcomes.
method Bayesian approach to infer unobserved strengths and mapping function.
result Bayesian approach robust to different model specifications.
Statistical framework improves LLM chatbot ranking.
problem Improving evaluation of LLM-based chatbots through pairwise comparisons.
method Factored tie model, covariance modeling, and parameter constraints.
result Substantial improvements in modeling pairwise comparison data.
Uncoupled regression is the problem to learn a model from unlabeled data and the set of target values while the correspondence between them is unknown. Such a situation arises in predicting anonymized targets that involve sensitive information, e.g., one's annual income. Since existing methods for uncoupled regression …
Study embeds PC matrices into Grassmannian manifold for geometric interpretation.
problem Understanding algebraic consistency of pairwise comparisons matrices.
method Leverages Plücker coordinates and geometric interpretation of Grassmannian manifold.
result Algebraic consistency condition is equivalent to geometric consistency in G(2,n). There is a growing need for discrete choice models that account for the complex nature of human choices, escaping traditional behavioral assumptions such as the transitivity of pairwise preferences. Recently, several parametric models of intransitive comparisons have been proposed, but in all cases the maximum likeliho…
Study metric learning from limited preference comparisons, showing how low-dimensional structure can still reveal metric information.
problem Learning metric from limited pairwise preference comparisons.
method Ideal point model, divide-and-conquer approach for low-dimensional structure.
result Metric can be jointly identified even with limited comparisons when items exhibit low-dimensional structure.
Exact pairwise ranking is achievable but not possible under noisy comparisons.
problem Recovering the exact rank of items from noisy pairwise comparisons.
method Information-theoretic upper and lower bounds using the SST model and combinatorial arguments.
result Sharp information-theoretic bounds match in the parametric limit and outperform previous methods.
The dueling bandit problem is a variation of the classical multi-armed bandit in which the allowable actions are noisy comparisons between pairs of arms. This paper focuses on a new approach for finding the "best" arm according to the Borda criterion using noisy comparisons. We prove that in the absence of structural a…
Post hoc test for Sharpe ratio improves pairwise comparisons.
problem Improving pairwise comparisons of Sharpe ratios.
method Analogous to Tukey's test, applied after rejecting equal Signal-Noise ratios.
result Maintains nominal type I rate and is moderately powerful.
The report studies ranking from pairwise comparisons in graphs, achieving optimal error bounds and proposing efficient algorithms.
problem Ranking items from pairwise comparisons in general graphs and graphs with locality.
method Maximum likelihood estimation (MLE) and preconditioned gradient descent for general graphs; divide-and-conquer algorithms for graphs with locality.
result MLE achieves optimal error bounds in general graphs and identifies conditions for locality.
New methods rank players using covariates and comparisons, outperforming existing algorithms.
problem Ranking players based on incomplete and noisy pairwise comparisons.
method Three spectral ranking methods incorporating player covariates.
result Proposed methods outperform existing algorithms in simulations.