arXiv research

We solve the multi-criteria benchmarking problem by formalizing it as a social choice problem and identifying conditions for meaningful rankings.

The heuristic identification of peaks from noisy complex spectra often leads to misunderstanding of the physical and chemical properties of matter. In this paper, we propose a framework based on Bayesian inference, which enables us to separate multipeak spectra into single peaks statistically and consists of two steps.…

WSqD extends learning rate schedules for large model training without fixed horizons.

Optimizes molecular generation for chemist preferences.

New method adapts to user preferences dynamically, improving recommendation models.

Many real-world engineering problems rely on human preferences to guide their design and optimization. We present PrefOpt, an open source package to simplify sequential optimization tasks that incorporate human preference feedback. Our approach extends an existing latent variable model for binary preferences to allow f…

Enhances preference learning by incorporating response times into binary choices.

Bayesian optimization learns DM preferences for multi-outcome experiments.

New study shows personalized content recommendations can lead to polarization of user preferences.

Bayesian optimization agent learns user preferences from pairwise comparisons.

New RLHF framework handles general preference oracles without reward functions.

This paper studies robust forward investment and consumption preferences within a zero-volatility context. Different from previous works, we consider an incomplete financial market model due to general investment portfolio constraints. We provide a new PDE characterization and a novel semi-explicit saddle-point constru…

In preference-based reinforcement learning (RL), an agent interacts with the environment while receiving preferences instead of absolute feedback. While there is increasing research activity in preference-based RL, the design of formal frameworks that admit tractable theoretical analysis remains an open challenge. Buil…

Study on identifying most preferred policy in bandits with vector-valued rewards.

Stable and consistent model alignment for language models without assuming human preference models.

Dropping a tiny fraction of preferences can significantly alter the rankings of top LLMs.

Paper explores limits and possibilities of aligning LLMs with human preferences.

Paper improves parameter estimation of continuous distributions using preference feedback.

Paper investigates monotonicity issues in AI preference learning.

DOPL learns from preference feedback to solve RMAB problems.

Direct Density Ratio Optimization aligns LLMs with human preferences without assuming specific models.

Bayesian optimization with preference learning identifies preferred solutions in multi-objective problems.

Training models to prefer certain responses can unintentionally shift probability to harmful ones.

This work proves win rate is key to understanding preference learning.

IDT learns human preferences from uncertain decisions, even when humans are suboptimal.

The paper proposes a method to infer multi-objective rewards from preferences.

The paper explores game-theoretic alignment of LLMs with human preferences, finding limitations and conditions.

Fashion preference is a fuzzy concept that depends on customer taste, prevailing norms in fashion product/style, henceforth used interchangeably, and a customer's perception of utility or fashionability, yet fashion e-retail relies on algorithmically generated search and recommendation systems that process structured d…

Bal-PM reduces preference labeling costs for LLMs.

DPA aligns LLMs with multi-objective rewards for diverse user preferences.

Tutorials on preference learning with Gaussian Processes.

Robot motions in the presence of humans should not only be feasible and safe, but also conform to human preferences. This, however, requires user feedback on the robot's behavior. In this work, we propose a novel approach to leverage the user's brain signals as a feedback modality in order to decode the judgment of rob…

Given an incomplete ratings data over a set of users and items, the preference completion problem aims to estimate a personalized total preference order over a subset of the items. In practical settings, a ranked list of top-$k$ items from the estimated preference order is recommended to the end user in the decreasing …

Improved DPO framework penalizes preference uncertainty to avoid overoptimization.

Response time improves alignment with diverse human preferences.

Derives a new objective to learn from human preferences without approximations.

Investor optimizes portfolio under dynamic risk preferences.

Study on tracking preference shifts in dueling bandits problems.

Paper analyzes finite-time guarantees for preference-based RL.

A new, computationally friendly formula for a class of risk-averse preferences.

Model learns metrics and preferences from user comparisons.

We state the problem of inverse reinforcement learning in terms of preference elicitation, resulting in a principled (Bayesian) statistical formulation. This generalises previous work on Bayesian inverse reinforcement learning and allows us to obtain a posterior distribution on the agent's preferences, policy and optio…

Pref-SHAP explains preferences using Shapley values.

Paper shows RLHF can be solved similarly to standard RL.

Paper tackles combinatorial reinforcement learning with preference feedback.

New methods improve LLM preference optimization by intelligently weighting multiple reference models.

New RLHF approach mitigates bias in aligning LLMs with human preferences.

MallowsPO enhances LLM fine-tuning with a dispersion index of human preferences.