Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

144287431574 · Jun 202019922001200920172026
48 results for Distributional preference

Paper improves parameter estimation of continuous distributions using preference feedback.

problem Improving parameter estimation of continuous distributions.
method Preference-based M-estimators and deterministic preferences.
result Preference-based estimators achieve an estimation error scaling of O(1/n), significantly faster than sample-only methods.

Study on tracking preference shifts in dueling bandits problems.

problem Tracking significant preference shifts in dueling bandits problems.
method Analysis of dueling bandits with distribution shifts, focusing on significant shifts (Suk and Kpotufe, 2022).
result Design of adaptive algorithms with O(KildeLT)O(\sqrt{K ilde{L}T}) dynamic regret for certain preference distribution classes.

Best-of-N sampling reveals reward targets from preference data, influencing N and base distribution choices.

problem Understanding reward extraction from Best-of-N preference data and optimal N and base distribution choices.
method Specialized analysis of preference data via induced conditional distribution, deriving reward targets and design principles.
result Reward targets are explicit functions of N and base distribution, and bounded-class minimizers approach these targets as N grows.

Stable and consistent model alignment for language models without assuming human preference models.

problem Lack of statistical consistency in existing alignment methods.
method Relative density ratio optimization between preferred and mixture of preferred and non-preferred data distributions.
result Our approach achieves statistical consistency and stability, providing tighter convergence guarantees.

New method accounts for hidden context in preference learning for RLHF models.

problem Incomplete data with hidden context affects RLHF model outcomes.
method Distributional Preference Learning (DPL) methods estimate hidden context distributions.
result DPL methods reduce RLHF vulnerabilities by accounting for hidden context.

Direct Density Ratio Optimization aligns LLMs with human preferences without assuming specific models.

problem Statistical inconsistency in aligning LLMs with human preferences.
method Direct Density Ratio Optimization (DDRO) estimates density ratio directly.
result DDRO is statistically consistent, converging to true human preferences as data grows.

The paper proposes a method to learn and leverage contextual preference distributions for better decision-making.

problem Heterogeneous and context-dependent human preferences in decision-making problems.
method A sequential learning-and-optimization pipeline using a bounded-variance score function gradient estimator to train a predictive model mapping contextual features to preference distributions.
result The approach reduces average post-decision surprise by up to 25 times compared to risk-averse baselines in a ridesharing environment.

A new method for incorporating preferences in multi-objective Bayesian optimization.

problem Incorporating preferences in computationally expensive multi-objective optimization problems.
method Building independent surrogate models on each objective function and using Generalised value distribution to approximate the scalarizing function.
result The proposed multi-surrogate approach outperforms the mono-surrogate approach on benchmark and real-world problems.

Investigates how diversification preferences relate to risk attitudes.

problem Connecting diversification preferences to risk attitudes.
method Analyzes diversification preferences for various pairs of risks under different conditions.
result Diversification preferences for certain pairs of risks imply specific levels of risk aversion.

Study explores how wealth dynamics change with preferential interactions in kinetic exchange models.

problem Investigate how preferential interactions affect wealth dynamics and distributions in kinetic exchange models.
method Conducted Monte Carlo simulations to explore two types of preferential interactions: one with random selection and another with wealth difference constraint.
result Emergence of quasi-oligarchic societies and segregation into economic classes observed in preferential interactions.

New RLHF approach mitigates bias in aligning LLMs with human preferences.

problem Algorithmic bias in RLHF leading to preference collapse.
method Preference Matching (PM) RLHF, using PM regularizer and conditional variant.
result 29% to 41% improvement in alignment with human preferences.

In preference-based reinforcement learning (RL), an agent interacts with the environment while receiving preferences instead of absolute feedback. While there is increasing research activity in preference-based RL, the design of formal frameworks that admit tractable theoretical analysis remains an open challenge. Buil…

2019-08-04abs ↗pdf ↗

Study optimal investment decisions for diverse risk-tolerant agents.

problem Optimizing investment choices for agents with varying risk preferences.
method Characterizes optimal behavior using certainty equivalents and lognormal risks.
result Derives optimal decision menus under known and uncertain preference distributions.

Study on identifying most preferred policy in bandits with vector-valued rewards.

problem Identifying the most preferred policy in bandits with vector-valued rewards.
method Derive a novel lower bound on sample complexity, design the Preference-based Track and Stop (PreTS) algorithm, and derive a new concentration inequality.
result The sample complexity of PreTS is asymptotically tight.

The paper addresses biased preferences in candidate selection, proposing a fair and utility-maximizing algorithm.

problem Selecting candidates for institutions with biased preferences and limited capacities.
method An algorithm that considers group fairness and true utility, proving near-optimal results under distributional assumptions.
result The proposed algorithm achieves near-optimal group fairness and near-maximal true utility, even in biased settings.

We state the problem of inverse reinforcement learning in terms of preference elicitation, resulting in a principled (Bayesian) statistical formulation. This generalises previous work on Bayesian inverse reinforcement learning and allows us to obtain a posterior distribution on the agent's preferences, policy and optio…

2011-04-29abs ↗pdf ↗

Mitigates overoptimization in RLHF by reformulating SFT loss as a preference optimization loss.

problem Overoptimization in RLHF where reward model misguides generative model.
method Proposes a theoretical algorithm that minimizes maximum likelihood estimation and reward penalty, reformulates as simple objective combining preference optimization and supervised learning losses.
result Improved performance of RPO compared to DPO baselines in aligning LLMs.

Unified Skew-Gaussian process framework for various regression and classification tasks.

problem Handling multiple types of regression and classification problems.
method Generalization of Skew-Gaussian processes to handle various types of data and likelihoods.
result Closed-form posterior distributions for multiple tasks.

Develops a statistical framework to measure uncertainty in model rankings based on human preferences.

problem Uncertainty in model rankings based on human preferences due to mismatch between human and model preferences.
method Statistical framework using pairwise comparisons by humans and models to provide rank-sets for each model.
result Rank-sets constructed using only pairwise comparisons by strong models often do not cover the true ranking of human preferences.

Proposes a novel algorithm for multi-objective reinforcement learning.

problem Challenges in setting numerical preferences for objectives in different units and scales.
method Learn action distributions for each objective and use supervised learning to fit a parametric policy.
result Demonstrates effectiveness on robotics tasks, allowing tracing out the space of nondominated solutions.

The goal of task transfer in reinforcement learning is migrating the action policy of an agent to the target task from the source task. Given their successes on robotic action planning, current methods mostly rely on two requirements: exactly-relevant expert demonstrations or the explicitly-coded cost function on targe…

2018-05-12abs ↗pdf ↗

Bal-PM reduces preference labeling costs for LLMs.

problem Efficiently acquiring human feedback for preference modeling in large language models.
method Bayesian Active Learning with entropy maximization in feature space.
result Bal-PM reduces the number of required preference labels by 33% to 68%.

Investment strategy in uncertain markets improved by learning and risk-ambiguity preferences.

problem Investment in financial markets with unknown drift coefficients.
method Optimization under KMM approach, considering risk and ambiguity preferences.
result Optimal investment strategy can be adjusted based on prior drift distribution.

Extends reinforcement learning alignment to scalar rewards, improving math reasoning.

problem Designing reinforcement learning algorithms for general LLM alignment.
method Introduces f-GRPO and f-HAL, estimating f-divergences between reward-aligned and unaligned distributions.
result Improves math reasoning RLVR tasks and mitigates reward hacking.

Bayesian optimization agent learns user preferences from pairwise comparisons.

problem Learning user preferences from unknown and infinite choices.
method Sequential Bayesian optimization with pairwise comparisons.
result Optimal agent strategy minimizes remaining system uncertainty.

Derives a new objective to learn from human preferences without approximations.

problem Learning from human preferences through RLHF relies on approximations that can lead to pitfalls.
method Derives a new general objective ΨΨPO that bypasses both approximations.
result Demonstrates the superiority of the new objective to Direct Preference Optimisation (DPO) empirically.

The Wallenius distribution is a generalisation of the Hypergeometric distribution where weights are assigned to balls of different colours. This naturally defines a model for ranking categories which can be used for classification purposes. Since, in general, the resulting likelihood is not analytically available, we a…

2017-01-27abs ↗pdf ↗

Investor optimizes portfolio under dynamic risk preferences.

problem Optimizing investment under uncertain future risk attitudes.
method Developed a general equilibrium framework and solved for subgame-perfect equilibrium policies.
result Equilibrium policies include a novel hedging component to counteract anticipated risk aversion changes.

The paper solves portfolio selection for complex preferences in continuous time.

problem Dynamic portfolio selection for nonlinear preferences with time inconsistency.
method Stochastic maximum principle and verification theorems for equilibrium strategies.
result Equilibrium strategies derived in closed form for CRRA and CARA preferences.

This paper improves sample efficiency for off-policy evaluation with preference data.

problem Improving sample efficiency for off-policy evaluation with preference data.
method Using a deep neural network to learn the value function and leveraging manifold structure.
result Established a provably efficient guarantee for off-policy evaluation with RLHF.

Unified framework for aligning LLMs from human feedback.

problem Lack of strong theoretical justification for RLHF and inability to compare methods.
method Reframed alignment as distribution learning from pairwise preferences, proposing three principled objectives.
result Proposed objectives achieve strong non-asymptotic convergence to target LM.

The paper designs tests for comparing ranked preference data and finds significant differences.

problem Comparing pairwise comparison and ranking data in various applications.
method Developed two-sample tests for pairwise comparison and ranking data, proving upper and lower bounds.
result Upper and lower bounds show tightness of the proposed tests, and significant differences in preferences were found.

Paper proposes f-DPG for aligning language models with preferences.

problem Aligning language models with user preferences.
method Uses f-divergence to approximate target distributions and minimizes a forward KL from it using DPG.
result Jensen-Shannon divergence often outperforms forward KL divergence, leading to significant improvements.

RLHF performs well despite violating social choice theory axioms.

problem RLHF's empirical success contradicts social choice theory axioms.
method Showed RLHF satisfies pairwise majority and Condorcet consistency under mild assumptions, and introduced new alignment criteria.
result RLHF satisfies pairwise majority and Condorcet consistency under mild assumptions, explaining its practical success.

Paper proposes a new framework for combining investment strategies without market-specific assumptions.

problem Lack of a distribution-free and consistent preference framework for decision-making in combining investment strategies.
method Introduces a novel framework for decision-making in combining strategies, free from market conditions and statistical assumptions.
result Proposed strategies outperform individual component strategies in long-term wealth accumulation, with small tradeoffs in Sharpe ratios.