Proposes a dynamic matching algorithm for two-sided online markets.
problem Dynamic preferences in two-sided online matching platforms.
method Dynamic Matching Bandit Algorithm with statistical preference ranking estimation.
result Agent-optimal stable matching result with logarithmic regret bound.
Algorithm identifies optimal stable matching in uncertain two-sided markets.
problem Sequential learning in two-sided markets with unknown preferences.
method Pure exploration approach with elimination-based algorithms exploiting partial preference information.
result Identification of pervasive stable matching for optimal stable matching identification.
New algorithms learn stable matchings from uncertain user preferences.
problem Learning stable matchings from uncertain user preferences.
method Stochastic multi-armed bandit problem, incentive-aware learning objective, primal-dual formulation.
result Near-optimal regret bounds for learning stable matchings.
The paper explores game-theoretic alignment of LLMs with human preferences, finding limitations and conditions.
problem Aligning LLMs with human preferences using game theory.
method Systematic study of payoff choices in a two-player zero-sum game for desirable alignment properties.
result Impossibility of preference matching in game-theoretic LLM alignment under standard assumptions.
Algorithm solves two-sided matching markets with unknown preferences and constraints.
problem Two-sided online matching markets with complementary preferences and quota constraints.
method Formulated as a bandit learning problem, proposed MMTS algorithm combining Thompson Sampling and double matching.
result MMTS achieves stability and linear Bayesian regret with respect to quota and time horizon.
New algorithm for learning preferences in decentralized matching markets reduces regret to logarithmic levels.
problem Learning preferences in decentralized matching markets without direct communication.
method Introduces a new algorithm for two-sided matching markets with competition.
result The algorithm achieves logarithmic stable regret in shared preferences and quadratic regret in general preferences.
Study on learning strategies in matching markets with uncertain preferences.
problem Decision-making in scarcity of shared resources with unknown agent preferences.
method Representation of preferences in a reproducing kernel Hilbert space, learning algorithm for uncertainty.
result Optimal strategies derived to maximize agents' expected payoffs, with stability and fairness properties.
Decentralized learning for matching markets with time-varying preferences.
problem Matching between competing agents and supply arms with time-varying preferences.
method Linear contextual bandit framework, learning algorithms to identify latent environment and stable matchings.
result Achieve instance-dependent logarithmic regret, applicable for large markets.
AI assistants often give convincing but incorrect responses to match user beliefs.
problem Sycophancy in AI assistants that use human feedback.
method Examined five AI assistants across four tasks, analyzed human preference data, and compared model outputs against preference models.
result Sycophancy is a general behavior of AI assistants, driven in part by human preference judgments.
RLHF performs well despite violating social choice theory axioms.
problem RLHF's empirical success contradicts social choice theory axioms.
method Showed RLHF satisfies pairwise majority and Condorcet consistency under mild assumptions, and introduced new alignment criteria.
result RLHF satisfies pairwise majority and Condorcet consistency under mild assumptions, explaining its practical success.
The paper models market dynamics using a limit order book system to explain slippage and inefficiency.
problem Inefficiency in matching markets due to structural liquidity constraints and slippage.
method Introduces a market microstructure framework with a latent preference state matrix and a dynamic discrete choice execution model.
result Persistent slippage and regional invariance of preference orderings are explained by liquidity thresholds.
Spacematch matches office workers with suitable workspaces based on their preferences.
problem Matching workers with suitable workspaces in ABW environments.
method Developed a web-based mobile app to collect occupant preferences and IoT sensors for real-time environmental feedback.
result Occupant preferences can be segmented and matched to build a recommendation platform.
Kernel ridge regression inference for nonstandard data.
problem Inferential theory for kernel ridge regression with nonstandard data.
method Constructs valid and sharp confidence sets using anti-symmetric multipliers.
result Develops a test for match effects in school matching mechanisms.
Paper shows faster core identification in matching markets.
problem Core Identification Problem in one-sided matching markets.
method Randomized SVD on preference-derived Markov matrix.
result CIP solved in O(Ln) time, matching lower bound.
Stable matching, a classical model for two-sided markets, has long been studied with little consideration for how each side's preferences are learned. With the advent of massive online markets powered by data-driven matching platforms, it has become necessary to better understand the interplay between learning and mark…
Efficiently poisons offline RLHF models by flipping preference labels.
problem Vulnerability of offline RLHF models to preference label flipping attacks.
method Developed two attack methods: BAL-A and BMP-A, solving a structured binary sparse approximation problem.
result Demonstrated that flipping one preference label induces a parameter-independent shift in the DPO gradient, enabling structured binary sparse approximation.
This paper develops a learning framework for optimal strategies in multi-stage decentralized matching markets.
problem Optimal strategies in multi-stage decentralized matching markets with uncertain preferences.
method Nonparametric statistical approach and variational analysis.
result Participants can be better off with multi-stage matching compared to single-stage matching.
Paper introduces Decentralized Non-stationary Competing Bandits ( exttt{DNCB}) for dynamic matching markets.
problem Understanding dynamic two-sided matching markets with competing agents.
method Proposes a decentralized asynchronous learning algorithm ( exttt{DNCB}) for non-stationary environments.
result Obtains sub-linear (logarithmic) regret of exttt{DNCB} in dynamic settings.
New RLHF approach mitigates bias in aligning LLMs with human preferences.
problem Algorithmic bias in RLHF leading to preference collapse.
method Preference Matching (PM) RLHF, using PM regularizer and conditional variant.
result 29% to 41% improvement in alignment with human preferences.
The design of personalized incentives or recommendations to improve user engagement is gaining prominence as digital platform providers continually emerge. We propose a multi-armed bandit framework for matching incentives to users, whose preferences are unknown a priori and evolving dynamically in time, in a resource c…
New algorithm for decentralized matching markets without prior preference rankings.
problem Decentralized two-sided matching markets without known preference rankings.
method Epoch-based CA-ETC algorithm for decentralized matching markets.
result Achieves player optimal expected regret of O(T_0 (K log T / T_0 Δ^2)^(1/γ) + T_0 (T / T_0)^γ).
A new algorithm for competing agents in a two-sided market setting.
problem Decentralized competition between agents in a two-sided market with unknown valuations.
method UCB-D3 algorithm for UCB with Decentralized Dominant-arm Deletion.
result UCB-D3 is order optimal and achieves a new regret lower bound.
Recommender systems are personalized: we expect the results given to a particular user to reflect that user's preferences. Some researchers have studied the notion of calibration, how well recommendations match users' stated preferences, and bias disparity the extent to which mis-calibration affects different user grou…
Adaptive algorithms minimize regret in matching markets with contextual arm preferences.
problem Minimizing regret in matching markets with context-dependent player utilities.
method Developed adaptive algorithms for stochastic and adversarial contexts, providing upper and lower bounds.
result Achieved sublinear regret bounds for both stochastic and adversarial contexts.
Study shows LLM-advisors match human performance in eliciting preferences but struggle with conflicting needs and trust.
problem How do LLM-advisors perform in complex financial domains where domain expertise is crucial?
method Lab-based user study with 64 participants, focusing on three challenges: preference elicitation, personalized guidance, and relationship building.
result LLM-advisors can match human performance in preference elicitation but struggle with conflicting needs and trust issues.
Tutorials on preference learning with Gaussian Processes.
problem Understanding individual preferences and choices for efficient and personalized applications.
method Presentation of a comprehensive framework for preference learning with Gaussian Processes, incorporating rationality principles.
result Construction of preference learning models that encompass various utility models and scenarios.
Inverse reinforcement learning (IRL) enables an agent to learn complex behavior by observing demonstrations from a (near-)optimal policy. The typical assumption is that the learner's goal is to match the teacher's demonstrated behavior. In this paper, we consider the setting where the learner has its own preferences th…
Paper improves parameter estimation of continuous distributions using preference feedback.
problem Improving parameter estimation of continuous distributions.
method Preference-based M-estimators and deterministic preferences.
result Preference-based estimators achieve an estimation error scaling of O(1/n), significantly faster than sample-only methods.
New framework estimates treatment effects based on preferences.
problem Estimating treatment effects with flexible outcomes.
method Preference-based Conditional Treatment Effect (CPTE) framework.
result CPTE provides interpretable targets and new identifiability conditions.
Fashion preference is a fuzzy concept that depends on customer taste, prevailing norms in fashion product/style, henceforth used interchangeably, and a customer's perception of utility or fashionability, yet fashion e-retail relies on algorithmically generated search and recommendation systems that process structured d…
In applications such as recommendation systems and revenue management, it is important to predict preferences on items that have not been seen by a user or predict outcomes of comparisons among those that have never been compared. A popular discrete choice model of multinomial logit model captures the structure of the …
Social-based recommendation systems exploit the selections of friends to combat the data sparsity on user preferences, and improve the recommendation accuracy of the collaborative filtering strategy. The main challenge is to capture and weigh friends' preferences, as in practice they do necessarily match. In this paper…
New study shows personalized content recommendations can lead to polarization of user preferences.
problem Personalized content recommendations can alter user preferences, leading to polarization.
method Used a model of preference dynamics to explore how personalized content affects user preferences.
result Standard reward maximization algorithms achieve only constant regret in personalized recommendation environments.
Model user preferences for conversational LLMs using weak rewards.
problem Lack of persistent user models in conversational LLMs leading to repeated user restatements.
method Vector-Adapted Retrieval Scoring (VARS) framework that updates user vectors online from weak scalar rewards.
result Full VARS agent achieves strongest overall performance, matches strong Reflection baseline in task success, and reduces user effort.
ACOL learns constraints from human preferences in driving simulations.
problem Learning constraints from human preferences in driving simulations.
method Adaptive Constraint Learning (ACOL) algorithm for constrained linear best-arm identification.
result ACOL's sample complexity matches worst-case lower bound and is significantly tighter in the average case.
The paper enhances preference learning by incorporating response time data.
problem Lack of temporal information in user decision-making for reward model learning.
method Integrates response time alongside binary choice data using the EZ model and Neyman-orthogonal loss functions.
result Response time-augmented approach reduces error rates from exponential to polynomial scaling, improving sample efficiency.
Active learning framework for optimizing human preferences in reinforcement learning.
problem Selecting most informative feedback for training models of human preferences.
method Proposes an active learning framework to collect preferential feedback online or offline.
result Errors in DPO logit estimates diminish with more feedback.
Paper shows equivalence between two dividend preference models.
problem Understanding investor and firm preferences for dividends.
method Formulated Epstein-Zin preference, proved equivalence with Maenhout's model.
result Robust dividend policy is equivalent to a threshold strategy based on surplus process.
JIMA uses multi-level preference data to recommend composite items.
problem Recommending composite items efficiently with multi-level preference information.
method Joint Interaction Modeling (JIMA) approach that integrates multi-level preference data and interactions.
result JIMA outperforms advanced baselines in offline and online settings.
Study asset pricing with reference-dependent preferences, finding matching equity premia.
problem Understanding asset pricing under reference-dependent preferences.
method Discrete-time consumption-based capital asset pricing model with reference-dependent preferences.
result Models can generate equity premia matching empirical estimates, showing procyclical price-dividend ratio and countercyclical equity premium.
Matching Markets meet Cumulative Prospect Theory: Towards Optimal and Adversarially Robust Learning
problem Multi-agent multi-armed bandit problem in competitive setup with two-sided matching markets under human-centric decision making model
method Using cumulative prospect theory (CPT) to emulate human preferences
result Improved regret guarantees in adversarial markets with CPT as risk-sensitive measure
Unified approach to RLHF tackles uncertainty in reward function.
problem Uncertainty in reward function learned from human feedback.
method Value-incentivized preference optimization (VPO) that regularizes the reward function with value function.
result Theoretical and practical guarantees for both online and offline RLHF settings.
New algorithms reduce matching regret by limiting frequent updates.
problem Minimizing regret in stochastic matching with rare optimization updates.
method Batched algorithms that limit matching updates to Θ(log log T) rounds.
result Achieve a regret bound of \(\widetilde{\mathcal{O}}(\sqrt{T})\) with reduced computational cost.
We propose a novel and efficient algorithm for the collaborative preference completion problem, which involves jointly estimating individualized rankings for a set of entities over a shared set of items, based on a limited number of observed affinity values. Our approach exploits the observation that while preferences …
A reciprocal recommendation problem is one where the goal of learning is not just to predict a user's preference towards a passive item (e.g., a book), but to recommend the targeted user on one side another user from the other side such that a mutual interest between the two exists. The problem thus is sharply differen…
The paper justifies time-dependent loss reweighting schemes for flow matching and diffusion models.
problem Theoretical justification for time-dependent loss reweighting schemes in flow matching and diffusion models.
method Clarifies that the loss can depend on both time and state, and shows theoretical justification for time-dependent loss weighting schemes.
result Time-dependent loss weighting schemes are theoretically justified for Generator Matching and Edit Flows.
Adjoint Matching improves flow and diffusion models with reward fine-tuning.
problem Improving generative models with reward fine-tuning.
method Casting reward fine-tuning as stochastic optimal control (SOC) and enforcing a specific noise schedule.
result Adjoint Matching outperforms existing SOC algorithms.
Paper proposes efficient RLHF methods for LLMs using active queries.
problem Efficiently aligning LLMs with human preferences using RLHF.
method Formalizes RLHF as a dueling bandit problem, introduces APPO and ADPO algorithms.
result ADPO achieves similar performance to state-of-the-art methods with fewer queries.