Response time improves alignment with diverse human preferences.
problem Standard aggregation of feedback ignores heterogeneity and anonymity.
method Augmenting feedback with response time data and modeling decisions with DDM.
result Estimator of heterogeneous preferences converges to true average preference.
Study shows how diverse investors' learning and preferences shape financial markets.
problem Understanding how diverse investor behaviors and preferences affect market dynamics.
method Developed a multi-agent reinforcement learning framework with heterogeneous preferences and learning mechanisms.
result Diverse investors develop differentiated strategies through interaction, leading to realistic market dynamics.
Study optimal investment decisions for diverse risk-tolerant agents.
problem Optimizing investment choices for agents with varying risk preferences.
method Characterizes optimal behavior using certainty equivalents and lognormal risks.
result Derives optimal decision menus under known and uncertain preference distributions.
A new method reduces preference distortion in LLM alignment.
problem Vulnerability of traditional LLM alignment methods to human preference heterogeneity.
method Sign Estimator: A simple, provably consistent, and efficient estimator using binary classification loss.
result Substantially reduces preference distortion over a panel of simulated personas.
The paper analyzes financial market equilibrium with heterogeneous risk preferences and convex constraints.
problem Characterizing equilibrium in a market with heterogeneous risk preferences and convex constraints.
method Continuous-time financial market model with heterogeneous agents and convex portfolio constraints.
result Margin constraints increase market price of risk and decrease interest rates, leading to higher equity risk premium and pro-cyclical leverage cycles.
This work frames reward modelling from preferences as a causal problem.
problem Reward modelling from preference data for AI alignment.
method Causal inference approach to identify challenges and assumptions.
result Causally-inspired approaches improve model robustness.
Paper introduces Functional Effects Models to account for individual heterogeneity in panel data.
problem Accounting for preference heterogeneity in panel data with machine learning.
method Functional Effects Models using gradient boosting decision trees and deep neural networks to learn individual-specific preference parameters.
result Functional Effects Models outperform traditional models in learning inter-individual heterogeneity and predictive performance.
Method estimates heterogeneous causal effects using instrumental variables.
problem Estimating heterogeneity in causal effects using instrumental variables.
method Two-part method: 1) Discover effect modifiers using machine learning, 2) Test for significance using closed testing.
result Evidence of heterogeneity found in specific subgroups.
The paper analyzes consumer preferences for restaurants using mobile location data.
problem Understanding and predicting consumer choices for restaurants based on location data.
method Bayesian approach with variational inference and stochastic gradient descent.
result The model outperforms standard models in predicting consumer behavior.
Active learning algorithm for diverse populations.
problem Learning preferences from a heterogeneous user population.
method Adaptive item selection based on user rewards.
result Provably efficient algorithms with sample complexity guarantees.
The paper sorts big data by revealed preferences, improving consumer and policy decisions.
problem Sorting diverse consumer preferences for big data objects like colleges.
method Endogenous weighting of revealed preferences, considering spillover effects.
result Consistent steady-state solution to counterbalance equilibrium.
Study risk sharing among agents with varying risk preferences.
problem Risk sharing among agents with heterogeneous risk measures.
method Derive explicit solutions for inf-convolution and counter-monotonic inf-convolution under varying risk seeking.
result Explicit solutions for inf-convolution and counter-monotonic inf-convolution can be represented by a generalization of distortion risk measures.
Generalizes risk sharing models to a continuum of agents.
problem Risk sharing among a large number of heterogeneous agents.
method Modeling agents as points in a measure space, using risk measures on a probability space, and deriving dual representations.
result Explicit formulas for specific risk measures (entropic and expected shortfall) and applications to Pareto efficiency.
LoCo-RLHF models diverse human feedback with contextual information.
problem Heterogeneous human feedback from diverse contexts and preferences.
method Low-rank contextual preference model, PRS policy.
result LoCo-RLHF achieves tighter sub-optimality gap than existing methods.
Study shows naive recommendation models are flawed due to user feedback.
problem Flaws in naive recommendation models due to user feedback.
method Proposed a model with heterogeneous user preferences and proved the inconsistency of naive estimators.
result Consistent estimators are efficient in the presence of myopic agents.
We propose an extended public goods interaction model to study the evolution of cooperation in heterogeneous population. The investors are arranged on the well known scale-free type network, the Barabási-Albert model. Each investor is supposed to preferentially distribute capital to pools in its portfolio based on the …
This paper models how funds choose between competing ESG rating methodologies based on investor preferences.
problem Competing ESG rating methodologies lead to different portfolio rewards and fund fees.
method Modeling funds with heterogeneous ESG priorities and analyzing portfolio changes and investor demand.
result Funds specialize more, but provider scores, investor participation, and equilibrium fees decrease in the benchmark equilibrium.
Method distills reward and strategies from diverse demonstrators.
problem Reward ambiguity and heterogeneity in human demonstrations.
method Reward network distillation to infer task goal and strategies.
result Better recovery of task and strategy rewards.
Method tackles uncertainty in reward models for LLMs from heterogeneous human feedback.
problem Uncertainty in reward models for LLMs from heterogeneous human feedback.
method Heterogeneous preference framework and alternating gradient descent algorithm.
result Established theoretical guarantees for estimator convergence and asymptotic distribution.
In market modeling, one often treats buyers as a homogeneous group. In this paper we consider buyers with heterogeneous preferences and products available in many variants. Such a framework allows us to successfully model various market phenomena. In particular, we investigate how is the vendor's behavior influenced by…
New framework estimates treatment effects based on preferences.
problem Estimating treatment effects with flexible outcomes.
method Preference-based Conditional Treatment Effect (CPTE) framework.
result CPTE provides interpretable targets and new identifiability conditions.
Paper uses machine learning to model travel mode switching under a new transit system.
problem Modeling individual travel mode preferences and response to new mobility options.
method Interpretable machine learning approach to predict and interpret mode-switching behavior.
result Machine learning captures individual heterogeneity in travel mode choice.
Bayesian model identifies three types of travelers adapting to feedback.
problem Capturing adaptive, feedback-driven travel behavior in heterogeneous individuals.
method Latent Class Reinforcement Learning (LCRL) model with Variational Bayes estimation.
result Three distinct traveler classes identified: context-dependent, persistent exploitative, and exploratory.
Paper proposes TPathMine model for more accurate user attribute prediction.
problem Predicting user attributes from click data in heterogeneous networks.
method HetPathMine model with meta-path weights optimized for user emotional preferences.
result TPathMine model achieves higher accuracy in user attribute prediction.
GBS uses machine learning to design products based on consumer preferences.
problem Designing products to meet consumer preferences.
method GBS is a discrete choice experiment that uses machine learning to adaptively construct paired comparison questions.
result GBS outperforms existing methods in accuracy and sample efficiency.
This paper develops, in a Brownian information setting, an approach for analyzing the preference for information, a question that motivates the stochastic differential utility (SDU) due to Duffie and Epstein [Econometrica 60 (1992) 353-394]. For a class of backward stochastic differential equations (BSDEs) including th…
The dynamics of many socioeconomic systems is determined by the decision making process of agents. The decision process depends on agent's characteristics, such as preferences, risk aversion, behavioral biases, etc.. In addition, in some systems the size of agents can be highly heterogeneous leading to very different i…
Bayesian framework learns latent preference archetypes for many-objective optimization.
problem Expanding space of trade-offs and context-dependent human values.
method Dirichlet-process mixture model for latent preference archetypes, hybrid queries for efficient information.
result Mixture-aware Bayesian optimization outperforms standard methods on synthetic and real-world benchmarks.
New study shows personalized content recommendations can lead to polarization of user preferences.
problem Personalized content recommendations can alter user preferences, leading to polarization.
method Used a model of preference dynamics to explore how personalized content affects user preferences.
result Standard reward maximization algorithms achieve only constant regret in personalized recommendation environments.
The paper proposes a method to learn and leverage contextual preference distributions for better decision-making.
problem Heterogeneous and context-dependent human preferences in decision-making problems.
method A sequential learning-and-optimization pipeline using a bounded-variance score function gradient estimator to train a predictive model mapping contextual features to preference distributions.
result The approach reduces average post-decision surprise by up to 25 times compared to risk-averse baselines in a ridesharing environment.
Upper bounds on utility for managing heterogeneous collectivised funds.
problem Managing pension funds with diverse investor preferences and mortality.
method Axiomatic approach to define optimal management strategies.
result Asymptotically optimal strategies for maximizing investor utility.
We consider the problem of learning the preferences of a heterogeneous population by observing choices from an assortment of products, ads, or other offerings. Our observation model takes a form common in assortment planning applications: each arriving customer is offered an assortment consisting of a subset of all pos…
Study asset pricing with transaction costs, showing unique equilibrium exists.
problem Risk-sharing economies with heterogeneous agents trading under quadratic transaction costs.
method Characterizes equilibrium asset prices and strategies via nonlinear, fully-coupled equations.
result Unique solution exists when agents' preferences are sufficiently similar, and empirical liquidity premia and discounts match transaction costs and volatility.
Optimizes pension mix of PAYGO, EET, and individual savings.
problem Balancing PAYGO, EET, and individual savings in funded pension schemes.
method Solves a Nash equilibrium between pension participants and government, considering age-dependent preferences and optimal asset allocation.
result Identifies critical ages and optimal contribution rates for maximizing overall utility.
Generative model reveals hidden interaction preferences in networks.
problem Separate analysis of community and hierarchy overlooks real-world network complexities.
method Generative model based on node preferences and hierarchical structures exploiting network sparsity.
result Model accurately identifies overall node preferences and discerns subsets with different behaviors.
We study the market selection hypothesis in complete financial markets, populated by heterogeneous agents. We allow for a rich structure of heterogeneity: individuals may differ in their beliefs concerning the economy, information and learning mechanism, risk aversion, impatience and 'catching up with Joneses' preferen…
We develop a finite horizon continuous time market model, where risk averse investors maximize utility from terminal wealth by dynamically investing in a risk-free money market account, a stock written on a default-free dividend process, and a defaultable bond, whose prices are determined via equilibrium. We analyze fi…
FedConPE improves conversational recommender systems efficiency and privacy.
problem Efficiently eliciting user preferences in interactive systems with heterogeneous clients.
method Phase elimination-based federated conversational bandit algorithm with adaptive key term construction.
result Minimizes uncertainty across all dimensions in feature space and offers improved efficiency and privacy.
Develops methods to correct bias in AI feedback for more accurate alignment.
problem Systematic bias in AI feedback compared to human labels.
method Two debiased alignment methods: DDPO and DIPO.
result Methods improve alignment efficiency and performance close to human-labeled data.
Collectivized funds need less initial capital to match individual funds, improving pension adequacy.
problem Determining optimal fund management for diverse investor needs.
method Modeling collectivized investment funds with realistic parameters and demonstrating their superiority over individual funds.
result Collectivized funds require less initial capital to match individual funds, enhancing pension adequacy.
Study on self-consuming generative models with diverse human curation, focusing on convergence and stability.
problem Analyzing self-consuming generative models with heterogeneous human curation.
method Investigates the asymptotic behavior of retraining dynamics using nonlinear Perron--Frobenius theory and Banach contraction mapping.
result Improves convergence results and provides stability and non-stability analyses for the model.
Optimizes investment strategies for retirees with longevity risk.
problem Maximizing retirement savings under longevity risk for a group of investors.
method Analytic and numerical solutions for investment strategies in both discrete and continuous time models.
result Analytic formulae for optimal investment strategies in both discrete and continuous time models.
We study consumption behaviour in systems with heterogeneous interacting agents. Two different models are introduced, respectively with long and short range interactions among agents. At any time step an agent decides whether or not to consume a good, doing so if this provides positive utility. Utility is affected by i…
The paper optimizes reinsurance under uncertain dependence among insurers.
problem Designing Pareto-optimal reinsurance contracts in a market with uncertain dependence.
method Robust optimization approach assuming known marginal distributions and unspecified dependence structure.
result Characterization of optimal indemnity schedules under worst-case scenario and derivation of optimal two-parameter layer contracts for independent risks.
IRL approach for studying consumer demand from observed behavior.
problem Confusing observational noise with consumer heterogeneity.
method Developed a Maximum Entropy IRL model for low-dimensional convex optimization.
result Observational noise can be mistaken for consumer heterogeneity.
Adaptive reward models capture individual preferences from human feedback.
problem Learning a reward model that can be specialised to a user.
method Empirical risk minimisation and PAC bound analysis.
result Adaptive reward models benefit from the heterogeneity of user preferences.
Proposes HTM for aggregating ranked data considering user accuracy.
problem Aggregating ranked data from heterogeneous users with varying accuracy levels.
method Heterogeneous Thurstone Model (HTM) with alternating gradient descent algorithm.
result Algorithm converges linearly and outperforms existing methods.
The paper optimizes pension policies with guarantees and sustainability constraints.
problem Designing optimal pension policies with guarantees and sustainability constraints.
method Dynamic utility model, stochastic domain, overlapping generations, time-consistent decision criterion.
result Optimal investment/pension policy computed for a general framework.