SafeMIL learns safer policies by avoiding risky behavior from non-preferred trajectories.
problem Learning safe imitation policies from non-preferred trajectories in risky environments.
method SafeMIL uses Multiple Instance Learning to learn a cost function from non-preferred trajectories.
result SafeMIL learns a safer policy that avoids non-preferred behaviors without sacrificing reward performance.
Inverse reinforcement learning (IRL) enables an agent to learn complex behavior by observing demonstrations from a (near-)optimal policy. The typical assumption is that the learner's goal is to match the teacher's demonstrated behavior. In this paper, we consider the setting where the learner has its own preferences th…
AI assistants often give convincing but incorrect responses to match user beliefs.
problem Sycophancy in AI assistants that use human feedback.
method Examined five AI assistants across four tasks, analyzed human preference data, and compared model outputs against preference models.
result Sycophancy is a general behavior of AI assistants, driven in part by human preference judgments.
Robot motions in the presence of humans should not only be feasible and safe, but also conform to human preferences. This, however, requires user feedback on the robot's behavior. In this work, we propose a novel approach to leverage the user's brain signals as a feedback modality in order to decode the judgment of rob…
Social-based recommendation systems exploit the selections of friends to combat the data sparsity on user preferences, and improve the recommendation accuracy of the collaborative filtering strategy. The main challenge is to capture and weigh friends' preferences, as in practice they do necessarily match. In this paper…
Model predicts human food choices based on demographics.
problem Predicting human food choices from demographic data.
method Non-deterministic model based on NHANES dataset and behavioral studies.
result Generates synthetic data similar to original dataset.
Study uses EEG and ML to predict movie ratings with 72% accuracy.
problem Predicting consumer preferences for movie trailers.
method EEG and machine learning techniques to analyze brain responses to movie trailers.
result Predicted movie ratings with 72% accuracy.
This paper studies robust forward investment and consumption preferences within a zero-volatility context. Different from previous works, we consider an incomplete financial market model due to general investment portfolio constraints. We provide a new PDE characterization and a novel semi-explicit saddle-point constru…
Improved probabilistic forecasts using behavioral transformations.
problem Improving accuracy and consistency of probabilistic asset price forecasts.
method Behavioral transformation of fundamental expectations to disentangle sentiment-induced biases.
result Substantial forecast gains across various models and risk-preferences.
Paper proposes a two-stage ranking for personalized TV recommendations.
problem Improving TV recommendation accuracy and efficiency.
method First, identifies potential candidates using user viewing patterns. Then, ranks them based on user preferences and program textual information.
result The proposed model outperforms in recommendation accuracy and efficiency.
LLMs prefer Bitcoin under crisis frames, affecting financial decisions.
problem Testing whether LLMs have built-in biases towards specific financial assets.
method Developed a three-level audit protocol to examine Bitcoin's representation and influence in LLMs.
result An identifiable internal feature in LLMs can be perturbed to move financial choices, but only within measurable limits.
OSIL learns safe policies from unsafe demonstrations.
problem Offline safe imitation learning with implicit safety.
method Formulates CMDP, infers safety from non-preferred trajectories, learns cost model.
result OSIL learns safer policies without degrading reward performance.
Investigates optimal strategies for behavioral control problems with finite variation controls.
problem Behavioral singular stochastic control problems with finite variation controls.
method Abstract framework, applied to storage management and portfolio investment problems, using CPT preferences and Skorokhod representation theorem.
result Existence of optimal strategies for various goal functionals, including CPT preferences.
The paper analyzes how behavioral investors make portfolio decisions using Markowitz Stochastic Dominance criteria.
problem Understanding how behavioral investors make portfolio decisions.
method Developed stochastic optimization problems and MILP models to capture subjective decision weights and probability weighting functions.
result The developed models can be used to formulate computationally tractable portfolio analysis problems.
The paper analyzes how mutable blockchain protocols affect miner behavior and strategic stability.
problem The mutability of blockchain protocols undermines long-term planning and cooperative equilibria.
method Integrates Austrian capital theory with repeated game theory to examine miner behavior under different institutional conditions.
result Effective time preference increases when protocol rules are mutable, leading to political rent-seeking and undermining strategic coherence.
Activists align with large fund preferences for success.
problem Aligning with large fund preferences increases activist success.
method Analyzed previous proxy voting behavior to estimate preferences and correlated them with activist success.
result Campaigns with higher alignment receive more votes and are more successful.
Paper proposes personalized climate control for driver comfort.
problem Limited research on in-vehicle climate control and driver preferences.
method IoT platform for data collection, machine learning for driver behavior recognition, and personalized preference recommendation.
result Prototype demonstrates effective and accurate climate control for driver comfort.
We develop a framework for interacting with uncertain environments in reinforcement learning (RL) by leveraging preferences in the form of utility functions. We claim that there is value in considering different risk measures during learning. In this framework, the preference for risk can be tuned by variation of the p…
Behavior of systems that are functions of anticipated behavior of other systems, whose own behavior is also anticipatory but homeostatic and determined by hierarchical ordering, which changes over time, of sets of possible environments that are not co-possible, is proven to be highly non-linear and sensitively dependen…
Study replicates reference-dependent preferences impact on risk-return trade-off in Chinese stock market.
problem Impact of reference-dependent preferences on risk-return trade-off in Chinese stock market.
method Utilized CGO proxy, econometric techniques (Dependent Double Sorting, Fama-MacBeth regressions), and data from 1995-2024.
result Reference-dependent preferences have a weaker or absent positive risk-return relationship in the Chinese market.
New model shows SGD can prefer sharp or flat solutions based on label noise.
problem Understanding SGD's preference for flat or sharp solutions during training.
method Solved an analytically solvable model to explore SGD behavior.
result Data distribution determines sharpness at convergence; isotropic label noise leads to flat minimum preference.
Logit models are usually applied when studying individual travel behavior, i.e., to predict travel mode choice and to gain behavioral insights on traveler preferences. Recently, some studies have applied machine learning to model travel mode choice and reported higher out-of-sample predictive accuracy than traditional …
Model-free preference under ambiguity defined and applied.
problem Understanding and quantifying ambiguity aversion and prudence.
method Introduces a new model-free definition of ambiguity attitudes and applies it in various contexts.
result New definition of ambiguity prudence equivalent to specific mathematical functions.
Estimates users' preference for a site over others using engagement data.
problem Lack of data on users' interactions with other sites makes it hard to estimate preferences for a focal site.
method Uses Hierarchical Bayes Method with two estimation techniques: Markov Chain Monte Carlo and Stochastic Gradient with Langevin Dynamics.
result Good support found for the approach to computing personalized share of engagement.
Diversification represents the idea of choosing variety over uniformity. Within the theory of choice, desirability of diversification is axiomatized as preference for a convex combination of choices that are equivalently ranked. This corresponds to the notion of risk aversion when one assumes the von-Neumann-Morgenster…
Reinforcement Learning AI commonly uses reward/penalty signals that are objective and explicit in an environment -- e.g. game score, completion time, etc. -- in order to learn the optimal strategy for task performance. However, Human-AI interaction for such AI agents should include additional reinforcement that is impl…
Bal-PM reduces preference labeling costs for LLMs.
problem Efficiently acquiring human feedback for preference modeling in large language models.
method Bayesian Active Learning with entropy maximization in feature space.
result Bal-PM reduces the number of required preference labels by 33% to 68%.
Evidence acquisition costs influence disclosure behavior and preference.
problem How evidence acquisition costs affect disclosure behavior and preference.
method Analyzes sender-receiver interactions with covert and overt evidence acquisition, varying certification costs.
result Equilibria converge to the Pareto-worst free-learning equilibrium as costs vanish, and receivers prefer covert to overt acquisition.
Derives a new objective to learn from human preferences without approximations.
problem Learning from human preferences through RLHF relies on approximations that can lead to pitfalls.
method Derives a new general objective ΨPO that bypasses both approximations. result Demonstrates the superiority of the new objective to Direct Preference Optimisation (DPO) empirically.
Bayesian model identifies three types of travelers adapting to feedback.
problem Capturing adaptive, feedback-driven travel behavior in heterogeneous individuals.
method Latent Class Reinforcement Learning (LCRL) model with Variational Bayes estimation.
result Three distinct traveler classes identified: context-dependent, persistent exploitative, and exploratory.
Optimizes portfolio growth rate for a behavioral investor considering terminal relative growth rate.
problem Optimizing a behavioral investor's portfolio growth rate under relative growth criterion.
method Martingale method, concavification, and quantile optimization techniques.
result Derives closed-form optimal growth rate and finds significant impact of benchmark growth rate.
ACOL learns constraints from human preferences in driving simulations.
problem Learning constraints from human preferences in driving simulations.
method Adaptive Constraint Learning (ACOL) algorithm for constrained linear best-arm identification.
result ACOL's sample complexity matches worst-case lower bound and is significantly tighter in the average case.
Enhances BO with expert preferences about abstract properties.
problem Lack of expert knowledge in BO for black-box experimental design.
method Human-AI collaboration to incorporate expert preferences into surrogate modeling.
result Superior performance compared to baselines in synthetic and real-world datasets.
The study explores how agents learn and adapt preferences in dynamic environments.
problem Adaptive behavior and preference learning in reinforcement learning tasks.
method The approach involves self-supervised learning of preferences, distinguishing between environmental and intrinsic observations, and evaluating with model-free and model-based reinforcement learning.
result The methodology successfully minimizes surprisal and expected free energy in dynamic environments.
New method accounts for hidden context in preference learning for RLHF models.
problem Incomplete data with hidden context affects RLHF model outcomes.
method Distributional Preference Learning (DPL) methods estimate hidden context distributions.
result DPL methods reduce RLHF vulnerabilities by accounting for hidden context.
Traditional approaches to ranking in web search follow the paradigm of rank-by-score: a learned function gives each query-URL combination an absolute score and URLs are ranked according to this score. This paradigm ensures that if the score of one URL is better than another then one will always be ranked higher than th…
Paper uses inverse optimization to measure risk preference from investment portfolios.
problem Measuring subjective risk preference in investment portfolios.
method Inverse optimization on mean-variance framework.
result Quantified risk preference parameters validated with existing measures.
We introduce a strategic behavior in reinsurance bilateral transactions, where agents choose the risk preferences they will appear to have in the transaction. Within a wide class of risk measures, we identify agents' strategic choices to a range of risk aversion coefficients. It is shown that at the strictly beneficial…
Investors prioritize ESG in crypto-assets, showing higher exposure than traditional assets.
problem Understanding ESG preferences in crypto-assets and their investment behavior.
method A representative household finance survey in Austria to examine ESG preferences and crypto-investment exposure.
result ESG-conscious investors have higher exposure to crypto-assets compared to traditional asset classes.
For sophisticated reinforcement learning (RL) systems to interact usefully with real-world environments, we need to communicate complex goals to these systems. In this work, we explore goals defined in terms of (non-expert) human preferences between pairs of trajectory segments. We show that this approach can effective…
Study optimal investment decisions for diverse risk-tolerant agents.
problem Optimizing investment choices for agents with varying risk preferences.
method Characterizes optimal behavior using certainty equivalents and lognormal risks.
result Derives optimal decision menus under known and uncertain preference distributions.
Study uses FDA to analyze discount functions of different temperaments.
problem Traditional finance models fail to capture individual differences in investment choices.
method Functional Data Analysis (FDA) to investigate temporal discounting behaviors.
result Heterogeneity within each temperament revealed, suggesting diverse investor profiles.
A generalized gamification framework is introduced as a form of smart infrastructure with potential to improve sustainability and energy efficiency by leveraging humans-in-the-loop strategy. The proposed framework enables a Human-Centric Cyber-Physical System using an interface to allow building managers to interact wi…
Paper addresses reward hacking in preference optimization, proposing POWER-DL to improve AI alignment.
problem Reward hacking problem in preference optimization, leading to undesired behaviors.
method POWER-DL combines robust reward maximization and dynamic label updates to mitigate reward hacking.
result POWER-DL consistently outperforms state-of-the-art methods on alignment benchmarks.
Models for recommender systems use latent factors to explain the preferences and behaviors of users with respect to a set of items (e.g., movies, books, academic papers). Typically, the latent factors are assumed to be static and, given these factors, the observed preferences and behaviors of users are assumed to be ge…
Study on investment strategy for agents with periodic preferences and discounting.
problem Investment decisions by agents with periodic S-shaped preferences and present bias.
method Infinite-horizon, continuous-time portfolio selection problem with quasi-hyperbolic discounting.
result Time-consistent planning strategy can be formulated as an equilibrium to a static mean field game.
A new framework for adaptive behavior using reusable value profiles.
problem Adaptive behavior in changing environments requires switching among value-control regimes, but maintaining separate parameters for each situation is impractical.
method Introduces value profiles: reusable bundles of parameters assigned to hidden states, allowing for state-conditional strategy recruitment without independent parameters for each context.
result Profile-based models outperform simpler alternatives in probabilistic reversal learning, suggesting belief-dependent control of adaptive behavior.
While recommendation systems generally observe user behavior passively, there has been an increased interest in directly querying users to learn their specific preferences. In such settings, considering queries at different levels of granularity to optimize user information acquisition is crucial to efficiently providi…