The paper addresses optimal control in modern tontines with bequest preferences, showing a linear investment strategy.
problem Optimal controls and decreasing allocation in modern tontines with bequest preferences.
method Dual approach to solve optimal control problems with power utilities, modeling bequest preferences.
result Investment strategy almost linearly adjusts from 0% to 100% over time.
Paper finds closed-form solutions for tontine with bequest motive.
problem Finding optimal fractional consumption rate and bequest amount under bequest motive.
method Relaxing fixed proportions assumption, introducing bequest proportion as control function.
result Closed-form solutions for fractional consumption rate, wealth, bequest amount, and proportion.
We determine the optimal strategies for purchasing term life insurance and for investing in a risky financial market in order to maximize the probability of reaching a bequest goal while consuming from an investment account. We extend Bayraktar and Young (2015) by allowing the individual to purchase term life insurance…
We introduce a new pension product that offers retirees the opportunity for a lifelong income and a bequest for their estate. Based on a tontine mechanism, the product divides pension savings between a tontine account and a bequest account. The tontine account is given up to a tontine pool upon death while the bequest …
In this paper, optimal consumption and investment decisions are studied for an investor who can invest in a fixed interest rate bank account and a stock whose price is a log normal diffusion. We present the method of the HJB equation in order to explicitly solve problems of this type with modifications such as a fixed …
Investigates timing and asset allocation for life insurance in uncertain financial planning.
problem Optimal timing and asset allocation for life insurance in uncertain financial planning.
method Analytical solutions using duality theory and free-boundary problems.
result Explicit expressions for value functions and optimal strategies in both scenarios.
We determine how an individual can use life insurance to meet a bequest goal. We assume that the individual's consumption is met by an income, such as a pension, life annuity, or Social Security. Then, we consider the wealth that the individual wants to devote towards heirs (separate from any wealth related to the afor…
Introduces RPU to explain randomization preference in dynamic settings.
problem Explains preference for randomization in dynamic investment problems.
method Introduces recursive perturbed utility (RPU) to incorporate randomization preference.
result Proves RPU-optimal portfolio policy is Gaussian and can be expressed in closed form.
We consider the problem of how an individual can use term life insurance to maximize the probability of reaching a given bequest goal, an important problem in financial planning. We assume that the individual buys instantaneous term life insurance with a premium payable continuously. By contrast with Bayraktar et al. (…
New tontine model with transaction costs for retirees.
problem Maximizing consumption and bequest utilities for retirees.
method Formulated as a stochastic and impulse control problem, characterized by viscosity solutions.
result V-shaped transaction region with two stages: smoothing and gambling.
New method uses adversarial training for structural model estimation.
problem Estimating structural models with high accuracy.
method Adversarial training between generator and discriminator.
result Adversarial estimator achieves parametric efficiency under correct specification.
The paper optimizes insurance purchases for financial goals.
problem Maximizing probability of achieving financial goals with insurance.
method Analyzes deferred term insurance in deterministic and stochastic frameworks, considering income, consumption, and risky investment.
result Provides optimal insurance and investment strategies for achieving financial goals.
The paper optimizes DIA purchase policies using lifecycle models and asset allocation.
problem Determining the optimal allocation to Deferred Income Annuities (DIAs).
method Employed a lifecycle model with utility of consumption and bequest, formalized optimization process, analyzed results, and extended model to include asset allocation.
result Optimal DIA allocation varies based on refundability, asset allocation, and perceived longevity.
We determine the optimal strategy for investing in a Black-Scholes market in order to maximize the probability that wealth at death meets a bequest goal b, a type of goal-seeking problem, as pioneered by Dubins and Savage (1965, 1976). The individual consumes at a constant rate c, so the level of wealth required fo…
This paper presents a dynamic model to study the impact on the economic outcomes in different societies during the Malthusian Era of individualism (time spent working alone) and collectivism (complementary time spent working with others). The model is driven by opposing forces: a greater degree of collectivism provides…
This paper offers a financial economic perspective on the optimal time (and age) at which the owner of a Variable Annuity (VA) policy with a Guaranteed Living Withdrawal Benefit (GLWB) rider should initiate guaranteed lifetime income payments. We abstract from utility, bequest and consumption preference issues by treat…
Deep neural network optimizes retirement consumption in defined contribution pensions.
problem Optimizing consumption in a defined contribution pension system.
method Formulated as a stochastic control problem, trained deep neural network policy.
result Deep neural network outperforms deterministic rules and adapts to market changes.
Homeownership boosts wealth and welfare compared to renting, according to new research.
problem The conventional wisdom that renting is better than owning a home.
method Block-bootstrap lifecycle simulation to compare homeownership and renting strategies.
result Homeownership generates more wealth and welfare gains than renting, especially for households with high labor income.
Optimizes molecular generation for chemist preferences.
problem Models lack inherent preferences for chemist-desired structures.
method Fine-tuning with Direct Preference Optimization.
result Approach is simple, efficient, and highly effective.
New method adapts to user preferences dynamically, improving recommendation models.
problem Current recommendation models lack dynamic adaptation to changing user preferences.
method Preference Discerning with LLM-Enhanced Generative Retrieval
result Mender achieves state-of-the-art performance in adapting to evolving user preferences.
Many real-world engineering problems rely on human preferences to guide their design and optimization. We present PrefOpt, an open source package to simplify sequential optimization tasks that incorporate human preference feedback. Our approach extends an existing latent variable model for binary preferences to allow f…
Enhances preference learning by incorporating response times into binary choices.
problem Limited information from binary choices about preference strength.
method Combines choices and response times using the EZ diffusion model.
result Response times improve utility estimation for strong preferences.
Bayesian optimization learns DM preferences for multi-outcome experiments.
problem Optimizing expensive experiments with unknown utility functions and multiple outcomes.
method Alternates preference learning and Bayesian optimization, using pairwise comparisons.
result Preference exploration strategies improve Bayesian optimization performance.
New study shows personalized content recommendations can lead to polarization of user preferences.
problem Personalized content recommendations can alter user preferences, leading to polarization.
method Used a model of preference dynamics to explore how personalized content affects user preferences.
result Standard reward maximization algorithms achieve only constant regret in personalized recommendation environments.
Bayesian optimization agent learns user preferences from pairwise comparisons.
problem Learning user preferences from unknown and infinite choices.
method Sequential Bayesian optimization with pairwise comparisons.
result Optimal agent strategy minimizes remaining system uncertainty.
New RLHF framework handles general preference oracles without reward functions.
problem Handling general preference oracles without assuming a reward function.
method Developed a minimax game between two LLMs for RLHF under a general preference oracle, focusing on KL-regularized preference.
result Proposed algorithms for efficient offline and online RLHF learning.
This paper studies robust forward investment and consumption preferences within a zero-volatility context. Different from previous works, we consider an incomplete financial market model due to general investment portfolio constraints. We provide a new PDE characterization and a novel semi-explicit saddle-point constru…
In preference-based reinforcement learning (RL), an agent interacts with the environment while receiving preferences instead of absolute feedback. While there is increasing research activity in preference-based RL, the design of formal frameworks that admit tractable theoretical analysis remains an open challenge. Buil…
Study on identifying most preferred policy in bandits with vector-valued rewards.
problem Identifying the most preferred policy in bandits with vector-valued rewards.
method Derive a novel lower bound on sample complexity, design the Preference-based Track and Stop (PreTS) algorithm, and derive a new concentration inequality.
result The sample complexity of PreTS is asymptotically tight.
Stable and consistent model alignment for language models without assuming human preference models.
problem Lack of statistical consistency in existing alignment methods.
method Relative density ratio optimization between preferred and mixture of preferred and non-preferred data distributions.
result Our approach achieves statistical consistency and stability, providing tighter convergence guarantees.
Dropping a tiny fraction of preferences can significantly alter the rankings of top LLMs.
problem Robustness of LLM ranking systems to small changes in preference data.
method A computational method based on the Bradley-Terry model to evaluate robustness.
result Top LLM rankings can be highly sensitive to the removal of a small fraction of preferences.
Paper explores limits and possibilities of aligning LLMs with human preferences.
problem Aligning LLMs with diverse human preferences to ensure fairness and informed outcomes.
method Analysis of probabilistic representation of human preferences and preservation of diverse preferences.
result LLMs can't fully align with human preferences using reward-based approaches due to Condorcet cycles, but mixed strategies are statistically possible.
Paper improves parameter estimation of continuous distributions using preference feedback.
problem Improving parameter estimation of continuous distributions.
method Preference-based M-estimators and deterministic preferences.
result Preference-based estimators achieve an estimation error scaling of O(1/n), significantly faster than sample-only methods.
Paper investigates monotonicity issues in AI preference learning.
problem AI models may violate monotonicity when learning preferences.
method Investigates root causes of non-monotonicity in comparison-based preference learning.
result Proves local pairwise monotonicity under mild assumptions.
DOPL learns from preference feedback to solve RMAB problems.
problem Learning optimal decisions in RMAB with limited reward information.
method Direct online preference learning (DOPL) for Pref-RMAB.
result DOPL achieves sublinear regret for RMAB with preference feedback.
Direct Density Ratio Optimization aligns LLMs with human preferences without assuming specific models.
problem Statistical inconsistency in aligning LLMs with human preferences.
method Direct Density Ratio Optimization (DDRO) estimates density ratio directly.
result DDRO is statistically consistent, converging to true human preferences as data grows.
Bayesian optimization with preference learning identifies preferred solutions in multi-objective problems.
problem Optimizing multiple criteria with decision maker preferences in expensive functions.
method Bayesian optimization with interactive preference learning and active acquisition function.
result Identifies the most preferred solution with reduced interaction cost.
Training models to prefer certain responses can unintentionally shift probability to harmful ones.
problem Likelihood displacement in DPO models, leading to unintended unalignment.
method Characterized and mitigated likelihood displacement using CHES score.
result Training models to prefer certain responses can unintentionally shift probability mass to harmful responses.
This work proves win rate is key to understanding preference learning.
problem Understanding preference learning from generative models.
method Analyzing preference learning methods as win rate optimization or non-WRO.
result Proves win rate is the only evaluation respecting preferences and prevalences.
IDT learns human preferences from uncertain decisions, even when humans are suboptimal.
problem Learning human preferences from uncertain and suboptimal decisions.
method Inverse decision theory (IDT) framework, statistical analysis of IDT, characterizing sample complexity.
result Learning preferences is easier when decisions are more uncertain, even if humans are suboptimal.
The paper proposes a method to infer multi-objective rewards from preferences.
problem Modeling preferences based on multiple, often competing objectives.
method Modeling priorities lexicographically and inferring multi-objective rewards from observed preferences.
result Lexicographically-ordered rewards provide a better understanding of preferences and improve policies.
The "standard" Merton formulation of optimal investment and consumption involves optimizing the integrated lifetime utility of consumption, suitably discounted, together with the discounted future bequest. In this formulation the utility of consumption at any given time depends only on the amount consumed at that time.…
The paper explores game-theoretic alignment of LLMs with human preferences, finding limitations and conditions.
problem Aligning LLMs with human preferences using game theory.
method Systematic study of payoff choices in a two-player zero-sum game for desirable alignment properties.
result Impossibility of preference matching in game-theoretic LLM alignment under standard assumptions.
Fashion preference is a fuzzy concept that depends on customer taste, prevailing norms in fashion product/style, henceforth used interchangeably, and a customer's perception of utility or fashionability, yet fashion e-retail relies on algorithmically generated search and recommendation systems that process structured d…
Bal-PM reduces preference labeling costs for LLMs.
problem Efficiently acquiring human feedback for preference modeling in large language models.
method Bayesian Active Learning with entropy maximization in feature space.
result Bal-PM reduces the number of required preference labels by 33% to 68%.
DPA aligns LLMs with multi-objective rewards for diverse user preferences.
problem Fine-grained control over LLMs for diverse user needs.
method Integrates multi-objective reward modeling and directional preference control.
result DPA offers better performance trade-offs and intuitive user control over LLM generation.
Tutorials on preference learning with Gaussian Processes.
problem Understanding individual preferences and choices for efficient and personalized applications.
method Presentation of a comprehensive framework for preference learning with Gaussian Processes, incorporating rationality principles.
result Construction of preference learning models that encompass various utility models and scenarios.
Robot motions in the presence of humans should not only be feasible and safe, but also conform to human preferences. This, however, requires user feedback on the robot's behavior. In this work, we propose a novel approach to leverage the user's brain signals as a feedback modality in order to decode the judgment of rob…