OptiGAN uses GAN and RL to optimize sequence generation for specific goals.
problem Challenging in sequence generation tasks to generate sequences with specific desired goals.
method Integrates GAN and RL to optimize desired goal scores using policy gradients.
result Achieves higher desired scores in text and real-valued sequence generation.
Deep RL optimizes goal-based investing strategies.
problem Optimizing investment strategies for achieving financial goals.
method Novel deep reinforcement learning approach for goal-based investing.
result Superior performance compared to benchmarks.
New approach to goal-based investing using hedging and reinforcement learning.
problem Maximizing probability of reaching investment goals with varying risk aversion.
method Lower partial moments, quantile hedging, efficient hedging, reinforcement learning.
result Optimal investment policies for goal-based investing are equivalent.
Many AI problems, in robotics and other domains, are goal-directed, essentially seeking a trajectory leading to some goal state. In such problems, the way we choose to represent a trajectory underlies algorithms for trajectory prediction and optimization. Interestingly, most all prior work in imitation and reinforcemen…
GOIMDA selects inputs to maximize expected influence on a goal functional, reducing data acquisition needs.
problem Challenges in active data acquisition for learning and optimization tasks in deep neural networks.
method GOIMDA uses inverse curvature and goal gradient to select inputs maximizing expected influence on a specified goal functional.
result GOIMDA achieves target performance with fewer labeled samples or function evaluations compared to baselines.
Investor aims to meet financial goals with deadlines and target amounts, considering stock trading costs.
problem Goal-based portfolio selection with fixed transaction costs.
method Stochastic Perron's method to show value function is unique viscosity solution to quasi-variational inequalities. Existence of optimal strategy established.
result Optimal trading strategy differs significantly from frictionless case, revealing complex regions and strategies.
GoBOED optimizes experiments for specific decision-making objectives, improving downstream outcomes.
problem Reducing parameter uncertainty does not always improve decision-making in critical settings.
method Combines variational posterior surrogate and differentiable convex decision layer for gradient-based design optimization.
result GoBOED identifies designs that better align with specific decision objectives and reveals wider optimal design windows.
The paper investigates Goodhart's law and its impact on goal alignment.
problem The adverse effects of optimizing a measure when it diverges from the true goal.
method Formal analysis of Goodhart's law, focusing on the tail distribution of discrepancies.
result Goodhart's law depends on the tail distribution of discrepancies between the true goal and the optimized measure.
C-Learning estimates reachability over time to solve multi-goal tasks.
problem Multi-goal reaching challenges in reinforcement learning.
method Cumulative accessibility functions and recurrence relations.
result Optimal cumulative accessibility functions are monotonic in horizon.
The paper examines how background risk affects portfolio selection and optimal reinsurance design.
problem Maximizing the probability of reaching a financial goal in the presence of background risk.
method Quantile formulation method to derive optimal solutions explicitly.
result The presence of background risk does not change the solution shape but alters the parameter values.
A key challenge in complex visuomotor control is learning abstract representations that are effective for specifying goals, planning, and generalization. To this end, we introduce universal planning networks (UPN). UPNs embed differentiable planning within a goal-directed policy. This planning computation unrolls a for…
Unified perspective unites Bayesian optimization and active learning for efficient goal-oriented optimization.
problem Efficiently optimize expensive engineering and scientific problems with limited data.
method Unified framework linking Bayesian infill criteria and active learning criteria.
result Unified approach formalizes Bayesian infill criteria and active learning criteria.
Max entropy exploration guides reinforcement learning agents to pursue achievable goals.
problem Achieving distant test-time goals in long-horizon tasks.
method Optimize entropy of historical achieved goals by focusing on sparsely explored areas.
result Order of magnitude better sample efficiency on long-horizon multi-goal tasks.
A new method for experimental design focuses on predicting downstream quantities of interest.
problem Designs that maximize parameter learning may not maximize downstream quantity prediction.
method Likelihood-free goal-oriented optimal experimental design (LF-GO-OED) using ABC density ratio estimation.
result LF-GO-OED maximizes the expected information gain for downstream quantities.
We extend the classical risk minimization model with scalar risk measures to the general case of set-valued risk measures. The problem we obtain is a set-valued optimization model and we propose a goal programming-based approach with satisfaction function to obtain a solution which represents the best compromise betwee…
Study optimal portfolio for households with two goals: random and fixed deadlines.
problem Optimal portfolio choice for households managing random and fixed deadlines.
method Maximizes weighted sum of probabilities of funding both goals in a Black-Scholes market.
result Non-monotonic value function due to interaction between goals under forced funding.
A new method reduces both input and output dimensions for better goal-oriented analysis.
problem Simultaneous reduction of input and output dimensions for more accurate analysis.
method Coupled input-output dimension reduction, optimizing gradient-based bounds.
result Determine most informative sensors and influential parameters efficiently.
The paper optimizes insurance purchases for financial goals.
problem Maximizing probability of achieving financial goals with insurance.
method Analyzes deferred term insurance in deterministic and stochastic frameworks, considering income, consumption, and risky investment.
result Provides optimal insurance and investment strategies for achieving financial goals.
Financial institutions use LSTM models to predict customer goals.
problem Predicting customer goals and actions in financial services.
method Used LSTM models with state-space graph embeddings on historical customer traces.
result Demonstrated the effectiveness of LSTM models in predicting customer goals and actions.
Deep reinforcement learning has recently gained a focus on problems where policy or value functions are independent of goals. Evidence exists that the sampling of goals has a strong effect on the learning performance, but there is a lack of general mechanisms that focus on optimizing the goal sampling process. In this …
In this paper the fractional trading ansatz of money management is reconsidered with special attention to chance and risk parts in the goal function of the related optimization problem. By changing the goal function with due regards to other risk measures like current drawdowns, the optimal fraction solutions reflect t…
In Multi-Goal Reinforcement Learning, an agent learns to achieve multiple goals with a goal-conditioned policy. During learning, the agent first collects the trajectories into a replay buffer, and later these trajectories are selected randomly for replay. However, the achieved goals in the replay buffer are often biase…
Semi-analytical approach for optimal wealth management contributions.
problem Optimizing contributions to achieve a financial goal with uncertain returns.
method Controlled backward Kolmogorov equation and Schrodinger equation solution.
result Semi-analytical solutions for efficient frontiers in control space.
A new EM framework for goal-conditioned RL improves performance on sparse reward tasks.
problem Handling sparse rewards in goal-conditioned reinforcement learning.
method A graphical model framework with an EM algorithm that includes a learning-in-hindsight E-step and a supervised M-step.
result hEM significantly outperforms model-free baselines on goal-conditioned benchmarks with sparse rewards.
Enhanced portfolio optimization for a financial goal with G-Learning and GIRL algorithms.
problem Maximize portfolio value while minimizing periodic contributions by a target date in volatile markets.
method Combines G-Learning and GIRL algorithms for dynamic portfolio optimization.
result Improved Sharpe Ratio from 0.42 to 0.483, demonstrating robust optimization in volatile markets.
A framework for goal-based investing with penalties for fund transfers.
problem Investors' mental accounting and multiple investment goals.
method Continuous-time portfolio selection with mental costs and penalties.
result The value function is the unique solution to a complex system of equations.
We show that many machine learning goals, such as improved fairness metrics, can be expressed as constraints on the model's predictions, which we call rate constraints. We study the problem of training non-convex models subject to these rate constraints (or any non-convex and non-differentiable constraints). In the non…
Generating novel graph structures that optimize given objectives while obeying some given underlying rules is fundamental for chemistry, biology and social science research. This is especially important in the task of molecular graph generation, whose goal is to discover novel molecules with desired properties such as …
Eikonal-Constrained QRL improves goal-reaching in reinforcement learning.
problem Reward design and out-of-distribution generalization in reinforcement learning.
method Eikonal-Constrained Quasimetric Reinforcement Learning (Eik-QRL) using the Eikonal PDE.
result Eik-QRL achieves state-of-the-art performance in offline goal-conditioned navigation and manipulation tasks.
Consider mutli-goal tasks that involve static environments and dynamic goals. Examples of such tasks, such as goal-directed navigation and pick-and-place in robotics, abound. Two types of Reinforcement Learning (RL) algorithms are used for such tasks: model-free or model-based. Each of these approaches has limitations.…
Improves RL planning by proposing sub-goals hierarchically.
problem Sequential planning assumption in RL.
method Divide-and-Conquer Monte Carlo Tree Search (DC-MCTS).
result Improves navigation and control tasks.
This paper examines how different decoding algorithms for LLMs align with various goals.
problem Consistency of decoding algorithms with different goals in LLMs.
method Analysis of greedy, lookahead, random sampling, and temperature-scaled random sampling algorithms.
result Random sampling is consistent with the true probability distribution, but other goals require optimal algorithms for specific probability distributions.
Learning to control an environment without hand-crafted rewards or expert data remains challenging and is at the frontier of reinforcement learning research. We present an unsupervised learning algorithm to train agents to achieve perceptually-specified goals using only a stream of observations and actions. Our agent s…
Physics-informed GCRL tackles sparse feedback learning with hybrid dynamics.
problem Sparse feedback learning with high-dimensional, hybrid, or contact-dependent dynamics.
method Introduces physics-informed inductive biases into goal-conditioned value learning.
result Contact-rich manipulation tasks degrade existing Pi-GCRL methods.
While reinforcement learning (RL) has the potential to enable robots to autonomously acquire a wide range of skills, in practice, RL usually requires manual, per-task engineering of reward functions, especially in real world settings where aspects of the environment needed to compute progress are not directly accessibl…
Generative neural nets learn deep policies conditioned on goals.
problem Learning optimal policies for specific goals in reinforcement learning.
method Goal-conditioned neural nets that generate deep neural policies.
result Single learned policy generator can achieve any desired return.
GO-CBED optimizes experiments for specific causal queries, improving efficiency.
problem Efficiently infer causal relationships with limited resources.
method Goal-oriented Bayesian framework that maximizes expected information gain on user-specified causal quantities.
result GO-CBED outperforms existing methods in various causal tasks, especially with limited budgets.
DEEPLY improves cloud service partitioning across multiple datasets and goals.
problem Finding a generally useful method to partition monolithic enterprise applications into cloud-based microservices.
method DEEPLY extends CO-GCN with a novel loss function and hyper-parameter optimization.
result DEEPLY outperforms prior work across multiple datasets and goals.
A popular approach of achieving fairness in optimization problems is by constraining the solution space to "fair" solutions, which unfortunately typically reduces solution quality. In practice, the ultimate goal is often an aggregate of sub-goals without a unique or best way of combining them or which is otherwise only…
This work tackles long-term visual planning by goal-conditioned hierarchical predictors.
problem Current learning approaches fail on long-horizon tasks due to lack of goal information and coarse-to-fine planning.
method Formulate goal-conditioned predictors (GCPs) and hierarchical models to predict trajectories between observations.
result GCPs enable effective long-term planning with much longer horizons than before.
Develops optimal transport in Lorentzian spaces with synthetic curvature bounds.
problem Synthetic curvature bounds for Lorentzian spaces.
method Optimal transport, convexity analysis of entropy functionals.
result Synthetic notion of timelike Ricci curvature lower bounds.
A new GP interpolation method for better predictive distributions in ranges of interest.
problem Improving predictive distributions in specific ranges of interest.
method Relaxed Gaussian process interpolation, relaxing interpolation constraints outside ranges of interest.
result Better predictive distributions in ranges of interest, especially in non-stationary cases.
Learning optimal policies from sparse feedback is a known challenge in reinforcement learning. Hindsight Experience Replay (HER) is a multi-goal reinforcement learning algorithm that comes to solve such tasks. The algorithm treats every failure as a success for an alternative (virtual) goal that has been achieved in th…
MAGIC method optimally estimates model predictions changes.
problem Estimating how training data affects model predictions in large-scale settings.
method Combines classical methods and recent advances in metadifferentiation.
result MAGIC method nearly optimally estimates model predictions changes.
Inferring a person's goal from their behavior is an important problem in applications of AI (e.g. automated assistants, recommender systems). The workhorse model for this task is the rational actor model - this amounts to assuming that people have stable reward functions, discount the future exponentially, and construc…
We determine the optimal amount to invest in a Black-Scholes financial market for an individual who consumes at a rate equal to a constant proportion of her wealth and who wishes to minimize the expected time that her wealth spends in drawdown during her lifetime. Drawdown occurs when wealth is less than some fixed pro…
We determine the optimal strategies for purchasing term life insurance and for investing in a risky financial market in order to maximize the probability of reaching a bequest goal while consuming from an investment account. We extend Bayraktar and Young (2015) by allowing the individual to purchase term life insurance…
The paper introduces an adjacency constraint to improve goal-conditioned HRL.
problem Training inefficiency in goal-conditioned HRL due to large action space.
method Restricting the high-level action space to a k-step adjacent region of the current state.
result The adjacency constraint preserves optimal hierarchical policies and improves HRL performance.