Paper proposes BMPO to optimize policies using bidirectional models.
problem Model-based reinforcement learning's reliance on forward model accuracy.
method Develops BMPO using both forward and backward models for policy optimization.
result BMPO outperforms state-of-the-art methods in sample efficiency and asymptotic performance.
Investigates optimal pension policies in PAYG systems with forward utility and ageing population.
problem Optimal investment and pension policies in PAYG systems with sustainability and adequacy constraints.
method Non-zero volatility forward CRRA utilities, closed-form optimal policies, detailed numerical analysis.
result Characterization of optimal policies and detailed impact analysis under various scenarios.
Backward exploration reduces sample complexity in policy evaluation.
problem Empirical policy evaluation in reinforcement learning.
method Backward exploration algorithms from high-cost states.
result Reduced average-case sample complexity to O(logS). This work trains GFlowNets using information geometry, improving inference efficiency.
problem Efficient inference over discrete and mixed objects with GFlowNets.
method Formulates forward-policy training through the Fisher-Rao metric of trajectory families.
result Derives exact decomposition of trajectory Fisher and identifies computational regimes.
Paper introduces a new value function for state transitions and optimal policy learning.
problem Learning optimal policies from state transitions and actions.
method Develops a forward dynamics model to maximize a novel value function Q(s,s′). result Demonstrates benefits in value function transfer, redundant action spaces, and off-policy learning.
FISAR uses neural networks to optimize safe reinforcement learning with forward-invariant constraints.
problem Safe reinforcement learning with constraints in safety-critical environments.
method Imposing linear constraints on policy parameters' updating dynamics, using a DNN-based optimizer to satisfy these constraints.
result The policy decreases constraint violation and maximizes cumulative reward monotonically.
The paper addresses human-like decision-making in multi-agent systems using bounded risk-sensitive Markov Games.
problem Modeling human-like decision-making in multi-agent systems with risk-seeking and loss-aversion behaviors.
method Forward policy design and inverse reward learning with iterative reasoning and cumulative prospect theory.
result The proposed algorithms demonstrate both risk-averse and risk-seeking behaviors in multi-agent systems.
Improved fast rates for decision making with forward-KL regularization in contextual bandits.
problem Improving fast rates for decision making with forward-KL regularization in contextual bandits.
method Streamlined analysis of forward-KL-regularized offline CBs, exploiting the pessimism principle and convex-analytical pipeline.
result First ildeO(ε−1) upper bounds in tabular and general function approximation settings. Algorithm optimizes electricity procurement costs by 1.65%.
problem Minimizing energy cost while covering forecast consumption.
method Deep learning forecasting and deviation indicator.
result Reduction of 1.65% in costs compared to uniform policy.
DG improves policy gradient efficiency by selectively backpropagating only valuable samples.
problem Expensive backward passes in policy gradient methods reduce efficiency.
method Introduces 'delight' as a forward-pass signal of learning value and a Kondo gate to selectively backpropagate.
result Selective backpropagation reduces backward pass costs without sacrificing learning quality.
Paper tests Markov assumption in sequential decision making.
problem Testing the Markov assumption in sequential decision making.
method Forward-Backward Learning procedure to test MA without assuming parametric forms.
result The proposed test plays a crucial role in identifying optimal policies in complex decision processes.
We study the optimal trading policies for a wind energy producer who aims to sell the future production in the open forward, spot, intraday and adjustment markets, and who has access to imperfect dynamically updated forecasts of the future production. We construct a stochastic model for the forecast evolution and deter…
New GFlowNet training framework using policy gradients for combinatorial object generation.
problem Training efficiency and robustness in GFlowNet models.
method Policy-dependent rewards and coupled training strategy for forward and backward policies.
result Advanced RL perspectives for robust gradient estimation improve GFlowNet performance.
SymNet uses neural models to solve RMDPs efficiently.
problem Limited effectiveness of generalized policies in RMDPs.
method SymNet trains shared parameters for an RDDL domain, converts instances to graphs, uses relational neural models for node embeddings, and scores actions based on embeddings.
result SymNet policies outperform random and state-of-the-art deep reactive policies on nine RDDL domains.
Backpropagation-free RL method trains layers using local signals.
problem Vanishing or exploding gradients in backpropagation-based RL.
method Local pairwise distance matching for layer-wise training without backpropagation.
result Backpropagation-free method achieves competitive performance and stability.
Accelerates policy optimization in RL with optimistic and adaptive updates.
problem Improving policy optimization methods in reinforcement learning.
method Integrates foresight into policy improvement step via optimistic and adaptive updates.
result Designs an optimistic policy gradient algorithm, adaptive via meta-gradient learning.
The paper analyzes performance criteria for competing fund managers in Ito-diffusion markets.
problem Analyzing performance of competing fund managers in Ito-diffusion markets.
method Developed forward relative performance criteria and forward Nash equilibrium for passive and competitive cases.
result Extended performance criteria for investment problems in Ito-diffusion markets.
A new algorithm learns diverse policies in reinforcement learning.
problem Learning diverse behaviors in reinforcement learning.
method Proposes Maximum Entropy Diverse Exploration (MEDE) algorithm.
result The set of policies learned by MEDE capture the same modalities as the optimal maximum entropy policy.
Local search improves GFlowNets' ability to generate high-reward samples.
problem GFlowNets struggle with over-exploration in high-reward space.
method Local search focusing on high-reward samples via backtracking and reconstruction.
result Significant performance improvement in biochemical tasks.
A new method learns physical system sensitivity to improve policy learning without needing a full model.
problem Expensive policy learning without models and model bias.
method Learn sensitivity of trajectories to parameter perturbations.
result Feasibility demonstrated on a physical robot.
A novel approach learns goal-conditioned policies for locomotion using batch RL.
problem Training goal-conditioned policies for rotation invariant locomotion.
method Data augmentation and Siamese framework for invariance.
result Our approach outperforms existing RL algorithms on 3D locomotion agents.
Introduces new performance criteria for investment under distorted probabilities.
problem Reconciling time-consistent performance with probability distortions.
method Two definitions of forward rank-dependent criteria, equivalence established; characterization of viable probability distortion processes.
result Characterization of optimal wealth process and new distorted measure.
Zero-shot imitation learns to imitate tasks without expert demonstrations.
problem Learning to imitate tasks without expert supervision.
method Explore world without supervision, distill experience into goal-conditioned policy, use forward consistency loss.
result Zero-shot imitation policy can mimic expert tasks after seeing just goal demonstrations.
Paper analyzes neural network complexity for planning problems.
problem Understanding neural network complexity for planning policies.
method Circuit complexity analysis for relational neural networks.
result Three classes of planning problems identified based on network complexity.
The paper analyzes insurance risks using stochastic models.
problem Interest rate and variance risks in unit-linked insurance policies.
method General stochastic volatility models and stochastic interest rates are used to price unit-linked life insurance contracts.
result A perfect hedging strategy is provided and compared with the Black-Scholes model.
Analyzes valuation of derivative claims with asymmetric funding costs and WWR.
problem Valuing and hedging derivative claims with bilateral cash flows in asymmetric funding and risk environments.
method Characterizes pre-default claim value as solution to a non-linear Cauchy problem, applies stochastic representation under linear funding policy.
result Derivative claim value can be represented as a portfolio of European options and admits an analytical formula involving elementary functions and Gaussian integrals.
TREK uses distillation to help students solve hard problems.
problem Stalled progress on hard prompts when current policy lacks useful reasoning trajectories.
method TREK combines distillation and reinforcement learning to expand student support.
result TREK significantly improves student performance on mathematical reasoning and agentic tasks.
We combine forward investment performance processes and ambiguity averse portfolio selection. We introduce the notion of robust forward criteria which addresses the issues of ambiguity in model specification and in preferences and investment horizon specification. It describes the evolution of time-consistent ambiguity…
New algorithms improve reinforcement learning stability and performance.
problem Stability issues in TD learning algorithms with function approximation and off-policy sampling.
method Developed and adapted emphatic temporal difference (ETD(λ)) algorithms for deep reinforcement learning. result Demonstrated improved performance in Atari games and small problems.
Bayesian sOED uses PG reinforcement learning for efficient experiment design.
problem Optimizing sequential experiments for nonlinear models with limited data.
method Formulated as POMDP, solved via PG methods with neural network parameterization.
result Demonstrated advantages over batch and greedy designs in contaminant source inversion.
Paper improves RL efficiency by learning dynamic embeddings.
problem Improving sample efficiency in reinforcement learning.
method Proposes a forward prediction objective for state and action embeddings that capture dynamics.
result Action embeddings alone improve RL performance; combined state and action embeddings achieve efficient learning.
A new framework improves reinforcement learning algorithms with policy guarantees.
problem Designing efficient and stable reinforcement learning algorithms.
method A general framework (FMA-PG) based on functional mirror ascent that constructs surrogate functions enabling policy improvement guarantees.
result The proposed framework enables policy improvement guarantees that hold regardless of policy parameterization, and recovers important heuristics.
FLAP adapts policies quickly to new tasks using shared linear representations.
problem Adapting policies to new tasks efficiently and effectively.
method FLAP uses a shared linear representation and a separate adapter network for quick adaptation.
result FLAP achieves up to 8X faster adaptation and significantly better performance on out-of-distribution tasks.
A deep learning model predicts future cache accesses with high accuracy.
problem Improving cache management and performance through better prediction of future data accesses.
method Proposed a LSTM-based recurrent neural network model to predict future cache accesses using only a cache trace as input.
result The proposed model achieves high prediction accuracy and outperforms state-of-the-art practical policies.
This paper improves entropy calculation for policy gradient in high-dimensional action spaces.
problem Calculating entropy and its gradient for high-dimensional action spaces is computationally infeasible.
method Developed unbiased estimators for entropy bonus and its gradient.
result Entropy estimators substantially improve performance with minimal additional computational cost.
SNAPO optimizes policies for complex sequential decisions using differentiable simulation.
problem Optimizing policies for high-dimensional, sequential decisions under uncertainty.
method Embeds neural policy in a differentiable simulator, computes gradients efficiently.
result Produces sensitivities at a cost proportional to one reverse pass, regardless of sensitivity count.
Paper uses neural networks to analyze oil price impact on Iranian stock and industry indices.
problem Impact of oil price volatility on Tehran stock and industry indices.
method Feed-forward neural networks analysis of two periods: sanctions and post-sanctions.
result Neural networks predict stock and industry indices well, showing significant oil price volatility impact.
New core inflation measure predicts future headline inflation.
problem Creating a better measure of core inflation for timely policy decisions.
method Assemblage Regression, a nonnegative ridge regression that optimizes subcomponent weights.
result Significant improvements in forecasting medium-term inflation developments.
NM-PPG optimizes adaptive feature acquisition in POMDPs for better predictions.
problem Optimizing adaptive feature acquisition in prediction problems with costly features.
method Non-myopic pathwise policy gradients (NM-PPG) with continuous relaxation and straight-through rollout.
result NM-PPG outperforms state-of-the-art AFA methods on synthetic and real-world datasets.
New algorithm scales model-based policy search for robotics to high-dimensional systems.
problem Inefficient scaling of model-based policy search algorithms in high-dimensional state/action spaces.
method Introduces parameterized black-box priors to scale up model learning and improve robustness to prior inaccuracies.
result Significantly more data-efficient than previous algorithms, learning gaits in 16-30 seconds.
Improved sample complexity for identifying best policies in risk-sensitive reinforcement learning.
problem Identifying approximately optimal policies in risk-sensitive reinforcement learning with exponential horizon dependence.
method Forward-model based algorithm with KL-based exploration bonuses adapted for entropic criterion, leveraging smoothness properties of exponential utility and a new stopping rule.
result Achieved sample complexity matching the lower bound, closing the gap between upper and lower bounds.
Develops a new reinforcement learning framework for complex control problems.
problem Continuous-time extended mean field control with deterministic policies.
method Model-free sensitivity formula, deterministic policy gradient, local value and advantage-rate representations.
result Demonstrates efficiency, stability, and robustness in solving complex control problems.
Proposes EM-C algorithm for solving stochastic control problems.
problem Solving multi-period finite time horizon stochastic control problems.
method Sequentially updates control policies using Monte Carlo simulation in a forward-backward manner.
result Demonstrates effectiveness in monopoly pricing and real business cycle studies.
New algorithm learns policies from expert observations alone, efficiently.
problem Imitation Learning from expert observations in large-scale MDPs.
method Forward Adversarial Imitation Learning (FAIL) algorithm, minimizing IP metric between expert and learner observation distributions.
result First provably efficient algorithm in ILFO setting, learning near-optimal policies with polynomial sample complexity.
Paper uses deep reinforcement learning for cryptocurrency market making.
problem Stochastic inventory control challenges faced by market makers.
method Two policy gradient-based algorithms interact with an environment representing limit order book data and order flow statistics. A forward-feed neural network approximates the policy function, and two reward functions are compared.
result Demonstrates deep reinforcement learning's effectiveness in solving market making challenges.
New methods minimize GFlowNet training divergences for better sampling.
problem Training GFlowNets with KL divergence leads to biased and high-variance estimators.
method Design and implement efficient estimators for four divergence measures.
result Properly minimizing these divergences yields a provably correct and effective training scheme.
Study shows activist board representation improves Japanese companies' performance.
problem Lack of innovation and improvement in Japanese companies.
method Examined two Japanese companies with activist board representation, analyzing performance metrics.
result Companies with activist board representation experienced significant improvements in stock returns and operational metrics.
Study optimal retirement time and consumption with habitual persistence.
problem Understanding retirement consumption patterns with habitual persistence.
method Established concise habitual evolution, used martingale and duality methods.
result Optimal consumption declines sharply at retirement but excess consumption increases.