Policy analysts wish to visualize a range of policies for large simulator-defined Markov Decision Processes (MDPs). One visualization approach is to invoke the simulator to generate on-policy trajectories and then visualize those trajectories. When the simulator is expensive, this is not practical, and some method is r…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “Policy Simulation”
Optimizes biomolecular simulations by ranking adaptive sampling policies.
problem Efficiently sampling biomolecular systems to capture complex dynamical behaviors.
method Metric-driven ranking of adaptive sampling policies to identify the optimal policy for each round.
result Different adaptive sampling policies lead to faster convergence and improved sampling performance.
Iterative Model-Based Reinforcement Learning Using Simulations in the Differentiable Neural Computercs.LG
A lifelong learning architecture improves reinforcement learning policies using simulations and a DNC model.
problem Improving reinforcement learning policies in dynamic environments.
method Iterative training of a Reinforcement Learning agent and a DNC model in conjunction.
result DNC models can continually learn from pixels alone to simulate new tasks.
Managers of US National Forests must decide what policy to apply for dealing with lightning-caused wildfires. Conflicts among stakeholders (e.g., timber companies, home owners, and wildlife biologists) have often led to spirited political debates and even violent eco-terrorism. One way to transform these conflicts into…
Monte Carlo Tree Search (MCTS) algorithms perform simulation-based search to improve policies online. During search, the simulation policy is adapted to explore the most promising lines of play. MCTS has been used by state-of-the-art programs for many problems, however a disadvantage to MCTS is that it estimates the va…
Computer simulation provides an automatic and safe way for training robotic control policies to achieve complex tasks such as locomotion. However, a policy trained in simulation usually does not transfer directly to the real hardware due to the differences between the two environments. Transfer learning using domain ra…
New approach reduces simulator exploitation by improving strategic robustness.
problem Simulator exploitation leading to reality gap between simulation and real-world performance.
method Formulated as a zero-sum minimax game, providing theoretical guarantees and a convergent active data selection algorithm.
result Proves convergence and reduces prediction error in strategically important regions by 1.5-2.2 times.
New approach reduces simulator exploitation by learning robust models.
problem Simulator exploitation leading to reality gap in reinforcement learning.
method Formulated as a zero-sum minimax game between model player and policy player, providing theoretical guarantees and a convergent active data selection algorithm.
result Reduces prediction error in strategically important regions by 1.5-2.2 times and enables near-optimal real-world performance.
Efficient local planning with linear approximations for agents with limited simulator access.
problem Planning with limited simulator access in reinforcement learning.
method Confident Monte Carlo Least Square Policy Iteration (Confident MC-LSPI) and Politex (Confident MC-Politex) algorithms.
result The algorithms can learn the optimal policy with local simulator access, even for linear Q-functions.
User Simulators are one of the major tools that enable offline training of task-oriented dialogue systems. For this task the Agenda-Based User Simulator (ABUS) is often used. The ABUS is based on hand-crafted rules and its output is in semantic form. Issues arise from both properties such as limited diversity and the i…
Develops methods to create consistent surrogate models for agent-based simulators.
problem High computational costs and misjudgment of interventions in agent-based models.
method Causal abstractions to learn interventionally consistent surrogate models.
result Surrogates trained for interventional consistency closely mimic the agent-based model's behavior under interventions.
New RL method reduces sample complexity for large policy spaces.
problem Large-scale RL with unknown optimal policies and state/action spaces.
method Introduces eluder dimension for policy space, proving near-optimal sample complexity.
result Near-optimal sample complexity upper bound that depends linearly on eluder dimension.
New method learns policies without limiting to Gaussian distributions.
problem Limitations of Gaussian parameterization in policy learning.
method Advantage weighted quantile regression for implicit policy modeling.
result Comparable or superior performance on MuJoCo benchmarks.
Bayesian method infers contextual bandit policies robustly.
problem Inference of contextual bandit policies in small sample sizes.
method Empirical likelihood for Bayesian inference.
result Accurate uncertainty measurements and policy comparison.
A green simulation-assisted reinforcement learning method for biomanufacturing.
problem Complexity, high variability, lead time, and limited historical data in biopharmaceutical manufacturing.
method Quantifies model risk, uses posterior distribution, and selectively reuses simulation data.
result Demonstrates promising performance in online learning and decision making.
This work shows how to use simulators to learn efficient exploration in real-world RL.
problem Sample complexity of real-world reinforcement learning.
method Coupling exploratory policies learned in simulators with practical approaches.
result Polynomial sample complexity in real world, exponential improvement over direct sim2real transfer.
Vision and learning have made significant progress that could improve robotics policies for complex tasks and environments. Learning deep neural networks for image understanding, however, requires large amounts of domain-specific visual data. While collecting such data from real robots is possible, such an approach lim…
AI-driven tax policies improve economic equality and productivity.
problem Lack of appropriate economic data and limited opportunity to experiment.
method Two-level deep reinforcement learning approach to learn dynamic tax policies from observational data.
result AI-driven tax policies improve the trade-off between equality and productivity by 16%.
SCPO learns robust policies without modeling disturbance, improving real-world task performance.
problem Poor performance of reinforcement learning in real-world tasks due to disturbance in transition dynamics.
method State-conservative policy optimization (SCPO) that reduces disturbance to state space and approximates it with a gradient-based regularizer.
result SCPO learns robust policies without prior knowledge of disturbance or simulators, improving performance in robot control tasks.
Automates infectious disease policy-making via inference in epidemiological models.
problem Improving policy-making for infectious diseases during pandemics.
method Performing inference in existing epidemiological models using a probabilistic programming language.
result Automated inference leads to better disease progression outcomes and policy prescriptions.
We present a novel solution to the problem of simulation-to-real transfer, which builds on recent advances in robot skill decomposition. Rather than focusing on minimizing the simulation-reality gap, we learn a set of diverse policies that are parameterized in a way that makes them easily reusable. This diversity and p…
Improves reinforcement learning policies for robustness.
problem Lack of robustness in reinforcement learning policies.
method Risk-aware Distributional Reinforcement Learning (SDPG) with CVaR.
result Risk-averse policies achieve robustness against disturbances.
Reinforcement Learning methods are capable of solving complex problems, but resulting policies might perform poorly in environments that are even slightly different. In robotics especially, training and deployment conditions often vary and data collection is expensive, making retraining undesirable. Simulation training…
This work models market regimes using CTMSTOU and simulates trading policies.
problem Defining and understanding market regimes in finance.
method Discrete event time multi-agent market simulation with CTMSTOU model.
result Illustrates the importance of regime-awareness in trading policies.
AMUSE uses reinforcement learning to predict optimal model updates.
problem Concept drift weakens model performance over time.
method Reinforcement learning in a simulated environment.
result AMUSE proactively recommends updates based on performance improvements.
SNAPO: Smooth Neural Adjoint Policy Optimization for Optimal Control via Differentiable Simulationcs.LG
SNAPO optimizes policies for complex sequential decisions using differentiable simulation.
problem Optimizing policies for high-dimensional, sequential decisions under uncertainty.
method Embeds neural policy in a differentiable simulator, computes gradients efficiently.
result Produces sensitivities at a cost proportional to one reverse pass, regardless of sensitivity count.
MULTIPOLAR aggregates diverse source policies for efficient transfer RL.
problem Efficiently transfer knowledge between different environmental dynamics.
method Adaptive action aggregation and residual prediction network.
result Demonstrated effectiveness across diverse simulated environments.
Simulation-to-real transfer is an important strategy for making reinforcement learning practical with real robots. Successful sim-to-real transfer systems have difficulty producing policies which generalize across tasks, despite training for thousands of hours equivalent real robot time. To address this shortcoming, we…
Simulation framework assesses ROI of chronic disease adherence and policy timing.
problem Uncertainty in ROI of adherence-enhancing interventions under heterogeneous patient behavior and socioeconomic variation.
method Simulation-based framework integrating disease progression, time-varying adherence, and policy timing.
result Early and adaptive interventions yield highest ROI, exceeding 20% under certain conditions.
Paper proposes a new framework to improve policy optimization by aligning real and simulated data distributions.
problem Inaccurate model estimation leads to performance degradation in model-based reinforcement learning.
method Introduces unsupervised model adaptation to minimize the IPM between real and simulated data distributions.
result Achieves state-of-the-art performance in sample efficiency on various continuous control tasks.
New algorithm reduces variance in Monte Carlo simulations using deep neural networks and policy gradients.
problem Reducing variance in Monte Carlo simulations for estimating function values.
method Optimal correlation search using deep neural networks and policy gradients.
result Optimal correlation function reduces variance by approximating and calibrating policy.
Proposes BSI for valid statistical inference on bandit algorithms.
problem Valid statistical inference on bandit algorithms' performance.
method Fits a simulator of the bandit environment from observed data and uses it to estimate mean reward under any policy.
result Proves asymptotically valid confidence intervals and maintains nominal coverage.
We present a method for fast training of vision based control policies on real robots. The key idea behind our method is to perform multi-task Reinforcement Learning with auxiliary tasks that differ not only in the reward to be optimized but also in the state-space in which they operate. In particular, we allow auxilia…
Optimizes antenna tilt for better QoS in cellular networks.
problem Hard to learn optimal antenna tilt policies in real networks due to risk and simulation gap.
method Uses off-policy Contextual Multi-Armed-Bandit (CMAB) techniques to learn from existing data.
result Trained policies show consistent improvements over existing logging policies.
Paper introduces a simulator-free approach to reinforcement learning policy distillation.
problem Learning multiplicity of cases corresponding to a given action in reinforcement learning.
method Generative adversarial approach to find multiple exemplars for each output class.
result Improves over state-of-the-art on data-free learning of student networks.
Extends double linear policy with time-varying weights and proves robust positive expectation.
problem Ensuring robustness in policy optimization with time-varying parameters.
method Employed a novel elementary symmetric polynomials characterization approach to prove robust positive expectation (RPE). Derived explicit expressions for expected cumulative gain-loss and variance.
result Proved the robust positive expectation property holds for the extended double linear policy.
Study agnostic RL in large state spaces with weak function approximation.
problem Statistical intractability of agnostic policy learning in various environments.
method Investigates agnostic policy learning with different forms of environment access.
result Agnostic policy learning remains statistically intractable with certain forms of environment access.
How Reliable are Test Numbers for Revealing the COVID-19 Ground Truth and Applying Interventions?cs.SI
Agent-based model compares different COVID-19 testing policies and their effectiveness.
problem Understanding how different testing policies reveal the true number of infected cases.
method Developed an agent-based simulation framework in Python to model various testing policies and interventions.
result Contact Tracing consistently captures more positive cases than Random Symptomatic Testing, and LBT performs similarly.
MVPI framework optimizes risk in reinforcement learning, improving performance in robot simulations.
problem Optimizing risk in reinforcement learning control problems.
method Mean-Variance Policy Iteration (MVPI) framework for risk-averse control in MDPs.
result Risk-averse TD3 outperforms previous methods in robot simulation tasks.
Modelling and computer simulation of an insurance policy: A search for maximum profitcond-mat.stat-mech
We have developed a model for a life insurance policy. In this model the net gain is calculated by computer simulation for a particular type of lifetime distribution function. We observed that the net gain becomes maximum for a particular value of upper age of last premium. This paper is dedicated to Professor Dietrich…
New framework improves restless bandit policies for large numbers of arms.
problem Efficiently compute policies for large numbers of arms in restless bandit problems.
method Follow-the-Virtual-Advice framework, converting single-armed policies to N-armed policies.
result Achieves an O(1/\sqrt{N}) optimality gap in both discrete and continuous settings.
New methods for evaluating and optimizing policies in offline RL with unobserved confounders.
problem Evaluating and optimizing policies in the presence of unobserved confounders.
method Characterized settings and algorithms for consistent value estimates and lower bounds, with sample complexity guarantees.
result Proved local convergence guarantees for offline policy improvement.
A method to learn robust policies for environments with model mismatches.
problem Training agents in high-stakes scenarios with mismatched training and real environments.
method Formalizes the perturbation as a zero-sum game to find Nash Equilibrium, which corresponds to the robust policy.
result Our algorithm can find a near-optimal robust policy with high probability using polynomial samples.
We study the problem of off-policy policy optimization in Markov decision processes, and develop a novel off-policy policy gradient method. Prior off-policy policy gradient approaches have generally ignored the mismatch between the distribution of states visited under the behavior policy used to collect data, and what …
We propose a new objective, the counterfactual objective, unifying existing objectives for off-policy policy gradient algorithms in the continuing reinforcement learning (RL) setting. Compared to the commonly used excursion objective, which can be misleading about the performance of the target policy when deployed, our…
We study the problem of off-policy policy evaluation (OPPE) in RL. In contrast to prior work, we consider how to estimate both the individual policy value and average policy value accurately. We draw inspiration from recent work in causal reasoning, and propose a new finite sample generalization error bound for value e…
PFPN: Continuous Control of Physically Simulated Characters using Particle Filtering Policy Networkcs.LG
PFPN uses particle filtering to improve character control in physics-based simulations.
problem Premature commitment to suboptimal actions in high-dimensional continuous control problems for articulated characters.
method Proposes a particle-based action policy using particle filtering to dynamically explore and discretize the action space.
result Demonstrates better imitation performance and robustness to external perturbations compared to Gaussian policies.
Recently, reinforcement learning (RL) algorithms have demonstrated remarkable success in learning complicated behaviors from minimally processed input. However, most of this success is limited to simulation. While there are promising successes in applying RL algorithms directly on real systems, their performance on mor…