Protects proprietary policies from imitation learning by training adversarial policy ensembles.
problem Protecting policies from external observers cloning them.
method Introduces a reinforcement learning framework that trains an ensemble of near-optimal policies, making demonstrations useless for external observers.
result Demonstrates the existence of 'non-clonable' ensembles and provides a solution to the optimization problem.
Ensemble learning is a very prevalent method employed in machine learning. The relative success of ensemble methods is attributed to their ability to tackle a wide range of instances and complex problems that require different low-level approaches. However, ensemble methods are relatively less popular in reinforcement …
SUNRISE improves off-policy RL algorithms by integrating ensemble methods.
problem Stability and exploration issues in off-policy RL algorithms.
method SUNRISE combines ensemble-based weighted Bellman backups and upper-confidence bounds for efficient exploration.
result SUNRISE improves the performance of off-policy RL algorithms across various domains.
POETS optimizes LLMs by combining policy ensembles and KL regularization.
problem Balancing exploration and exploitation in decision-making and optimization.
method POETS uses policy ensembles and KL regularization to optimize LLMs efficiently.
result POETS achieves state-of-the-art sample efficiency across various domains.
AEA dynamically aggregates ensemble targets for actor-critic learning.
problem Static ensemble aggregation methods struggle with overestimation bias and variance.
method Adaptive Ensemble Aggregation (AEA) dynamically constructs ensemble-based targets.
result AEA converges to optimal variance reduction and maximal Fisher information.
Optimizes biomolecular simulations by ranking adaptive sampling policies.
problem Efficiently sampling biomolecular systems to capture complex dynamical behaviors.
method Metric-driven ranking of adaptive sampling policies to identify the optimal policy for each round.
result Different adaptive sampling policies lead to faster convergence and improved sampling performance.
The paper improves QD policy ensembles using distribution ratio estimators.
problem Training diverse and high-quality reinforcement learning agents.
method Using Stein variational gradient descent and distribution ratio estimators.
result The method generates diverse and high-quality reinforcement learning agents.
Model-based reinforcement learning approaches carry the promise of being data efficient. However, due to challenges in learning dynamics models that sufficiently match the real-world dynamics, they struggle to achieve the same asymptotic performance as model-free methods. We propose Model-Based Meta-Policy-Optimization…
LEARN-SAM improves RL from sub-optimal demonstrations by localizing expert policies and selectively using demonstrations.
problem Improving RL from sub-optimal or sparse demonstrations.
method Local Ensemble and Reparameterization with Split and Merge of expert policies (LEARN-SAM).
result LEARN-SAM boosts learning speed and accuracy by selectively using demonstrations.
Paper tackles overestimation bias in continuous control, improving performance by 25%.
problem Overestimation bias in off-policy learning.
method Truncated Quantile Critics (TQC) combines distributional representation, truncation, and ensembling of critics.
result TQC outperforms state-of-the-art methods by 25% on the Humanoid environment.
UTE improves reinforcement learning by measuring action uncertainty, enhancing policy learning efficiency.
problem Degrading performance of action repetition in reinforcement learning, especially with sub-optimal actions.
method UTE uses ensemble methods to measure uncertainty during action extension, allowing strategic exploration or certainty.
result UTE outperforms existing action repetition algorithms, significantly enhancing policy learning efficiency.
Deep RL ensemble strategy outperforms individual algorithms in stock trading.
problem Designing profitable stock trading strategies in a complex market.
method Ensemble of three deep reinforcement learning algorithms (PPO, A2C, DDPG) for stock trading.
result Deep ensemble strategy outperforms individual algorithms and traditional min-variance portfolio.
We present new methods to estimate causal effects retrospectively from micro data with the assistance of a machine learning ensemble. This approach overcomes two important limitations in conventional methods like regression modeling or matching: (i) ambiguity about the pertinent retrospective counterfactuals and (ii) p…
Single autoregressive model outperforms ensemble methods in offline reinforcement learning.
problem Offline reinforcement learning with limited data and model errors.
method Infer system dynamics from data and optimize policies on model rollouts, using a single autoregressive model.
result Single autoregressive model achieves better performance than ensembles on the D4RL benchmark.
PC-PG balances exploration and exploitation in reinforcement learning.
problem Local policy gradient methods struggle with exploration.
method PC-PG uses an ensemble of learned policies (policy cover) to balance exploration and exploitation.
result PC-PG provides strong theoretical guarantees and empirical validation.
An ensemble method enhances cryptocurrency trading strategies using deep reinforcement learning.
problem Improving generalization performance in stochastic cryptocurrency trading environments.
method Model selection and mixture distribution policy to ensemble deep reinforcement learning models.
result Improved out-of-sample performance compared to benchmarks.
A novel approach uses an ensemble of Gaussian processes for robust and adaptive reinforcement learning.
problem Adaptive reinforcement learning in large or continuous state spaces.
method Online scalable (OS) approach with a weighted ensemble of Gaussian processes.
result The ensemble approach improves performance in adversarial settings.
PS framework selects best policy from library for CSO problems.
problem Policy selection in CSO with heterogeneous performance across covariate space.
method PS framework constructs library of candidate policies and learns a meta-policy to select the best one.
result PS consistently outperforms best single policy in heterogeneous CSO problems.
We propose a new policy iteration theory as an important extension of soft policy iteration and Soft Actor-Critic (SAC), one of the most efficient model free algorithms for deep reinforcement learning. Supported by the new theory, arbitrary entropy measures that generalize Shannon entropy, such as Tsallis entropy and R…
Paper improves financial trading models using GPU parallelism.
problem Challenges in policy instability and sampling bottlenecks in reinforcement learning for financial tasks.
method Revisits ensemble methods with massively parallel simulations on GPUs.
result Significantly improved computational efficiency and robustness of financial decision-making strategies.
Study evaluates reinforcement learning algorithms for sequential experimental design.
problem Lack of generalization in reinforcement learning for experimental design.
method Investigated several reinforcement learning algorithms for sequential experimental design.
result Certain algorithms, using dropout or ensemble approaches, show attractive generalization properties.
We build a statistical ensemble representation of two economic models describing respectively, in simplified terms, a payment system and a credit market. To this purpose we adopt the Boltzmann-Gibbs distribution where the role of the Hamiltonian is taken by the total money supply (i.e. including money created from debt…
The paper proposes an iterative approach to batch reinforcement learning for safer and more informative data collection.
problem Learning policies that are too rigid and do not adapt to new data.
method Safe diversified model-based policy search in an iterative batch reinforcement learning framework.
result Improved learned policies through continuous data collection and adaptation.
UVU simplifies value uncertainty quantification in RL.
problem Estimating epistemic uncertainty in value functions for reinforcement learning.
method UVU uses squared prediction errors between an online learner and a fixed, randomly initialized target network, incorporating policy-conditional value uncertainty.
result UVU achieves equal performance to large ensembles on challenging offline RL settings, with computational savings.
Reinforcement learning agents that operate in diverse and complex environments can benefit from the structured decomposition of their behavior. Often, this is addressed in the context of hierarchical reinforcement learning, where the aim is to decompose a policy into lower-level primitives or options, and a higher-leve…
Higher conservative training increases reward-hacking in reasoning models.
problem Reward hacking during online adaptation in reasoning models.
method Conservative offline training with varying levels of conservatism (β) was applied to a Qwen3-14B policy, and online adaptation was measured against a reward ensemble.
result Higher conservatism (β) increases reward-hacking damage, measured by the Goodhart gap and AUGC.
New algorithms reduce dynamic regret in online MDPs with changing losses.
problem Online MDPs with adversarial loss changes and known transitions.
method Dynamic regret measure, novel ensemble algorithms for three models.
result Provably optimal dynamic regret bounds for episodic SSP, improved bounds for predictable environments.
Recently deep reinforcement learning (DRL) has achieved outstanding success on solving many difficult and large-scale RL problems. However the high sample cost required for effective learning often makes DRL unaffordable in resource-limited applications. With the aim of improving sample efficiency and learning performa…
RH-UCRL combines pessimism and optimism for robust RL.
problem Ensuring reliable performance in real-world RL tasks with worst-case scenarios.
method RH-UCRL is a model-based RL algorithm that optimizes between an agent and an adversary, distinguishing between epistemic and aleatoric uncertainty.
result RH-UCRL achieves near-optimal sample complexity guarantees and outperforms other robust RL algorithms in adversarial environments.
In statistical dialogue management, the dialogue manager learns a policy that maps a belief state to an action for the system to perform. Efficient exploration is key to successful policy optimisation. Current deep reinforcement learning methods are very promising but rely on epsilon-greedy exploration, thus subjecting…
SPReD uses uncertainty to decide imitation from demonstrations.
problem Learning from sparse rewards with few demonstrations.
method Ensemble methods to model Q-value distributions, probabilistic and advantage-based uncertainty quantification.
result Significant gains in reinforcement learning across multiple tasks.
BREMEN optimizes policies offline with fewer data, achieving efficient deployment.
problem High cost of updating policies in real-world applications.
method Behavior-Regularized Model-ENsemble (BREMEN) algorithm for offline optimization.
result BREMEN achieves impressive deployment efficiency with 5-10 deployments, outperforming standard RL methods.
The exploration mechanism used by a Deep Reinforcement Learning (RL) agent plays a key role in determining its sample efficiency. Thus, improving over random exploration is crucial to solve long-horizon tasks with sparse rewards. We propose to leverage an ensemble of partial solutions as teachers that guide the agent's…
Proposes a method to imitate active learning heuristics for better performance.
problem The performance of active learning heuristics depends on the classifier model and data structure.
method Imitates the selection of the best active learning heuristic using DAGGER.
result Outperforms state-of-the-art imitation learners and heuristics on well-known datasets.
Reinforcement learning (RL) methods have been shown to be capable of learning intelligent behavior in rich domains. However, this has largely been done in simulated domains without adequate focus on the process of building the simulator. In this paper, we consider a setting where we have access to an ensemble of pre-tr…
RL agents fail to generalize to unseen environments, even when dynamics are similar.
problem RL agents fail to generalize to unseen environments despite similar dynamics.
method Analyzed policy learning in POMDPs, formalized training dynamics as instances, and introduced a shared belief representation over an ensemble of specialized policies.
result Maximizing rewards induces instance-specific policies that are suboptimal on the training set.
New algorithm reduces decision switching in dynamic environments.
problem Online learning with memory and non-stationary environments.
method Dynamic policy regret, novel ensemble approach, meta-base decomposition.
result Proves optimal dynamic policy regret for memory length, non-stationarity, and time horizon.
This paper sets a lower bound for sample complexity in inverse reinforcement learning.
problem Finding a reward function that generates a desired optimal policy in MDPs.
method Information-theoretic lower bound using geometric construction and Fano's inequality.
result An O(nlogn) sample complexity lower bound for IRL problems. PriMORL trains private RL policies on offline data.
problem Private reinforcement learning on offline data.
method PriMORL learns DP models of the environment and optimizes a policy on the penalized private model.
result PriMORL enables training of private RL agents on complex tasks.
A new confidence measure improves self-training in biased data.
problem Improving self-training in biased data.
method Proposes a new confidence measure, T-similarity, based on ensemble diversity of linear classifiers.
result Empirically shows the benefit of T-similarity for pseudo-labeling policies on various datasets.
Probabilistic vehicle trajectory prediction is essential for robust safety of autonomous driving. Current methods for long-term trajectory prediction cannot guarantee the physical feasibility of predicted distribution. Moreover, their models cannot adapt to the driving policy of the predicted target human driver. In th…
In modern portfolio theory, the balancing of expected returns on investments against uncertainties in those returns is aided by the use of utility functions. The Kelly criterion offers another approach, rooted in information theory, that always implies logarithmic utility. The two approaches seem incompatible, too loos…
Off-policy reinforcement learning (RL) using a fixed offline dataset of logged interactions is an important consideration in real world applications. This paper studies offline RL using the DQN replay dataset comprising the entire replay experience of a DQN agent on 60 Atari 2600 games. We demonstrate that recent off-p…
Dynamic treatment strategies on networks amplify policy impact through spillovers.
problem Effective dynamic treatment allocation in network settings.
method Q-Ising, a three-stage pipeline integrating Bayesian dynamic Ising model, treatment adoption histories, and offline reinforcement learning.
result Adaptive targeting outperforms static centrality benchmarks in Indian village microfinance networks and synthetic data.
The estimation of advantage is crucial for a number of reinforcement learning algorithms, as it directly influences the choices of future paths. In this work, we propose a family of estimates based on the order statistics over the path ensemble, which allows one to flexibly drive the learning process, towards or agains…
MAP-Elites generates diverse trading strategies for improved execution performance.
problem Optimizing trading execution schedules in volatile market conditions.
method Quality-diversity algorithm (MAP-Elites) generating a portfolio of specialized strategies.
result Diverse strategies achieve 8-10% performance improvements, validating quality-diversity methods.
Data-driven predictive analytics are in use today across a number of industrial applications, but further integration is hindered by the requirement of similarity among model training and test data distributions. This paper addresses the need of learning from possibly nonstationary data streams, or under concept drift,…
Simple model-based reinforcement learning outperforms model-free methods in complex tasks.
problem Lagging performance of model-based reinforcement learning agents in non-trivial environments.
method Combining soft value estimates with stochastic value gradients.
result Simple model-based agents achieve state-of-the-art results in a high-dimensional humanoid control task.