A novel framework uses goal-conditioned reinforcement learning to generate diverse samples.
problem Generating high-quality, diverse samples from generative models.
method Two agents: GC-agent learns to reconstruct the training set, S-agent learns to imitate GC-agent without knowing the goals.
result Empirically, the method generates diverse and high-quality samples in image synthesis.
Framework for multi-agent RL with human feedback in a Snake game.
problem Improving multi-agent reinforcement learning with human feedback.
method Developed a simulated game environment for offline model training and online competitions. Introduced HILL methods and reward manipulation heuristics.
result Agents with HILL methods outperform those without in online competitions.
Unified policy controls diverse agents through modular neural networks.
problem Learning control policies for various agent morphologies.
method Shared Modular Policies (SMP) with decentralized control and message passing.
result A single modular policy controls multiple agent morphologies.
Algorithm learns to switch control among agents in a team.
problem Learning to switch control among reinforcement learning agents.
method 2-layer Markov decision process, upper confidence bounds, shared confidence bounds.
result Sublinear total regret with shared confidence bounds.
Task-agnostic RL tackles exploration in MDPs with multiple tasks.
problem Challenges in reinforcement learning with multiple tasks or conflicting objectives.
method Task-agnostic RL framework, UCBZero algorithm.
result UCBZero finds near-optimal policies for multiple tasks efficiently.
New federated learning protocols improve on knowledge distillation's poor performance.
problem Designing a universal API for federated learning without public data.
method Proposed Federated Kernel ridge regression using knowledge distillation.
result Performance of new protocols closely matches theoretical predictions.
DAC-SSM learns domain-agnostic states for better imitation learning.
problem Domain shifts hinder imitation learning in partially observable tasks.
method DAC-SSM uses adversarial training to remove domain-dependent information from states.
result DAC-SSM achieves comparable performance to experts in sparse reward tasks.
This paper proposes a new method to connect language and physical actions in reinforcement learning.
problem Connecting linguistic representations to the physical world in embodied agents.
method Language-conditioned goal generators to decouple sensorimotor learning from language acquisition.
result Agents can demonstrate a diversity of behaviors for any given instruction.
Catastrophic forgetting is the notorious vulnerability of neural networks to the change of the data distribution while learning. This phenomenon has long been considered a major obstacle for allowing the use of learning agents in realistic continual learning settings. A large body of continual learning research assumes…
In this paper we target the problem of transferring policies across multiple environments with different dynamics parameters and motor noise variations, by introducing a framework that decouples the processes of policy learning and system identification. Efficiently transferring learned policies to an unknown environme…
Proposes a new theoretical framework for PbRL that requires less human feedback.
problem Lack of theoretical work capturing practical PbRL frameworks.
method Introduces a reward-agnostic PbRL framework that acquires exploratory trajectories before human feedback.
result Demonstrates improved sample complexity for learning optimal policies in linear and low-rank MDPs.
Generically learns movement control policies from exploration data.
problem Movement optimization in physically based characters.
method Parameterizes actions as target states, learns low-level control policy.
result Improves movement optimization across multiple tasks and algorithms.
CODA resolves coordination issues in offline multi-agent reinforcement learning.
problem Coordination failure in offline multi-agent reinforcement learning.
method Diffusion-based multi-agent trajectory generator for data augmentation.
result CODA resolves coordination pathologies in continuous polynomial games and complex benchmarks.
While model-based deep reinforcement learning (RL) holds great promise for sample efficiency and generalization, learning an accurate dynamics model is often challenging and requires substantial interaction with the environment. A wide variety of domains have dynamics that share common foundations like the laws of clas…
Inverse reinforcement learning (IRL) enables an agent to learn complex behavior by observing demonstrations from a (near-)optimal policy. The typical assumption is that the learner's goal is to match the teacher's demonstrated behavior. In this paper, we consider the setting where the learner has its own preferences th…
RADIAL-RL improves deep RL agents' robustness against adversarial attacks.
problem Vulnerability of deep reinforcement learning agents to small adversarial perturbations.
method RADIAL-RL, a principled framework for training robust reinforcement learning agents.
result RADIAL-RL-trained agents consistently outperform prior methods in robustness tests.
We consider a scenario where an agent has multiple available strategies to explore an unknown environment. For each new interaction with the environment, the agent must select which exploration strategy to use. We provide a new strategy-agnostic method that treat the situation as a Multi-Armed Bandits problem where the…
PeL separates sensory interface optimization from decision learning.
problem Optimizing sensory interfaces without task-specific information.
method Formal separation of perception and decision learning, using metrics for stability, informativeness, and geometry.
result Updates preserving invariants are orthogonal to decision gradients.
This paper optimizes model-based RL for two-player zero-sum games with near-optimal sample complexity.
problem Optimizing model-based reinforcement learning for two-player zero-sum games with minimal samples.
method Model-based reinforcement learning approach for two-player discounted zero-sum Markov games with a generative model.
result Achieves a sample complexity of ildeO(∣S∣∣A∣∣B∣(1−γ)−3ε−2) for finding the Nash equilibrium and ε-NE policies. We propose a method for tackling catastrophic forgetting in deep reinforcement learning that is \textit{agnostic} to the timescale of changes in the distribution of experiences, does not require knowledge of task boundaries, and can adapt in \textit{continuously} changing environments. In our \textit{policy consolidati…
The composition of elementary behaviors to solve challenging transfer learning problems is one of the key elements in building intelligent machines. To date, there has been plenty of work on learning task-specific policies or skills but almost no focus on composing necessary, task-agnostic skills to find a solution to …
CUDC collects diverse data for offline RL by predicting future states.
problem Challenges in collecting task-agnostic data for offline RL.
method Adaptive temporal distances for curiosity-driven data collection.
result CUDC outperforms existing unsupervised methods in offline RL tasks.
A multi-agent system improves crypto portfolio management by processing diverse data types.
problem Managing cryptocurrency portfolios requires processing various data types under high volatility.
method A multi-agent system with three specialized agents for market dynamics, news sentiment, and signal fusion.
result The best configuration, Hierarchical (Skill), achieved a 133.52% cumulative return and 1.502 Sharpe ratio.
In order to make good decision under uncertainty an agent must learn from observations. To do so, two of the most common frameworks are Contextual Bandits and Markov Decision Processes (MDPs). In this paper, we study whether there exist algorithms for the more general framework (MDP) which automatically provide the bes…
E-valuator converts verifier scores into reliable decision rules.
problem Ensuring the correctness of agent trajectories based on heuristic scores.
method Sequential hypothesis testing framework for online monitoring of agent trajectories.
result E-valuator provides better false alarm rate control and statistical power than other strategies.
Traditional reinforcement learning agents learn from experience, past or present, gained through interaction with their environment. Our approach synthesizes experience, without requiring an agent to interact with their environment, by asking the policy directly "Are there situations X, Y, and Z, such that in these sit…
Efficient exploration is an unsolved problem in Reinforcement Learning which is usually addressed by reactively rewarding the agent for fortuitously encountering novel situations. This paper introduces an efficient active exploration algorithm, Model-Based Active eXploration (MAX), which uses an ensemble of forward mod…
New method handles unknown task boundaries in continual learning.
problem Catastrophic forgetting in neural networks.
method Fixed-point equations for online variational Bayes optimization.
result Approximates online Bayes update for non-stationary data.
Optimizes bidding strategies for LinkedIn ads across multiple platforms.
problem Optimizing automated bidding agents for dynamic online marketplaces.
method Developed a general optimization framework for buyer's interest, agnostic to auction mechanisms.
result Automatically guarantees the optimality of budget allocation across ad units and platforms.
This study uses DRL to hedge American put options, outperforming traditional methods.
problem Hedging American put options with high accuracy and low transaction costs.
method Deep Deterministic Policy Gradient (DDPG) method, trained on stochastic volatility models.
result DRL agents outperform traditional methods in both simulated and real-world scenarios.
While recent progress has spawned very powerful machine learning systems, those agents remain extremely specialized and fail to transfer the knowledge they gain to similar yet unseen tasks. In this paper, we study a simple reinforcement learning problem and focus on learning policies that encode the proper invariances …
Event-based learning reduces communication in distributed networks.
problem Distributed learning with diverse data distributions and communication inefficiencies.
method A distributed learning algorithm using ADMM with event-triggered communication.
result The algorithm converges even with distinct local data distributions and achieves accelerated convergence in convex settings.
Fractal neural networks play SimCity and Conway's Game of Life on varying scales.
problem Generalizing agents' performance to larger gameboards than during training.
method Reinforcement learning in a custom environment, using fractal neural networks.
result Agents can generalize to larger gameboards, solving a minigame unsolvable with local strategies.
RL learns to ignore factors in factor investing portfolios.
problem Combining factor investing and reinforcement learning for optimal portfolio allocation.
method RL agent learns through sequential allocations based on firms' characteristics using Dirichlet distributions.
result RL-based portfolios are very close to equally-weighted allocations, indicating agnostic factor learning.
Adaptive Bayesian learning agent for non-stationary bandits.
problem Non-stationary rewards in reinforcement learning.
method Dynamic memory and statistical hypothesis testing.
result Adapts to changing rewards with minimal regret.
JPS improves joint policies for multi-agent collaboration in imperfect information games.
problem Learning good joint policies for multi-agent collaboration with imperfect information.
method Decomposes global changes to localized policy changes, iteratively improving joint policies without re-evaluating the entire game.
result JPS improves solutions provided by unilateral approaches and outperforms algorithms designed for collaborative policy learning.
Developed an explainable DRL model for financial portfolio management.
problem Inability of DRL agents to provide interpretable financial investment policies.
method Integrating PPO with feature importance techniques (SHAP, LIME) to enhance transparency.
result Ability to interpret DRL agent actions in prediction time.
Our research is focused on understanding and applying biological memory transfers to new AI systems that can fundamentally improve their performance, throughout their fielded lifetime experience. We leverage current understanding of biological memory transfer to arrive at AI algorithms for memory consolidation and repl…
Paper proposes a policy-search algorithm to learn entropy-maximizing exploration policies in reward-free environments.
problem Reward-free learning in high-dimensional, continuous-control domains.
method Maximum Entropy POLicy optimization (MEPOL) algorithm that maximizes a non-parametric state entropy estimate.
result MEPOL learns a maximum-entropy exploration policy that facilitates learning various reward-based tasks.
Deep RL learns effective job shop scheduling rules from raw features.
problem Designing effective priority dispatching rules for job shop scheduling is challenging.
method End-to-end deep reinforcement learning using Graph Neural Networks.
result Agent learns high-quality dispatching rules from raw features and generalizes well to unseen instances.
New method evaluates AI stock prediction systems based on decision-making processes.
problem Lack of evaluation for AI systems' decision-making processes.
method Scores intermediate decision process using large language models and closed-loop reinforcement learning feedback.
result Composite behavioral score correlates with Sharpe ratio and reduces prediction error.
We consider tackling a single-agent RL problem by distributing it to n learners. These learners, called advisors, endeavour to solve the problem from a different focus. Their advice, taking the form of action values, is then communicated to an aggregator, which is in control of the system. We show that the local plan…
Study uses agent-based simulation to analyze impact of OBI strategy on financial markets.
problem Improving execution in markets with supply-demand imbalance.
method Built an execution algorithm that accounts for OBI, tested it in artificial markets.
result OBI strategy can improve execution, especially in volatile markets.
First proper learning algorithm for Gaussian halfspaces with matching sample and computational complexity.
problem Agnostically learning halfspaces under Gaussian distribution.
method First proper learning algorithm with matching sample and computational complexity.
result First proper learning algorithm for agnostically learning halfspaces under Gaussian distribution with matching sample and computational complexity.
When learning behavior, training data is often generated by the learner itself; this can result in unstable training dynamics, and this problem has particularly important applications in safety-sensitive real-world control tasks such as robotics. In this work, we propose a principled and model-agnostic approach to miti…
This paper introduces CENIE to quantify environment novelty for better UED.
problem Challenges in measuring environment novelty for effective UED.
method CENIE framework using state-action space coverage and Gaussian Mixture Models.
result CENIE improves UED performance across multiple benchmarks.
Building an open-domain conversational agent is a challenging problem. Current evaluation methods, mostly post-hoc judgments of static conversation, do not capture conversation quality in a realistic interactive context. In this paper, we investigate interactive human evaluation and provide evidence for its necessity; …
Extends boosting to multiclass online agnostic classification.
problem Online multiclass classification with weak learners.
method Reduces multiclass online agnostic boosting to online convex optimization.
result First boosting algorithm for online agnostic multiclass classification.