VPP learns joint policies for multi-agent RL through interactions.
problem Learning effective joint policies for multi-agent reinforcement learning.
method VPP integrates variational inference into policy layers for efficient sampling and differentiability.
result VPP outperforms previous methods on large-scale multi-agent tasks.
JPS improves joint policies for multi-agent collaboration in imperfect information games.
problem Learning good joint policies for multi-agent collaboration with imperfect information.
method Decomposes global changes to localized policy changes, iteratively improving joint policies without re-evaluating the entire game.
result JPS improves solutions provided by unilateral approaches and outperforms algorithms designed for collaborative policy learning.
CODA resolves coordination issues in offline multi-agent reinforcement learning.
problem Coordination failure in offline multi-agent reinforcement learning.
method Diffusion-based multi-agent trajectory generator for data augmentation.
result CODA resolves coordination pathologies in continuous polynomial games and complex benchmarks.
New approach for open ad hoc teamwork using graph-based policy learning.
problem Designing autonomous agents to collaborate with changing teams without prior coordination.
method Graph-based policy learning to adapt to dynamic team compositions.
result Successfully models the effects of other agents, leading to robust adaptation and superior performance.
In many real-world settings, a team of agents must coordinate their behaviour while acting in a decentralised way. At the same time, it is often possible to train the agents in a centralised fashion in a simulated or laboratory setting, where global state information is available and communication constraints are lifte…
The paper prices a new life insurance policy for couples, considering various contingent benefits.
problem Valuation of life insurance policies for dependent coupled lives.
method Employed the QP-rule combining real-world and risk-neutral measures for pricing. result Explicit pricing expressions for the new insurance policy were derived and analyzed.
QMIX combines per-agent values to create decentralised policies.
problem Training decentralised policies from centralised learning.
method QMIX uses a mixing network to estimate joint action-values as a monotonic combination of per-agent values.
result QMIX significantly outperforms existing methods on the StarCraft Multi-Agent Challenge (SMAC).
We introduce reinforcement learning for heterogeneous teams in which rewards for an agent are additively factored into local costs, stimuli unique to each agent, and global rewards, those shared by all agents in the domain. Motivating domains include coordination of varied robotic platforms, which incur different costs…
The paper proposes a new policy for optimal treatment allocation based on quantile treatment effects.
problem Optimal treatment allocation policies that target distributional welfare, especially when individuals are heterogeneous.
method The approach involves allocating treatments based on the conditional quantile of individual treatment effects (QoTE), considering both prudent and negligent policymakers.
result The proposed minimax policies are robust to model uncertainty and can be generalized to various settings.
This paper tackles robust policy learning under concept drifts, improving upon existing methods.
problem Tackles robust policy learning under concept drifts, improving upon existing methods.
method Develops a doubly-robust estimator and a learning algorithm to maximize policy value within a given policy class.
result The proposed algorithm achieves sub-optimality gap of the order κ(Π)n−1/2, demonstrating substantial improvement over existing benchmarks. Algorithm optimizes system design and control for better rewards.
problem Optimizing system design and control for maximum rewards.
method Deep reinforcement learning combining policy gradient and model-based optimization.
result DEPS algorithm outperforms state-of-the-art methods in various environments.
Modelling and exploiting teammates' policies in cooperative multi-agent systems have long been an interest and also a big challenge for the reinforcement learning (RL) community. The interest lies in the fact that if the agent knows the teammates' policies, it can adjust its own policy accordingly to arrive at proper c…
FACMAC combines deep policy gradients with factored critic for multi-agent reinforcement learning.
problem Cooperative multi-agent reinforcement learning in discrete and continuous action spaces.
method FACMAC uses a centralised but factored critic, combining per-agent utilities into a joint action-value function.
result FACMAC outperforms MADDPG and other baselines on multi-agent particle environments and StarCraft II tasks.
Missing data and noisy observations pose significant challenges for reliably predicting events from irregularly sampled multivariate time series (longitudinal) data. Imputation methods, which are typically used for completing the data prior to event prediction, lack a principled mechanism to account for the uncertainty…
Bayesian method infers contextual bandit policies robustly.
problem Inference of contextual bandit policies in small sample sizes.
method Empirical likelihood for Bayesian inference.
result Accurate uncertainty measurements and policy comparison.
MA-COPP predicts multi-agent system outcomes using data from a different policy, with probabilistic guarantees.
problem Predicting outcomes in multi-agent systems using data from a different policy.
method Conformal prediction framework applied to multi-agent systems, avoiding exhaustive search.
result Achieves probabilistic guarantees for multi-agent system predictions.
Estimates personalized policies robust to shifts in target populations.
problem Estimating policies that perform well in diverse target populations.
method Develops methods for estimating robust policies considering shifts in outcomes and characteristics.
result Welfare-maximizing policies are robust to certain shifts in potential outcomes.
Cost-effectiveness analyses (CEAs) are at the center of health economic decision making. While these analyses help policy analysts and economists determine coverage, inform policy, and guide resource allocation, they are statistically challenging for several reasons. Cost and effectiveness are correlated and follow com…
Develops a new reinforcement learning framework for complex control problems.
problem Continuous-time extended mean field control with deterministic policies.
method Model-free sensitivity formula, deterministic policy gradient, local value and advantage-rate representations.
result Demonstrates efficiency, stability, and robustness in solving complex control problems.
APQ jointly optimizes neural architecture, pruning, and quantization for efficient inference.
problem Efficient deep learning inference on resource-constrained hardware.
method Joint optimization of neural architecture, pruning, and quantization policy using a quantization-aware accuracy predictor.
result Joint optimization leads to 2.3% higher ImageNet accuracy with reduced latency and energy consumption.
The paper tackles optimal policy learning with asymmetric counterfactual utilities in healthcare decisions.
problem Learning optimal policies from observed data with asymmetric counterfactual utilities.
method The approach involves identifying and minimizing the maximum expected utility loss using statistical decision theory and solving intermediate classification problems.
result One can learn minimax loss decision rules from observed data.
This paper proposes a definition of system health in the context of multiple agents optimizing a joint reward function. We use this definition as a credit assignment term in a policy gradient algorithm to distinguish the contributions of individual agents to the global reward. The health-informed credit assignment is t…
Proposes KStar Diffuser for kinematics-aware bimanual robotic manipulation.
problem Challenges in applying imitation learning to bimanual robotic tasks.
method Integrates physical robot structure into action prediction using a dynamic spatial-temporal graph and differentiable kinematics.
result Effective generation of kinematics-aware actions in both simulation and real-world environments.
Most deep reinforcement learning algorithms are data inefficient in complex and rich environments, limiting their applicability to many scenarios. One direction for improving data efficiency is multitask learning with shared neural network parameters, where efficiency may be improved through transfer across related tas…
Robotic table tennis learns efficient policies to return balls at 100Hz.
problem Developing efficient robotic table tennis strategies.
method Model-free reinforcement learning using evolutionary search on CNN-based policies.
result Robots can develop multi-modal styles (forehand and backhand) with 80% return rate.
Introduces LTQL for factored policies in cooperative MARL.
problem Learning optimal joint policies in collaborative MARL scenarios.
method Logical Team Q-learning (LTQL) as a stochastic approximation to dynamic programming.
result LTQL provides factored policies for optimal joint behavior in cooperative MARL.
This paper proposes a method to safely adjust exploration in RL to satisfy constraints.
problem Unsafe exploration in reinforcement learning violates constraints on controlled object states.
method Automatic adjustment of exploration inputs and variance-covariance matrix for safety.
result The method guarantees satisfaction of joint chance constraints with specified probability.
We present a novel solution to the problem of simulation-to-real transfer, which builds on recent advances in robot skill decomposition. Rather than focusing on minimizing the simulation-reality gap, we learn a set of diverse policies that are parameterized in a way that makes them easily reusable. This diversity and p…
Private RL algorithm with privacy guarantees for personalized medicine decisions.
problem Privacy-preserving reinforcement learning for personalized medicine decisions.
method Developed a private optimism-based RL algorithm using joint differential privacy (JDP).
result Achieved strong PAC and regret bounds with a privacy guarantee.
A method for accurate pricing of multidimensional derivatives under uncertain volatility.
problem High-dimensional stochastic control problem in uncertain volatility model.
method Backward actor-critic stochastic policy gradient scheme combining DP, PPO, and neural networks.
result Accurate and efficient pricing of multidimensional derivatives compared to benchmarks.
Joint replacement is the most common inpatient surgical treatment in the US. We investigate the clinical pathway optimization for knee replacement, which is a sequential decision process from onset to recovery. Based on episodic claims from previous cases, we view the pathway optimization as an intelligence crowdsourci…
Novel framework learns policies from noisy expert demonstrations.
problem Learning effective policies with noisy expert demonstrations.
method Adaptive learning framework that jointly interacts with the environment and expert demonstrations, assigning weights to filter out noisy demonstrations.
result The proposed approach learns robustly with noisy demonstrations and achieves higher performance in fewer iterations.
This study evaluates prewar Japanese financial market efficiency using time-varying models.
problem Determining when prewar Japanese financial market lost its price formation function.
method Time-varying parameter model, generalized least squares-based time-varying vector autoregressive model.
result The prewar Japanese financial market lost its price formation function in 1932.
Galen algorithm compresses neural networks for specific hardware with reduced latency.
problem Finding optimal compression policies for neural networks on specific hardware.
method Reinforcement learning using pruning and quantization to optimize inference latency.
result Compressed ResNet18 for ARM processor reduced inference latency by 80%.
Policy Gradient (PG) algorithms are among the best candidates for the much-anticipated applications of reinforcement learning to real-world control tasks, such as robotics. However, the trial-and-error nature of these methods poses safety issues whenever the learning process itself must be performed on a physical syste…
In many reinforcement learning tasks, the goal is to learn a policy to manipulate an agent, whose design is fixed, to maximize some notion of cumulative reward. The design of the agent's physical structure is rarely optimized for the task at hand. In this work, we explore the possibility of learning a version of the ag…
Study finds multifractal cross-correlations between agricultural markets and external uncertainties.
problem Investigating relationships between agricultural spot markets and external uncertainties.
method Multifractal detrending moving-average cross-correlation analysis (MF-X-DMA).
result Maize exhibits intrinsic joint multifractality with all uncertainty proxies.
We consider the problem of group testing with sum observations and noiseless answers, in which we aim to locate multiple objects by querying the number of objects in each of a sequence of chosen sets. We study a probabilistic setting with entropy loss, in which we assume a joint Bayesian prior density on the locations …
New method for estimating value of optimal policies in uncertain scenarios.
problem Inference for optimal policies when they are non-unique or nearly deterministic.
method Semiparametric efficiency bound, uniformly weighted estimator, NSAVE method.
result Proposes NSAVE method for robust inference in uncertain optimal policies.
Fairness is essential for human society, contributing to stability and productivity. Similarly, fairness is also the key for many multi-agent systems. Taking fairness into multi-agent learning could help multi-agent systems become both efficient and stable. However, learning efficiency and fairness simultaneously is a …
The paper tackles counterfactual learning for stochastic policies with continuous actions.
problem Learning stochastic policies with continuous actions from logged data.
method Introduces a joint kernel embedding of contexts and actions to model continuous actions, and uses proximal point algorithms and smooth estimators for optimization.
result Demonstrates the benefits of using proximal point algorithms and smooth estimators for counterfactual learning.
In multi-agent reinforcement learning, discovering successful collective behaviors is challenging as it requires exploring a joint action space that grows exponentially with the number of agents. While the tractability of independent agent-wise exploration is appealing, this approach fails on tasks that require elabora…
Systematic discriminatory biases present in our society influence the way data is collected and stored, the way variables are defined, and the way scientific findings are put into practice as policy. Automated decision procedures and learning algorithms applied to such data may serve to perpetuate existing injustice or…
Connectionist temporal classification (CTC) is widely used for maximum likelihood learning in end-to-end speech recognition models. However, there is usually a disparity between the negative maximum likelihood and the performance metric used in speech recognition, e.g., word error rate (WER). This results in a mismatch…
NDPs embed dynamical systems into neural networks for efficient sensorimotor learning.
problem Training policies directly in raw action spaces limits scalability for continuous tasks.
method Embed dynamical systems into neural networks to learn robot behaviors via demonstrations.
result NDPs outperform prior methods in both imitation and reinforcement learning setups.
Improved Q-learning for multi-agent reinforcement learning by weighting joint action values.
problem QMIX restricts Q-values to monotonic mixtures, limiting complex value functions. method Introduced weighted projection to recover optimal policies, improving performance.
result CW QMIX and OW QMIX outperform baseline QMIX on multi-agent tasks.
A new approach for cooperative multi-agent reinforcement learning with limited communication, reducing the number of communication rounds.
problem Limited communication in decentralized MARL systems leads to outdated information and unstable learning.
method Base policy prediction technique to estimate gradients and collect samples for a sequence of base policies.
result The proposed algorithm converges to an ε-Nash equilibrium with significantly fewer communication rounds and samples.
A new method for estimating joint value functions in multi-scene reinforcement learning.
problem High variance in samples for policy gradient computations in multi-scene environments.
method Sparse attention mechanism over multiple value function hypotheses to approximate the true joint value function.
result Significant improvements in reward scores and enhanced navigation efficiency across OpenAI ProcGen environments.