Novel framework for data sharing and coordinated exploration in concurrent RL with non-identical environments.
problem Learning more data-efficient and better policies in concurrent RL with non-identical environments.
method Proposes a novel algorithmic framework that leverages causal inference via ANM-MM to extract model parameters and a new data sharing scheme based on similarity measures.
result Demonstrates superior learning speeds on various tasks and effectiveness of diverse action selection.
In this work we describe a novel deep reinforcement learning architecture that allows multiple actions to be selected at every time-step in an efficient manner. Multi-action policies allow complex behaviours to be learnt that would otherwise be hard to achieve when using single action selection techniques. We use both …
New offline RL method works with limited data and function approximators.
problem Sample efficiency with limited data and weak function approximators.
method Pessimistic algorithm based on version space formed by marginalized importance sampling (MIS), with gap assumption.
result Guarantees sample efficiency for simple algorithm under specific assumptions.
Meta-learning is a tool that allows us to build sample-efficient learning systems. Here we show that, once meta-trained, LSTM Meta-Learners aren't just faster learners than their sample-inefficient deep learning (DL) and reinforcement learning (RL) brethren, but that they actually pursue fundamentally different learnin…
A new method improves ridesharing efficiency using QMIX.
problem Improving ridesharing dispatch efficiency with complex environments.
method QMIX for centralized training with decentralized execution.
result QMIX outperforms IDQN in various scenarios.
In many real-world reinforcement learning (RL) problems, besides optimizing the main objective function, an agent must concurrently avoid violating a number of constraints. In particular, besides optimizing performance it is crucial to guarantee the safety of an agent during training as well as deployment (e.g. a robot…
This paper examines a generalized Kropina metric and its geometric properties.
problem Investigating geometric properties of a generalized Kropina metric.
method Analyzing a Finsler manifold with a concurrent π-vector field and examining the φ-concurrent generalized Kropina change. result The geodesic sprays of the original Finsler metric and the modified metric are never projectively related.
New approach uses unlabeled prior data to accelerate exploration in sparse reward tasks.
problem Sparse reward tasks in reinforcement learning.
method Learn reward model from online experience, label prior data, and use concurrently.
result Rapid exploration in challenging sparse-reward domains.
The paper studies Finsler spaces with semi-concurrent vector fields and their equivalence to Riemannian spaces.
problem Characterizing Finsler spaces with semi-concurrent vector fields.
method Analyzing various Finsler spaces and proving conditions for equivalence to Riemannian spaces.
result Various Finsler spaces (quasi-C-reducible, C3-like, Ch-recurrent, P2-like) are equivalent to Riemannian spaces if they admit a semi-concurrent vector field. A Ricci soliton (Mn,g,v,λ) on a Riemannian manifold (Mn,g) is said to have concurrent potential field if its potential field v is a concurrent vector field. In the first part of this paper we completely classify Ricci solitons with concurrent potential fields. In the second part we derive a necessary and suffic…
A Ricci soliton (M,g,v,λ) on a Riemannian manifold (M,g) is said to have concurrent potential field if its potential field v is a concurrent vector field. Ricci solitons arisen from concurrent vector fields on Riemannian manifolds were studied recently in \cite{CD2}. The most important concurrent vector field is …
In the present paper, we introduce and investigate the notion of a semi concurrent vector field on a Finsler manifold. We show that some special Finsler manifolds admitting such vector fields turn out to be Riemannian. We prove that Tachibana's characterization of Finsler manifolds admitting a concurrent vector field l…
We propose a mechanism for distributed resource management and interference mitigation in wireless networks using multi-agent deep reinforcement learning (RL). We equip each transmitter in the network with a deep RL agent that receives delayed observations from its associated users, while also exchanging observations w…
Modelling and exploiting teammates' policies in cooperative multi-agent systems have long been an interest and also a big challenge for the reinforcement learning (RL) community. The interest lies in the fact that if the agent knows the teammates' policies, it can adjust its own policy accordingly to arrive at proper c…
Speeds up deep neural networks training by 10x using GPU concurrency.
problem Training deep residual neural networks efficiently.
method Layer-wise parallel training with GPU concurrency and Nonlinear Multigrid.
result 10.2x speedup over traditional techniques.
Paper applies RL to optimize inventory management across multiple products and nodes.
problem Optimizing inventory management for a large number of products with shared capacity in a multi-node supply chain.
method Novel multi-agent hierarchical reinforcement learning framework with A2C algorithm and quantised action spaces.
result The approach optimizes for maximizing product sales and minimizing wastage of perishable products.
This research tackles balancing exploration and exploitation in deep RL for partially observable systems.
problem Balancing exploration and exploitation in deep RL for partially observable systems.
method Deployed and tested several techniques including adaptive and deterministic exploration strategies, and a modified quadratic loss function.
result Adaptive methods better approximate the trade-off between exploration and exploitation.
A new reinforcement learning method for robots thinking and moving simultaneously.
problem Concurrent control in robotic systems where actions must be decided while the system is still evolving.
method Continuous-time Bellman equations, discretization aware of system delays, and architectural extension to deep reinforcement learning.
result The method successfully handles tasks requiring simultaneous decision-making and action execution.
BCO* improves BCO by concurrently training inverse dynamics and expert policy.
problem Efficiently learn from unlabeled demonstrations without requiring many initial interactions.
method Introduce BCO* that concurrently trains an inverse dynamics model and expert policy.
result BCO* eliminates the need for initial interactions and improves sample complexity.
The present paper deals with an \emph{intrinsic} investigation of the notion of a concurrent π-vector field on the pullback bundle of a Finsler manifold (M,L). The effect of the existence of a concurrent π-vector field on some important special Finsler spaces is studied. An intrinsic investigation of a particular…
We consider the problem of concurrent portfolio losses in two non-overlapping credit portfolios. In order to explore the full statistical dependence structure of such portfolio losses, we estimate their empirical pairwise copulas. Instead of a Gaussian dependence, we typically find a strong asymmetry in the copulas. Co…
Paper proposes a method to learn and exceed expert demonstrations in unknown reward environments.
problem Learning to outperform expert demonstrations in unknown reward environments.
method A novel concurrent reward and action policy learning approach with a stereo utility definition.
result The proposed method can outperform expert demonstrations in various environments.
In this paper, we completely classify almost Yamabe solitons on hypersurfaces in Euclidean spaces arisen from the position vector field. Some results of almost Yamabe solitons with a concurrent vector field and almost Yamabe solitons on submanifolds in Riemannian manifolds equipped with a concurrent vector field are al…
Recent progress in artificial intelligence through reinforcement learning (RL) has shown great success on increasingly complex single-agent environments and two-player turn-based games. However, the real-world contains multiple agents, each learning and acting independently to cooperate and compete with other agents, a…
We generalize Matsumoto metrics with a special π-form and explore their geometric properties.
problem Exploring the geometric properties of generalized Matsumoto metrics with a special π-form.
method Considering a Finsler manifold with a concurrent π-vector field, we introduce a change in the metric and analyze its geometric properties.
result The generalized φ-Matsumoto metric can never be projectively related to the original metric.
Non-negative matrix factorization (NMF) is a fundamental non-convex optimization problem with numerous applications in Machine Learning (music analysis, document clustering, speech-source separation etc). Despite having received extensive study, it is poorly understood whether or not there exist natural algorithms that…
Deep learning model classifies concurrent human interactions from WiFi data with high accuracy.
problem Classifying concurrent human interactions from WiFi data with high accuracy.
method Attention-BiGRU deep learning model using Multiple Input Multiple Output radio link.
result Maximum benchmark accuracy of 94% for a single subject-pair, 88% for ten subject pairs.
Improved gap-dependent bounds for reinforcement learning with linear approximations.
problem Achieving nearly minimax-optimal performance with linear function approximation.
method Developed and analyzed the LSVI-UCB++ algorithm and its concurrent variant.
result First gap-dependent regret bound for nearly minimax-optimal algorithm LSVI-UCB++.
Training neural network often uses a machine learning framework such as TensorFlow and Caffe2. These frameworks employ a dataflow model where the NN training is modeled as a directed graph composed of a set of nodes. Operations in neural network training are typically implemented by the frameworks as primitives and rep…
New algorithm trains neural nets on simple skills to learn complex tasks faster.
problem Learning complex tasks through simple imitation.
method Train neural networks on simple, easy-to-learn skills to accelerate learning of complex, hard-to-learn tasks.
result Consistently outperforms state-of-the-art baseline in training speed and performance.
A novel method optimizes variable-stiffness structures for better strength and weight.
problem Optimizing variable-stiffness structures for higher strength and lighter weight.
method A novel multi-stage concurrent topology optimization scheme combining DMO, S-BPTO, and CFAO.
result The method ensures better fibre angle convergence and stable optimization.
A decentralized deep RL controller improves hexapod locomotion learning.
problem Deep RL struggles with real-world legged robot control.
method Decentralized deep RL on a hexapod robot.
result Decentralized approach learns better and faster.
We study the equilibrium positions of three points on a convex curve under influence of the Coulomb potential. We identify these positions as orthotripods, three points on the curve having concurrent normals. This relates the equilibrium positions to the caustic (evolute) of the curve. The concurrent normals can only m…
We consider a team of reinforcement learning agents that concurrently operate in a common environment, and we develop an approach to efficient coordinated exploration that is suitable for problems of practical scale. Our approach builds on seed sampling (Dimakopoulou and Van Roy, 2018) and randomized value function lea…
Interference among concurrent transmissions in a wireless network is a key factor limiting the system performance. One way to alleviate this problem is to manage the radio resources in order to maximize either the average or the worst-case performance. However, joint consideration of both metrics is often neglected as …
The study confirms conjectures about normals to convex polytopes in 3D space.
problem Concurrent normals problem for convex polytopes in 3D.
method Analyzes the PL concurrent normals problem for convex polytopes, proving conjectures for specific cases.
result Polytopes in 3D have points with 10 normals from interior points, confirmed for all tetrahedra and triangular prisms.
Hybrid RL algorithms improve offline and online RL in linear MDPs.
problem Improving RL performance without single-policy concentrability.
method Developed computationally efficient algorithms for PAC and regret-minimizing RL in linear MDPs.
result Achieved sharper error or regret bounds for linear MDPs.
The paper discusses the impossibility of eliminating surplus intersections in Lagrangian submanifolds.
problem Can surplus intersections in Lagrangian submanifolds be eliminated by Hamiltonian isotopy?
method Analyzing the intersections and isotopies of Lagrangian submanifolds and auxiliary Lagrangians.
result Surplusection cannot be eliminated in several important situations, highlighting the need for better understanding.
This work explores representation complexity in RL paradigms, revealing model-based RL as the easiest task.
problem Investigating the representation complexity gap among model-based, policy-based, and value-based RL.
method Demonstrated through analysis of Markov decision processes (MDPs) and introduced new classes of MDPs.
result Representation complexity hierarchy: model-based RL > policy-based RL > value-based RL.
Study on vector fields on Lie groups reveals surprising algebraic coincidences.
problem Characterizing vector fields on Lie groups with Riemannian metrics.
method Algebraic and geometric analysis of left-invariant vector fields on nilpotent Lie groups.
result Spaces of Killing, one-harmonic, and conformal vector fields coincide with the center of the Lie algebra on nilpotent Lie groups.
The aim of this paper is to train an RBF neural network and select centers under concurrent faults. It is well known that fault tolerance is a very attractive property for neural networks. And center selection is an important procedure during the training process of an RBF neural network. In this paper, we devise two n…
New method defends RL agents from poisoning attacks without MDP knowledge.
problem Poisoning attacks on RL systems can cause learning failures.
method Generic poisoning framework for online RL, Vulnerability-Aware Adversarial Critic Poison (VA2C-P).
result Successfully prevents RL agents from learning good policies or converging to target policies.
In dialogues, an utterance is a chain of consecutive sentences produced by one speaker which ranges from a short sentence to a thousand-word post. When studying dialogues at the utterance level, it is not uncommon that an utterance would serve multiple functions. For instance, "Thank you. It works great." expresses bot…
Reincarnating RL reuses prior work to accelerate RL progress.
problem Efficiency and accessibility in reinforcement learning for large-scale applications.
method Transfer of learned policies between RL agents or design iterations, focusing on value-based RL.
result Demonstrated gains in performance over tabula rasa RL on various tasks.
Paper shows RLHF can be solved similarly to standard RL.
problem Difficulty of RLHF compared to standard RL.
method Reduction to reward-based RL techniques.
result RLHF can be solved using existing algorithms for reward-based RL.
RL tackles decision making in unknown environments, focusing on efficiency and efficacy.
problem Efficiency and efficacy in RL algorithms for sample-starved situations.
method Markov Decision Processes, model-based and value-based approaches, policy optimization.
result Enhanced understanding and improvements in sample and computational efficacies of RL algorithms.
Catalyst.RL accelerates RL research with efficient training.
problem Efficient reinforcement learning training in complex environments.
method Open-source PyTorch framework with distributed training and RL algorithms.
result Catalyst.RL achieved 2nd place in a computationally expensive RL challenge.
Study shows effectiveness of offline RL in online RL tasks.
problem Improving online RL efficiency using offline RL data.
method Formalized framework for incorporating offline RL as online RL subroutines, introducing techniques to enhance effectiveness.
result Effectiveness of the framework depends on task nature, techniques greatly enhance effectiveness, and existing methods are ineffective.