Iroko enables RL for datacenter CC, outperforming TCP on fat-tree and dumbbell topologies.
problem Stability and over-fitting issues in RL for datacenter networks.
method Developed Iroko emulator to support various network conditions and algorithms.
result Deep RL algorithms outperform TCP on fat-tree and dumbbell topologies.
When will a server fail catastrophically in an industrial datacenter? Is it possible to forecast these failures so preventive actions can be taken to increase the reliability of a datacenter? To answer these questions, we have studied what are probably the largest, publicly available datacenter traces, containing more …
Edge filters reduce video data transmission to datacenters.
problem Strain on wide area network infrastructure due to video camera deployments.
method FilterForward system with lightweight edge filters and microclassifiers.
result Reduces bandwidth use by an order of magnitude.
Modern information technology services largely depend on cloud infrastructures to provide their services. These cloud infrastructures are built on top of datacenter networks (DCNs) constructed with high-speed links, fast switching gear, and redundancy to offer better flexibility and resiliency. In this environment, net…
This work surveys attacks and defenses on edge neural networks.
problem Security challenges of edge neural networks due to their compute and memory intensity, data-independence, and privacy risks.
method Taxonomy of attacks and defenses on edge-deployed neural networks.
result New security considerations and approaches are needed for edge DNNs.
NEST optimizes deep learning training by placing devices efficiently across networks and memory.
problem Inefficient device placement in distributed deep learning leads to high communication and memory overhead.
method NEST uses network-, compute-, and memory-aware dynamic programming to optimize device placement.
result NEST achieves up to 2.43 times higher throughput and better memory efficiency.
New activation networks improve model efficiency and performance.
problem Creating hardware-efficient deep learning models.
method Restructurable Activation Networks (RANs) with RAN-explicit and RAN-implicit methods.
result RANs achieve state-of-the-art results with improved hardware efficiency.
In recent years, deep neural networks (DNN) have demonstrated significant business impact in large scale analysis and classification tasks such as speech recognition, visual object detection, pattern extraction, etc. Training of large DNNs, however, is universally considered as time consuming and computationally intens…
Machine learning (ML), especially deep learning is made possible by the availability of big data, enormous compute power and, often overlooked, development tools or frameworks. As the algorithms become mature and efficient, more and more ML inference is moving out of datacenters/cloud and deployed on edge devices. This…
Hybrid RL algorithms improve offline and online RL in linear MDPs.
problem Improving RL performance without single-policy concentrability.
method Developed computationally efficient algorithms for PAC and regret-minimizing RL in linear MDPs.
result Achieved sharper error or regret bounds for linear MDPs.
This work explores representation complexity in RL paradigms, revealing model-based RL as the easiest task.
problem Investigating the representation complexity gap among model-based, policy-based, and value-based RL.
method Demonstrated through analysis of Markov decision processes (MDPs) and introduced new classes of MDPs.
result Representation complexity hierarchy: model-based RL > policy-based RL > value-based RL.
New method defends RL agents from poisoning attacks without MDP knowledge.
problem Poisoning attacks on RL systems can cause learning failures.
method Generic poisoning framework for online RL, Vulnerability-Aware Adversarial Critic Poison (VA2C-P).
result Successfully prevents RL agents from learning good policies or converging to target policies.
Reincarnating RL reuses prior work to accelerate RL progress.
problem Efficiency and accessibility in reinforcement learning for large-scale applications.
method Transfer of learned policies between RL agents or design iterations, focusing on value-based RL.
result Demonstrated gains in performance over tabula rasa RL on various tasks.
A new decentralized federated learning approach tackles network capacity challenges.
problem Efficiently utilizing network capacities between nodes in federated learning.
method Proposes a segmented gossip approach for decentralized federated learning.
result Demonstrates significant reduction in training time compared to centralized federated learning.
Paper shows RLHF can be solved similarly to standard RL.
problem Difficulty of RLHF compared to standard RL.
method Reduction to reward-based RL techniques.
result RLHF can be solved using existing algorithms for reward-based RL.
The paper aims to define a benchmark for deep learning recommendation models.
problem Insufficient benchmarking for deep learning recommendation models.
method Synthesizes modeling strategies, defines desirable characteristics, and summarizes advice from the MLPerf Recommendation Advisory Board.
result Defines an industry-relevant benchmark for deep learning recommendation models.
MaxEnt RL optimizes decision-making in uncertain environments.
problem MaxEnt RL optimizes decision-making in uncertain environments.
method Formal analysis of MaxEnt RL in POMDPs and adversarial reward games.
result MaxEnt RL optimally solves certain classes of control problems with uncertain reward functions.
RL tackles decision making in unknown environments, focusing on efficiency and efficacy.
problem Efficiency and efficacy in RL algorithms for sample-starved situations.
method Markov Decision Processes, model-based and value-based approaches, policy optimization.
result Enhanced understanding and improvements in sample and computational efficacies of RL algorithms.
Catalyst.RL accelerates RL research with efficient training.
problem Efficient reinforcement learning training in complex environments.
method Open-source PyTorch framework with distributed training and RL algorithms.
result Catalyst.RL achieved 2nd place in a computationally expensive RL challenge.
Study shows effectiveness of offline RL in online RL tasks.
problem Improving online RL efficiency using offline RL data.
method Formalized framework for incorporating offline RL as online RL subroutines, introducing techniques to enhance effectiveness.
result Effectiveness of the framework depends on task nature, techniques greatly enhance effectiveness, and existing methods are ineffective.
MOReL learns offline RL policies using pessimistic MDPs.
problem Offline RL's data efficiency and velocity.
method Two-step process: learn P-MDP and near-optimal policy in it.
result MOReL is minimax optimal and matches state-of-the-art results.
catalyst.RL simplifies RL research, enabling reproducibility and efficiency.
problem Difficulty in reproducing and comparing RL algorithms.
method Open-source framework with distributed training, flexible configurations, and efficient RL algorithms.
result Demonstrated effectiveness on AI for Prosthetics Challenge, achieving 3rd place.
This paper compares expected and distributional reinforcement learning methods.
problem Understanding why distributional reinforcement learning performs better than expected reinforcement learning.
method Analyzes differences in tabular, linear, and non-linear approximation settings.
result Distributional RL can hurt performance if it does not induce identical behavior.
RL research overhypes potential but lacks deployable solutions.
problem Current RL research overhypes potential and lacks deployable solutions.
method Identifies and critiques current RL research practices.
result Current RL research direction is unlikely to lead to practical, economically viable solutions.
New algorithms improve federated learning accuracy and stability with real-world data.
problem Real-world data diversity and imbalance challenge federated learning.
method Developed new algorithms (FedVC, FedIR) to resample and reweight data.
result Significant improvements in accuracy and stability of federated learning.
New method makes deep RL agents more understandable.
problem Incomprehensible decision-making in NN-based RL agents limits their applications.
method Derives a secondary comprehensible agent from a NN-based RL agent.
result Empirical evaluation supports the possibility of building a comprehensible agent.
Decentralized deep learning improves efficiency with arbitrary compression.
problem Limited network bandwidth hinders decentralized deep learning.
method Proposed Choco-SGD with arbitrary compression for non-convex functions.
result Choco-SGD achieves linear speedup and higher compression than previous methods.
RL applied to finance tasks, highlighting challenges and future directions.
problem Decision-making tasks in finance using RL.
method Meta-analysis of RL applications, identifying challenges and proposing future directions.
result Challenges in RL performance and future research directions.
A simple approach to offline RL without additional complexity.
problem Learning from a fixed dataset of actions with value estimation errors.
method Adding a behavior cloning term to the policy update of an online RL algorithm and normalizing the data.
result Matches the performance of state-of-the-art offline RL algorithms with minimal changes.
MVFST-RL tackles real-time network congestion control with RL.
problem Real-time network congestion control with RL in asynchronous environments.
method Asynchronous RL framework for delayed actions in QUIC.
result Scalable framework for congestion control in QUIC with improved state-of-the-art RL.
This paper analyzes CNNs for malware detection in cloud IaaS.
problem Malware vulnerability in cloud IaaS environments.
method Analysis of Convolutional Neural Networks (CNNs) for online malware detection using process-level performance metrics.
result State-of-the-art DenseNets and ResNets effectively detect malware in online cloud systems.
MiniHack simplifies creation of complex RL environments.
problem Limited availability of challenging RL benchmarks.
method Develops a sandbox framework for easy RL environment design.
result MiniHack enables rapid creation of diverse RL testbeds.
Generative flow networks use RL to learn probabilistic models efficiently.
problem Training generative models with RL for compositional discrete objects.
method Reformulate GFlowNet training as entropy-regularized RL with specific reward and regularizer.
result Entropy-regularized RL can be competitive with established GFlowNet training methods.
RL agents outperform baselines in asset allocation.
problem Optimizing asset allocation using reinforcement learning.
method Model-free deep RL agents trained on real-world stock prices.
result RL agents significantly outperformed random and uniform allocation.
This paper shows RL with KL penalties is equivalent to Bayesian inference for fine-tuning LMs.
problem Fine-tuning large language models to avoid undesirable features.
method Analyzed KL-regularized RL and showed it's equivalent to variational inference.
result KL-regularized RL avoids distribution collapse and is more insightful as Bayesian inference.
HTMRL uses HTM for RL, adapting faster to changing environments.
problem Adapting to non-stationary environments in RL.
method Strictly HTM-based RL algorithm.
result HTMRL adapts faster to changing environments in a 10-armed bandit.
New benchmarks for offline RL from diverse datasets.
problem Measuring progress in offline RL due to lack of suitable benchmarks.
method Developed benchmarks tailored for offline RL, focusing on diverse dataset properties.
result Revealed deficiencies in existing offline RL algorithms.
Introduces a new RL formalism focusing on event desirability.
problem Limited consideration of performance distribution in RL.
method Introduces micro-objective reinforcement learning.
result New formalism allows for prior knowledge and event desirability.
M3PO improves model-based meta-RL with theoretical guarantees.
problem Improving sample efficiency in multi-task RL with theoretical guarantees.
method Extending Janner et al. (2019) theorems, proposing M3PO with performance guarantees.
result M3PO outperforms existing methods in continuous-control benchmarks.
evo-RL combines evolutionary computation with reinforcement learning for better adaptability.
problem Improving reinforcement learning algorithms' adaptability and performance in environments with rewardless states.
method Embedding reinforcement learning in an evolutionary cycle, distinguishing instinctive from learnable behavior.
result evo-RL leads to state-of-the-art performance on OpenAI Gym control problems with rewardless states.
This work bridges offline RL and DRL to address distributional shift.
problem Distributional shift in offline RL due to difference in state-action visitation distributions.
method Proposes offline RL algorithms using DRL framework, characterizes sample complexity under single policy concentrability.
result Demonstrates superior performance of proposed algorithms through simulations.
RL Unplugged benchmarks offline RL methods across diverse domains.
problem Evaluate offline reinforcement learning methods without online data collection.
method Proposes a benchmark suite with diverse datasets and detailed evaluation protocols.
result Demonstrates the effectiveness of offline RL methods across various domains.
Optimistic RL algorithms are simplified for deep RL with competitive performance.
problem Achieving accurate optimism in model-based RL for large-scale problems.
method Interpreting scalable optimistic model-based algorithms as solving a tractable noise augmented MDP.
result Competitive regret bound of i l d e O ( ∣ S ∣ H ∣ A ∣ T ) ilde{\mathcal{O}}( |\mathcal{S}|H\sqrt{|\mathcal{A}| T } ) i l d e O ( ∣ S ∣ H ∣ A ∣ T ) for Gaussian noise augmentation. Paper proposes a hybrid RL algorithm that combines offline and online data without needing reward info.
problem How to efficiently use online data to improve RL policies using only offline data.
method A three-stage hybrid RL algorithm that uses reward-agnostic exploration and model-based offline RL.
result The hybrid RL algorithm outperforms both pure offline and pure online RL in sample complexity.
Critic-regularized regression improves offline RL performance.
problem Poor performance of off-policy algorithms in offline RL.
method Critic-regularized regression (CRR) for policy learning from fixed datasets.
result CRR outperforms state-of-the-art offline RL algorithms significantly.
This paper surveys RL for combinatorial optimization, focusing on TSP.
problem Optimizing solutions for combinatorial optimization problems.
method Reinforcement learning applied to combinatorial optimization problems, specifically the TSP.
result Deep learning mechanisms enhance RL algorithms for near-optimal solutions.
MushroomRL simplifies RL experiments for researchers.
problem Complexity in implementing and testing RL experiments.
method Provides a comprehensive and flexible framework to minimize effort.
result Significantly benefits RL researchers in empirical analysis.
Hierarchical RL simplifies exploration in RL tasks.
problem Why does hierarchy work well in RL?
method Evaluated hierarchical RL on various tasks, focusing on exploration benefits.
result Most benefits of hierarchy can be attributed to improved exploration.