AR-A3C improves A3C's robustness to noisy environments.
problem Neural networks are vulnerable to noise from unexpected sources in reinforcement learning.
method Introduced an adversarial agent to make the learning process more robust.
result AR-A3C outperforms A3C in both clean and noisy environments.
Adversarial RL recovers agent rewards from financial market data simulations.
problem Recovering agent rewards in volatile financial markets with unknown dynamics.
method Adversarial inverse reinforcement learning in latent space simulations.
result Adversarial RL can robustly recover agent rewards from latent space representations of real market data.
New snooping attacks exploit deep RL without access to environment.
problem Security vulnerabilities in deep reinforcement learning.
method Proposes snooping threat models and attacks on RL agents.
result Adversaries can launch attacks without interacting with the environment.
New reward function improves GAIL performance in task-based environments.
problem Reward bias in adversarial imitation learning.
method Proposed a new reward function to overcome existing biases.
result New reward function outperforms existing methods in task-based environments.
Develops a strategy to minimize loss in both stochastic and adversarial environments for linear contextual bandits.
problem Linear contextual bandits with adversarial corruption.
method Proposes a novel strategy called Best-of-Both-Worlds (BoBW) RealFTRL, extending RealLinExp3 and FTRL.
result Regret upper bound of $O\left(\min\left\{\frac{(\log(T))^3}{Δ_{*}} + \sqrt{\frac{C(\log(T))^3}{Δ_{*}}},\ \ \sqrt{T}(\log(T))^2
ight\}
ight)$ , showing effectiveness in both stochastic and adversarial environments.
Adversarial policies can defeat RL agents in multi-agent environments.
problem Vulnerability of RL policies to adversarial perturbations.
method Demonstrated adversarial policies in a multi-agent, high-dimensional environment.
result Adversarial policies reliably win against RL agents but generate seemingly random behavior.
Optimal semi-bandit algorithm for both stochastic and adversarial environments.
problem Optimal semi-bandit algorithm for both stochastic and adversarial environments.
method Developed a general semi-bandit algorithm that achieves O ( log T ) \mathcal{O}(\log T) O ( log T ) regret for stochastic and O ( T ) \mathcal{O}(\sqrt{T}) O ( T ) regret for adversarial environments without regime or T T T knowledge. result First algorithm to achieve optimal O ( log T ) \mathcal{O}(\log T) O ( log T ) and O ( T ) \mathcal{O}(\sqrt{T}) O ( T ) regret simultaneously for stochastic and adversarial environments. End-to-end Dyna-AIL combines planning and learning for faster convergence.
problem Efficient imitation learning with minimal interactions.
method Dyna-like framework integrating model-based planning and model-free learning.
result Our approach converges to optimal policies with fewer interactions.
Imaginative RL uses GANs to simulate real environments, making RL more efficient.
problem Lack of data efficiency and safety constraints in reinforcement learning.
method Generative Adversarial Imaginative Reinforcement Learning (GAIL) algorithm.
result The proposed algorithm more efficiently utilizes real-world experience.
Deep learning models are vulnerable to adversarial examples that can fool them.
problem Vulnerability of deep learning models to adversarial examples in safety-critical environments.
method Review and taxonomy of methods for generating adversarial examples, countermeasures, and challenges.
result Deep learning models are vulnerable to adversarial examples that can fool them.
Optimal algorithm for both stochastic and adversarial bandits without prior knowledge.
problem Optimal algorithm for both stochastic and adversarial bandits.
method Online mirror descent with Tsallis entropy regularization and reduced-variance loss estimators.
result Achieves optimal pseudo-regret in both adversarial and stochastic bandits.
Develops model selection for bandits balancing adversarial and stochastic guarantees.
problem Model selection in bandit scenarios with simultaneous adversarial and stochastic high-probability regret.
method Nested policy classes, balanced candidate regret bounds, mis-specification tests.
result Best of both world guarantees in linear bandits with simultaneous adversarial and stochastic environments.
Generative adversarial tree search fails to outperform DQN in Atari environments.
problem Improving deep reinforcement learning algorithms for Atari environments.
method Proposes generative adversarial tree search (GATS) that learns the environment model and implements Monte Carlo tree search (MCTS) on the learned model.
result GATS fails to outperform DQN, despite theoretical analysis showing potential benefits.
New algorithm for multi-player bandits in changing environments.
problem Sequential action selection with collisions and adversarial losses.
method First provable Multi-player Bandit algorithm for changing environments.
result Resolves open problem in adversarial multi-player settings.
Adaptive linear bandit algorithm with best-of-three-worlds regret bounds.
problem Adaptive to adversarial and stochastic environments with varying sub-optimality gaps and corruption.
method Combines SCRiBLe algorithm with scaled-up sampling and optimistic online learning.
result Achieves best-of-three-worlds regret bounds of O ( T log T ) O(\sqrt{T \log T}) O ( T log T ) for adversarial and O ( log T Δ min + C log T Δ min ) O(\frac{\log T}{Δ_{\min}} + \sqrt{\frac{C \log T}{Δ_{\min}}}) O ( Δ m i n l o g T + Δ m i n C l o g T ) for stochastic environments. AMIGo uses adversarial intrinsic goals to teach RL agents new skills.
problem Learning in sparse reward environments.
method Adversarial intrinsic goals to generate a curriculum for a student policy.
result AMIGo enables agents to learn new skills without extrinsic rewards.
AIRL learns robust, generalizable reward functions from demonstrations.
problem Learning robust reward functions from demonstrations for changing environments.
method Adversarial Inverse Reinforcement Learning (AIRL) with hierarchical disentangled rewards.
result Generalizable policies and comparable results to state-of-the-art methods.
RIA method improves OoD generalization for covariate shift.
problem Improving out-of-distribution generalization under covariate shift.
method Adversarial label invariant graph data augmentations for OoD generalization.
result RIA method achieves high accuracy compared to OoD baselines.
FAIR-NN finds invariant variables for causal inference across diverse environments.
problem Nonparametric invariance and causal learning in regression models with varying joint distributions.
method FAIR-NN framework using adversarial optimization and neural networks.
result FAIR-NN identifies invariant variables and quasi-causal variables under minimal conditions.
Adaptive PCA algorithms for changing environments.
problem Static adversarial regret is not suitable for changing environments.
method Online adaptive algorithms for PCA and variance minimization with sub-linear adaptive regret guarantees.
result The proposed algorithms adapt to changing environments.
Paper proposes a user model for RL in recommendation systems.
problem Challenges in applying RL to recommendation systems due to unclear reward and dynamics.
method Develops a generative adversarial network to model user behavior and reward function, then uses this model in a novel RL algorithm.
result The RL policy based on the proposed user model leads to better user reward and higher click rates.
Adaptive framework generates challenging adversarial scenarios for autonomous vehicles.
problem Lack of efficient and adaptable evaluation methods for autonomous vehicles.
method Adaptive evaluation framework using ensemble models and nonparametric Bayesian clustering.
result Adversarial scenarios significantly degrade tested autonomous vehicles' performance.
New framework detects adversarial concept drift in streaming data.
problem Adversarial concept drift in dynamic environments.
method Predict-Detect streaming framework for unsupervised drift detection and recovery.
result Framework detects adversarial drift with <6% labeled data, improving active learning for imbalanced data.
GAN Q-learning uses GANs for distributional RL in tabular and gym environments.
problem Complex MDPs in nonlinear function approximation.
method Generative adversarial networks (GANs) for distributional reinforcement learning.
result Empirically shows GAN Q-learning is a viable alternative to traditional methods.
Study examines unsupervised and graph-based methods for anomaly detection in IoBT, outperformed by supervised stacking ensemble.
problem Anomaly detection in adversarial environments of IoBT.
method Unsupervised learning, graph-based methods, ensemble supervised learning, adversarial training.
result Supervised stacking ensemble method outperforms unsupervised and graph-based methods in detecting anomalies.
Deep RL policies share adversarial features across different MDPs.
problem Understanding decision boundaries and loss landscapes in neural policies.
method Investigating similarities in high sensitivity directions across MDPs using Arcade Learning Environment.
result High sensitivity directions for neural policies are correlated across MDPs, suggesting shared non-robust features.
Paper introduces timing-based adversarial attacks on DRL-based navigation systems.
problem Vulnerability of DRL-based navigation systems to adversarial attacks.
method Timing-based adversarial strategies using physical noise patterns.
result Adversarial timing attacks significantly degrade DRL-based navigation performance.
Defense strategy improves controller robustness against adversarial attacks.
problem Adversarial attacks on learning-enabled controllers in CPS.
method Two-stage defense strategy treating controller and environment as black-boxes with unknown dynamics.
result Defense strategy effectively improves controller robustness in realistic control domains.
New algorithm reduces sample inefficiency and reward bias in AI learning.
problem Implicit reward bias and high sample inefficiency in AI learning.
method Discriminator-Actor-Critic using off-policy Reinforcement Learning.
result Average 10x reduction in policy-environment interaction samples.
New adversarial training method improves robustness of power system controllers.
problem Designing robust controllers for complex cyber-physical power systems.
method Adversarial training approach with fixed opponent policy.
result Adversarial trained controllers show useful preventive behaviors in the N-1 problem.
Paper tackles noise-robust domain adaptation in noisy environments.
problem Learning machines struggle with domain adaptation in noisy environments.
method The paper proposes offline curriculum learning and proxy distribution based margin discrepancy to mitigate label and feature noise.
result The proposed algorithm significantly outperforms state-of-the-art methods in noisy environments.
The paper improves safety in autonomous systems using adversarial learning.
problem Ensuring safety in real-time control systems of autonomous vehicles.
method The paper introduces a dual anomaly detection framework (CFAM and SFAM) using generative adversarial networks (GANs) and video prediction.
result Demonstrated effectiveness on both indoor and outdoor autonomous ground vehicles.
This research develops secure DSVM algorithms using game theory.
problem Vulnerability of DSVM in adversarial environments.
method Game-theoretic framework to model conflicting interests between adversary and DSVM units.
result Guaranteed convergence of distributed learning algorithms without data or network topology assumptions.
Algorithm improves imitation learning from visual data in partially observable environments.
problem Imitation learning from visual observations with missing expert actions and partial observability.
method Theoretical analysis and Latent Adversarial Imitation from Observations algorithm combining adversarial and latent representations.
result Latent Adversarial Imitation from Observations achieves state-of-the-art performance in high-dimensional robotic tasks.
False negatives hinder adversarial imitation learning, leading to worse performance.
problem False negatives impede the performance of adversarial imitation learning.
method Proposed a method to alleviate the impact of false negatives.
result The method consistently improves sample efficiency by at least an order of magnitude.
New algorithms adaptively calibrate predictions in non-stationary environments, matching optimal rates.
problem Designing online prediction algorithms that adapt to varying levels of non-stationarity.
method Epoch-based scheduling and non-uniform partitioning of the prediction space.
result Achieves adaptive calibration guarantees under multiple measures with optimal rates.
Adversarial CBO optimizes under interventions by adversaries and non-stationarities.
problem Optimizing in the presence of adversaries and non-stationary factors.
method Formalizes CBO as ACBO, introduces CBO-MW algorithm combining online learning and causal modeling.
result First algorithm with bounded regret for ACBO, achieving superior performance in synthetic and real-world environments.
New insights link no-regret learning to online conformal prediction in adversarial settings.
problem Understanding the relationship between no-regret learning and online conformal prediction in adversarial environments.
method Analysis of existing algorithms and new connections between no-regret learning and conformal prediction.
result No-regret learning algorithms can provide group-conditional coverage guarantees in adversarial settings.
Improved ExO method achieves near-optimal bounds in both stochastic and adversarial settings.
problem Finding optimal exploration strategies in online decision-making with limited feedback.
method Exploration by Optimization with hybrid regularizers for locally observable games.
result Achieved nearly optimal bounds of O ( ∑ a e q a ∗ k 2 m 2 log T / Δ a ) O(\sum_{a
eq a^*} k^2 m^2 \log T / Δ_a) O ( ∑ a e q a ∗ k 2 m 2 log T / Δ a ) in stochastic and adversarial environments. New framework for robust reinforcement learning policies in uncertain environments.
problem Robust reinforcement learning policies in environments with distributional shifts.
method Comprehensive modeling framework centered around robust Markov decision processes (RMDPs).
result Existence and conditions for the dynamic programming principle (DPP) in RMDPs.
Improves RL generalization by minimizing adversarial risk.
problem Overfitting to training environments and poor generalization to unseen scenarios.
method Introduces minimax formulation and distributional framework to RL.
result Trained policy shows improved generalization to different environments.
Attackers can poison environments to force RL agents to follow target policies.
problem Security threat to reinforcement learning where attackers manipulate environments to force agents into following target policies.
method Optimization framework for finding optimal stealthy attacks under different measures of attack cost.
result Attackers can easily succeed in teaching any target policy to RL agents under mild conditions.
Adapts models incrementally for continual changes in environments.
problem Continual appearance shifts impact machine learning model performance.
method Adversarial approach for lifelong, incremental domain adaptation.
result Improves handling of large appearance changes compared to direct methods.
CopyCAT attacks neural policies by manipulating observations, not states.
problem Adversarial attacks on neural reinforcement learning agents.
method CopyCAT is a targeted attack that lures agents into following outsider policies.
result CopyCAT is effective in a read-only setting where adversaries can only manipulate observations.
Paper proposes a hybrid model for financial time series prediction using sentiment analysis.
problem Challenges in forecasting in non-stationary, complex environments with heterogeneous data.
method Hybrid model combining GANs with NLP-based sentiment analysis.
result Hybrid model enhances robustness in non-stationary environments.
New method detects anomalies in systems influenced by their environment.
problem Detecting anomalies in systems under environmental influence.
method Adversarial learning and time series representation learning.
result Successfully addresses label sparsity and subjectivity in anomaly detection.
Study robust control for systems with continuous states using adversarial perturbations.
problem Fragile policies in Markov control models under internal or external perturbations.
method Distributionally robust stochastic control with adaptive adversarial perturbations.
result Optimal robust policies for continuous state systems with uniform learning guarantees.
Adversarial trading samples hurt financial markets.
problem Impact of adversarial samples on financial markets.
method Implemented adversarial samples in a trading environment.
result Adversarial samples negatively impact certain market participants.