New algorithm improves RL performance across different environments.
problem Improving reinforcement learning performance across various environments.
method Designing a fully model-free DRRL algorithm that learns from a single trajectory.
result Demonstrates superior robustness and sample efficiency compared to existing methods.
New algorithms tackle robust RL with linear models, revealing unique challenges.
problem Distributionally robust offline RL with uncertainty in dynamics.
method Proposes minimax optimal and computationally efficient algorithms using novel function approximation mechanisms.
result Function approximation in robust offline RL is distinct and harder than in standard offline RL.
Develops ODRPO to improve RL algorithms with better performance and stability.
problem RL algorithms converge to sub-optimal solutions due to limited policy representation.
method Integrates DRO approach to solve trust region constrained optimization problem without parameterizing policies.
result Achieves globally optimal policy update and higher sample efficiency.
This work bridges offline RL and DRL to address distributional shift.
problem Distributional shift in offline RL due to difference in state-action visitation distributions.
method Proposes offline RL algorithms using DRL framework, characterizes sample complexity under single policy concentrability.
result Demonstrates superior performance of proposed algorithms through simulations.
This paper tackles RL issues with robust policies using historical data.
problem Limited data and mismatch between training and testing environments.
method Distributionally robust offline RL with linear function approximation.
result Achieved error bounds for sample complexity in RL.
Paper tackles robust reinforcement learning with minimal data.
problem Learning robust policies from limited data in uncertain environments.
method Distributionally robust formulation, model-based algorithm combining value iteration and pessimism.
result Proves near-optimal sample complexity for robust offline RL.
Paper studies S-rectangular DR-RL models for robust reinforcement learning with near-optimal sample complexity.
problem Addressing distributional discrepancies in reinforcement learning environments.
method Empirical value iteration algorithm for divergence-based S-rectangular DR-RL models.
result Near-optimal sample complexity bound of O ( ∣ S ∣ ∣ A ∣ ( 1 − γ ) − 4 ε − 2 ) O(|\mathcal{S}||\mathcal{A}|(1-γ)^{-4}\varepsilon^{-2}) O ( ∣ S ∣∣ A ∣ ( 1 − γ ) − 4 ε − 2 ) . New RL method tackles sim-to-real gap using interactive data collection.
problem Sim-to-real gap in reinforcement learning.
method Distributionally robust reinforcement learning with interactive data collection.
result Proves sample-efficient learning is impossible without additional assumptions.
DR-RPO optimizes robust policies in RL with limited interaction, achieving sublinear regret.
problem Policy optimization in RL under distribution shift and adversarial dynamics.
method DR-RPO algorithm incorporating reference-policy regularization and upper confidence bonus for exploration.
result DR-RPO achieves sublinear regret and polynomial suboptimality bounds in robust RL.
Real-world applications require RL algorithms to act safely. During learning process, it is likely that the agent executes sub-optimal actions that may lead to unsafe/poor states of the system. Exploration is particularly brittle in high-dimensional state/action space due to increased number of low-performing actions. …
New model-free DR-RL algorithm with finite sample complexity.
problem Limited model-free DR-RL methods with convergence guarantees or sample complexities.
method Integrates Multi-level Monte Carlo (MLMC) technique with threshold mechanism.
result First model-free DR-RL approach with finite sample complexity for total variation and Chi-square divergence.
Improved RL algorithm for robustness against parameter mismatches.
problem Learning robust control policies against parameter mismatches between training and testing environments.
method Formulated as DR-RL problem, proposed RPVL algorithm for tabular episodic learning with four divergences.
result Achieved i l d e O ( ∣ S ∣ ∣ A ∣ H 5 ) ilde{\mathcal{O}}(|\mathcal{S}||\mathcal{A}| H^{5}) i l d e O ( ∣ S ∣∣ A ∣ H 5 ) sample complexity uniformly better than existing results. This work tackles robust RL in multi-agent settings, improving sample efficiency.
problem Overcoming environmental uncertainties in multi-agent reinforcement learning.
method Proposes DRNVI, a sample-efficient algorithm for learning robust equilibria in RMGs.
result Establishes near-optimal sample complexity for solving RMGs.
A new algorithm improves offline reinforcement learning robustness.
problem Finding optimal policies in perturbed environments from offline data.
method Doubly Pessimistic Model-based Policy Optimization (P^2MPO) framework.
result Proves sample efficiency with robust partial coverage data.
Improved reinforcement learning for environments with distributional shifts.
problem Learning optimal policies in environments with distributional shifts.
method Distributionally robust Q-learning with multi-level Monte Carlo estimator.
result Proved upper bound on sample complexity for robust RL.
We develop a robust RL algorithm for off-dynamics environments with improved suboptimality bounds and computational efficiency.
problem Learning policies robust to uncertainties in transition dynamics between training and deployment environments.
method Distributionally robust Markov decision processes (DRMDPs) with a novel algorithm We-DRIVE-U.
result Improved suboptimality bound of O ~ ( d H ⋅ min { 1 / ρ , H } / K ) \widetilde{\mathcal{O}}\big({d H \cdot \min \{1/ρ, H\}/\sqrt{K} }\big) O ( d H ⋅ min { 1/ ρ , H } / K ) , near-optimal up to O ( H ) \mathcal{O}(\sqrt{H}) O ( H ) . STEEL tackles batch RL with singularity, improving policy optimization.
problem Existing RL methods assume absolutely continuous data, but STEEL handles non-overlapping regions.
method Proposes STEEL algorithm using maximum mean discrepancy and distributionally robust optimization.
result First finite-sample regret guarantee for batch RL with singularity.
Algorithm learns robust equilibrium in online Markov games with interactive data.
problem Sim-to-real gap in reinforcement learning.
method Distributionally robust RL with minimum value assumption, least square value iteration.
result Sample-efficient algorithm for robust equilibrium in online Markov games.
Paper tackles robust offline RL for non-Markovian processes, improving efficiency and applicability.
problem Learning robust policies for non-Markovian decision processes with limited offline data.
method Proposes a novel algorithm with dataset distillation and LCB design for robust values, derived new dual forms, and introduces concentrability coefficients.
result Proves polynomial sample efficiency for finding ε-optimal robust policies.
Paper tackles offline preference-based RL with human feedback.
problem Offline Preference-based Reinforcement Learning with preference feedback.
method Two-step approach: MLE for reward estimation and distributionally robust planning.
result First guarantee for learning any target policy with polynomial samples.
Study online RL with mismatched dynamics, achieving sublinear regret.
problem Exploration challenges in online RL with mismatched training and deployment dynamics.
method Introduce supremal visitation ratio, propose efficient algorithm with f f f -divergence. result Achieves sublinear regret in online RMDPs with optimal dependence on supremal visitation ratio and interaction episodes.
A general duality proof for Wasserstein distributionally robust optimization.
problem Optimizing under uncertainty with Wasserstein distance.
method One-dimensional convex analysis and interchangeability principle.
result General duality result holds for various distributions and costs.
Develops robust learning framework under distributional perturbations.
problem Learning robust to data distributional changes.
method Distributionally Robust Optimization (DRO) under Wasserstein metric.
result Establishes performance guarantees and tractable formulations.
Wasserstein distributionally robust optimization estimators are obtained as solutions of min-max problems in which the statistician selects a parameter minimizing the worst-case loss among all probability models within a certain distance (in a Wasserstein sense) from the underlying empirical measure. While motivated by…
Paper develops a robust Bayesian optimization method for noisy zeroth-order settings.
problem Achieving robustness to distributional shift in machine learning.
method Distributionally robust Bayesian optimization (DRBO) algorithm for noisy zeroth-order optimization.
result DRBO algorithm provably obtains sub-linear robust regret in various settings.
New scalable methods for robust model learning from large datasets.
problem Training robust models resistant to data distribution shifts.
method Composite optimization for distributionally robust optimization (DRO).
result Scalable methods for learning robust models from large datasets.
Tikhonov regularization is robust under specific martingale constraints in distributionally robust optimization.
problem Distributionally robust optimization and regularization of learning models.
method Optimal transport approach with martingale constraints.
result Tikhonov regularization is optimal transport robust under specified martingale constraints.
A new framework for performative prediction robust to distributional misspecification.
problem Performative prediction models can be influenced by their own predictions, leading to suboptimal outcomes.
method Introduces distributionally robust performative prediction (DRPO) to approximate the true performative optimum (PO) robustly.
result DRPO provides provable guarantees as a robust approximation to the true PO when the nominal distribution map is misspecified.
Paper proves robust estimators' generalization guarantees without dimensionality issues.
problem Generalization guarantees for Wasserstein distributionally robust models.
method Analyzes and extends existing guarantees to broader classes of models and regularized versions.
result Generalization guarantees hold without dimensionality issues and cover distribution shifts.
New approach improves model generalization through distributionally robust learning.
problem Improving model generalization in machine learning.
method Stochastic gradient descent applied to the outer minimization problem, with gradient estimation through multi-level Monte Carlo randomization.
result Our approach yields significant benefits over previous work in numerical experiments.
Study uses online bootstrap for RL inference, showing effectiveness.
problem Statistical inference for RL parameters in online settings.
method Online bootstrap method applied to TD and GTD algorithms in RL.
result Method is distributionally consistent for policy evaluation inference.
New regularizer for machine learning using private data.
problem Machine learning with private data.
method Distributionally-robust optimization with locally-differentially-private datasets.
result New regularizer for training linear regression models.
The paper connects three machine learning methods to reduce generalization errors.
problem Reducing generalization errors in machine learning models.
method Distributionally robust optimization, Bayesian methods, and regularization.
result Machine learning models can be characterized using distributional uncertainty and robustness measures.
Proposes using Wasserstein barycenters for robust optimization with multiple data sources.
problem Distributionally robust optimization with multiple heterogeneous data sources.
method Construct nominal distribution through Wasserstein barycenter of multiple data samples, reformulates as a finite convex program.
result Proposed scheme outperforms other estimators in sparse inverse covariance matrix estimation.
Improved estimator reduces bias in statistical learning models.
problem Asymptotic bias in classic WDRO estimator.
method Adjusted Wasserstein distributionally robust estimator.
result Asymptotic unbiased estimator with smaller MSE.
Extends ML fairness to handle minority groups over time.
problem Limitations of existing fairness criteria.
method Performative Distributionally Robust Optimization.
result Improves fairness for minority groups over time.
A method for robust reinforcement learning in large state spaces.
problem Challenges in RL with large state spaces, costly data, and real-world dynamics deviation.
method Distributionally robust Markov decision processes with Gaussian Processes and maximum variance reduction.
result Efficient learning of multi-output nominal transition dynamics with statistical sample complexity bounds.
This work evaluates risks over time using robust measures and neural networks.
problem Distributionally robust risk evaluation over temporal data.
method Characterizes alternative measures using causal optimal transport, approximates test functions by neural networks, and proves sample complexity.
result Framework outperforms classic counterparts in portfolio selection problems.
Optimizes distributions robustly with Sinkhorn distance.
problem Distributionally robust optimization with Wasserstein distance.
method Convex programming dual reformulation, stochastic mirror descent algorithm.
result Demonstrates superior performance in synthetic and real data.
End-to-end portfolio system accounts for model risk.
problem Model risk in portfolio selection.
method Distributionally robust optimization with convex duality.
result Explicitly accounts for model risk in portfolio selection.
Study improves adversarial classification using distributionally robust models.
problem Improving robustness against adversarial attacks in classification models.
method Distributionally robust chance constraints with Wasserstein ambiguity, reformulated as a regularized ramp loss minimization problem.
result Standard descent methods can converge to the global minimizer for the distributionally robust adversarial classification model.
Proposes a risk parity portfolio optimization method that accounts for uncertainty in asset returns.
problem Risk parity portfolio optimization under uncertainty.
method Distributionally robust optimization with ambiguity set for worst-case scenario analysis.
result Distributionally robust risk parity portfolios can yield higher risk-adjusted returns.
We study a distributionally robust mean square error estimation problem over a nonconvex Wasserstein ambiguity set containing only normal distributions. We show that the optimal estimator and the least favorable distribution form a Nash equilibrium. Despite the non-convex nature of the ambiguity set, we prove that the …
Scaff-PD improves fairness and robustness in federated learning with reduced communication.
problem Improving fairness and robustness in federated learning with limited communication.
method Scaff-PD uses a family of distributionally robust objectives and an accelerated primal dual algorithm with bias-corrected steps.
result Scaff-PD achieves significant gains in communication efficiency and convergence speed while maintaining fairness and robustness.
Paper introduces DOO models to outperform SAA out-of-sample.
problem Outperforming SAA in out-of-sample performance.
method Introduces DOO models that consider both worst-case and best-case scenarios.
result DOO models can always outperform SAA out-of-sample.
Data-driven Distributionally Robust Optimization (DD-DRO) via optimal transport has been shown to encompass a wide range of popular machine learning algorithms. The distributional uncertainty size is often shown to correspond to the regularization parameter. The type of regularization (e.g. the norm used to regularize)…
A new algorithm reduces bias and variance in distributionally robust optimization.
problem Distributionally robust optimization with bias and variance issues.
method Prospect, a stochastic gradient-based algorithm that reduces hyperparameter tuning.
result Prospect achieves linear convergence and 2-3x faster convergence on various benchmarks.
Exact generalization guarantees for robust models using Wasserstein distance are established.
problem Capturing data uncertainty and distribution shifts in machine learning models.
method Establishes exact generalization guarantees for robust models based on the Wasserstein distance, covering various cases and transport costs.
result Exact generalization guarantees are provided for a wide range of cases, including deep learning objectives with nonsmooth activations.