Paper introduces DOO models to outperform SAA out-of-sample.
problem Outperforming SAA in out-of-sample performance.
method Introduces DOO models that consider both worst-case and best-case scenarios.
result DOO models can always outperform SAA out-of-sample.
Develops ODRPO to improve RL algorithms with better performance and stability.
problem RL algorithms converge to sub-optimal solutions due to limited policy representation.
method Integrates DRO approach to solve trust region constrained optimization problem without parameterizing policies.
result Achieves globally optimal policy update and higher sample efficiency.
The likelihood function is a fundamental component in Bayesian statistics. However, evaluating the likelihood of an observation is computationally intractable in many applications. In this paper, we propose a non-parametric approximation of the likelihood that identifies a probability measure which lies in the neighbor…
New framework improves learning across multiple distributions.
problem Modeling uncertainty in sensitive machine learning applications.
method Inspired by multi-armed bandits, provides distribution-dependent guarantees.
result Enhanced dependence on suboptimality gaps and sample size.
Real-world applications require RL algorithms to act safely. During learning process, it is likely that the agent executes sub-optimal actions that may lead to unsafe/poor states of the system. Exploration is particularly brittle in high-dimensional state/action space due to increased number of low-performing actions. …
New optimization method corrects data-driven optimizer's curse.
problem Over-optimistic evaluation in data-driven optimization.
method Smoothed f f f -Divergence Distributionally Robust Optimization (DRO). result Statistical bound on out-of-sample performance nearly tightest.
Paper proposes a framework for reliable off-policy evaluation in reinforcement learning.
problem Quantifying uncertainty in off-policy estimates for safe deployment of target policies.
method Distributionally robust optimization for creating confidence bounds.
result Non-asymptotic and asymptotic guarantees for robust cumulative reward estimates.
DR-RPO optimizes robust policies in RL with limited interaction, achieving sublinear regret.
problem Policy optimization in RL under distribution shift and adversarial dynamics.
method DR-RPO algorithm incorporating reference-policy regularization and upper confidence bonus for exploration.
result DR-RPO achieves sublinear regret and polynomial suboptimality bounds in robust RL.
A general duality proof for Wasserstein distributionally robust optimization.
problem Optimizing under uncertainty with Wasserstein distance.
method One-dimensional convex analysis and interchangeability principle.
result General duality result holds for various distributions and costs.
Wasserstein distributionally robust optimization estimators are obtained as solutions of min-max problems in which the statistician selects a parameter minimizing the worst-case loss among all probability models within a certain distance (in a Wasserstein sense) from the underlying empirical measure. While motivated by…
Optimistic NPG improves policy optimization in online RL with efficient sample complexity.
problem Limited theoretical understanding of policy optimization, especially in online RL.
method Combines natural policy gradient with optimistic policy evaluation.
result Achieves optimal dimension dependence sample complexity for learning near-optimal policies.
Develops robust learning framework under distributional perturbations.
problem Learning robust to data distributional changes.
method Distributionally Robust Optimization (DRO) under Wasserstein metric.
result Establishes performance guarantees and tractable formulations.
New method corrects bias in estimating entropic risk for better decision-making.
problem Underestimation of entropic risk when data are limited.
method Parametric bootstrap procedure to overestimate entropic risk.
result Corrected method provides better risk estimates, leading to improved decision-making.
Paper develops a robust Bayesian optimization method for noisy zeroth-order settings.
problem Achieving robustness to distributional shift in machine learning.
method Distributionally robust Bayesian optimization (DRBO) algorithm for noisy zeroth-order optimization.
result DRBO algorithm provably obtains sub-linear robust regret in various settings.
Omega method mitigates noise in stochastic game optimization.
problem Noise sensitivity and convergence issues in stochastic game optimization.
method Omega method incorporates EMA of historic gradients in its update rule.
result Omega method outperforms optimistic gradient method in stochastic games.
Optimizes distributions robustly with Sinkhorn distance.
problem Distributionally robust optimization with Wasserstein distance.
method Convex programming dual reformulation, stochastic mirror descent algorithm.
result Demonstrates superior performance in synthetic and real data.
Tikhonov regularization is robust under specific martingale constraints in distributionally robust optimization.
problem Distributionally robust optimization and regularization of learning models.
method Optimal transport approach with martingale constraints.
result Tikhonov regularization is optimal transport robust under specified martingale constraints.
New algorithms reduce forecasting errors by leveraging optimistic learning and hinting.
problem Real-time climate forecasting with delayed feedback.
method Developed optimistic online learning algorithms that reduce regret under delayed feedback.
result Optimistic learning algorithms achieve optimal regret guarantees under delayed feedback.
New scalable methods for robust model learning from large datasets.
problem Training robust models resistant to data distribution shifts.
method Composite optimization for distributionally robust optimization (DRO).
result Scalable methods for learning robust models from large datasets.
New regularizer for machine learning using private data.
problem Machine learning with private data.
method Distributionally-robust optimization with locally-differentially-private datasets.
result New regularizer for training linear regression models.
Proposes Optimistic Pessimistically Initialised Q-Learning (OPIQ) for better exploration in RL.
problem Pessimistic initialisation of Q-values in deep RL leads to poor exploration performance.
method Augments pessimistically initialised Q-values with count-based bonuses to ensure optimism.
result OPIQ outperforms non-optimistic DQN variants in hard exploration tasks.
This paper tackles cost-sensitive portfolio optimization under ambiguous return distributions.
problem Tackles cost-sensitive distributionally robust log-optimal portfolio problem with ambiguous return distributions.
method Uses Wasserstein metric for distributional ambiguity, incorporates convex transaction costs, and approximates infinite-dimensional problem with finite convex program.
result Establishes conditions for robustly survivable trades and validates theoretical framework with empirical studies.
Proposes using Wasserstein barycenters for robust optimization with multiple data sources.
problem Distributionally robust optimization with multiple heterogeneous data sources.
method Construct nominal distribution through Wasserstein barycenter of multiple data samples, reformulates as a finite convex program.
result Proposed scheme outperforms other estimators in sparse inverse covariance matrix estimation.
Extends ML fairness to handle minority groups over time.
problem Limitations of existing fairness criteria.
method Performative Distributionally Robust Optimization.
result Improves fairness for minority groups over time.
Study shows convergence of stochastic gradient method for unregularized Wasserstein optimization.
problem Wasserstein distributionally robust optimization under potential distribution shifts.
method Regularized approximation with stochastic gradient methods, convergence analysis.
result Stochastic gradient method converges to subgradients of unregularized objective as regularization vanishes.
A new algorithm reduces bias and variance in distributionally robust optimization.
problem Distributionally robust optimization with bias and variance issues.
method Prospect, a stochastic gradient-based algorithm that reduces hyperparameter tuning.
result Prospect achieves linear convergence and 2-3x faster convergence on various benchmarks.
New algorithm reduces regret in sequential decision-making problems.
problem Balancing exploration and exploitation in online sequential decision problems.
method Variational Bayesian optimistic sampling (VBOS) for optimizing policies.
result VBOS achieves i l d e O ( A T ) ilde O(\sqrt{AT}) i l d e O ( A T ) Bayesian regret for stochastic multi-armed bandits. Study optimal transport for robust optimization, showing how adversary's strategy relates to regularization.
problem Optimizing under uncertain parameters with a fictitious adversary reshaping a reference distribution.
method Introduces optimal transport and regularization to relate robustification to variation and Lipschitz norms.
result Conditions for existence and computability of Nash equilibrium between decision-maker and adversary.
Optimistic method adapted for faster convex-concave min-max problems.
problem Solving convex-concave min-max optimization problems efficiently.
method Adaptive, line search-free second-order methods combining optimistic updates and second-order information.
result Achieves optimal convergence rate without line search or backtracking.
This work evaluates risks over time using robust measures and neural networks.
problem Distributionally robust risk evaluation over temporal data.
method Characterizes alternative measures using causal optimal transport, approximates test functions by neural networks, and proves sample complexity.
result Framework outperforms classic counterparts in portfolio selection problems.
Unified framework for model- and value-optimistic reinforcement learning.
problem Designing and analyzing algorithms for episodic reinforcement learning.
method Lagrangian duality for a unified view of optimism.
result Unified algorithms with both efficient implementation and probabilistic analysis.
Paper proposes online optimization for uncertain systems using machine learning and DRO.
problem Optimization of uncertain dynamical systems with distributional uncertainty.
method Combines machine learning with Distributional Robust Optimization (DRO) to handle uncertainty.
result Online solutions with probabilistic regret bounds for uncertain systems.
A new gradient flow framework for distributionally robust optimization.
problem Optimizing under uncertainty with worst-case distributional constraints.
method Gradient flow theory applied to distributionally robust optimization.
result Practical algorithms for sampling from worst-case distributions.
Bayesian quadrature optimization tackles uncertainty in distributional samples.
problem Maximizing an expensive black-box integrand under distributional uncertainty.
method Distributionally robust optimization perspective, posterior sampling.
result Empirical effectiveness and theoretical convergence demonstrated.
Proposes a risk parity portfolio optimization method that accounts for uncertainty in asset returns.
problem Risk parity portfolio optimization under uncertainty.
method Distributionally robust optimization with ambiguity set for worst-case scenario analysis.
result Distributionally robust risk parity portfolios can yield higher risk-adjusted returns.
Accelerates policy optimization in RL with optimistic and adaptive updates.
problem Improving policy optimization methods in reinforcement learning.
method Integrates foresight into policy improvement step via optimistic and adaptive updates.
result Designs an optimistic policy gradient algorithm, adaptive via meta-gradient learning.
Data-driven Distributionally Robust Optimization (DD-DRO) via optimal transport has been shown to encompass a wide range of popular machine learning algorithms. The distributional uncertainty size is often shown to correspond to the regularization parameter. The type of regularization (e.g. the norm used to regularize)…
Drago optimizes DRO problems with faster convergence.
problem Distributionally robust optimization with closed, convex uncertainty sets.
method Primal-dual coupled variance reduction algorithm with cyclic and randomized updates.
result Achieves state-of-the-art linear convergence rate on strongly convex-strongly concave problems.
This paper develops a model of reference-dependent assessment of subjective beliefs in which loss-averse people optimally choose the expectation as the reference point to balance the current felicity from the optimistic anticipation and the future disappointment from the realisation. The choice of over-optimism or over…
Unified framework for DRO using OT with constraints.
problem Handling ambiguity in likelihood ratios and outcomes.
method Unified framework leveraging optimal transport with conditional moment constraints.
result Unified approach enables adversarial perturbation of likelihood ratios and outcomes.
A new framework for performative prediction robust to distributional misspecification.
problem Performative prediction models can be influenced by their own predictions, leading to suboptimal outcomes.
method Introduces distributionally robust performative prediction (DRPO) to approximate the true performative optimum (PO) robustly.
result DRPO provides provable guarantees as a robust approximation to the true PO when the nominal distribution map is misspecified.
CADRO optimizes DRO by reducing conservatism through cost-aware ambiguity sets.
problem Optimizing solutions under uncertainty with reduced conservatism.
method CADRO uses a cost-aware ambiguity set to reduce DRO's conservatism.
result CADRO provides high-confidence upper bounds and consistent estimators of out-of-sample expected cost.
The concepts of risk-aversion, chance-constrained optimization, and robust optimization have developed significantly over the last decade. Statistical learning community has also witnessed a rapid theoretical and applied growth by relying on these concepts. A modeling framework, called distributionally robust optimizat…
New algorithm improves RL performance across different environments.
problem Improving reinforcement learning performance across various environments.
method Designing a fully model-free DRRL algorithm that learns from a single trajectory.
result Demonstrates superior robustness and sample efficiency compared to existing methods.
A new method uses GANs for robust optimization under uncertain data.
problem Optimizing supply chains under demand uncertainty with ambiguous distributions.
method Generative adversarial networks (GANs) for data-driven distributionally robust chance constrained programming.
result The approach effectively handles uncertain data distributions and improves supply chain optimization.
Efficiently addresses federated learning challenges with reduced communication and sample complexity.
problem Heterogeneity in data volumes and distributions at different clients compromises model generalization ability.
method Introduces algorithms for communication-efficient Federated Group Distributionally Robust Optimization (FGDRO).
result Communication complexity reduced to O ( 1 / ε 4 ) O(1/ε^4) O ( 1/ ε 4 ) for FGDRO-CVaR and O ( 1 / ε 3 ) O(1/ε^3) O ( 1/ ε 3 ) for FGDRO-KL. Optimistic Hedge achieves optimal regret bounds in two-player zero-sum games.
problem Achieving optimal regret bounds for optimistic Hedge in two-player zero-sum games.
method Refined regret analysis and optimization problem formulation.
result Optimistic Hedge achieves O ( log m log n ) O(\sqrt{\log m \log n}) O ( log m log n ) regret bounds, matching upper and lower bounds. CreDRO learns credal ensembles via distributionally robust optimization, improving EU quantification.
problem Quantifying predictive epistemic uncertainty in credal models.
method Distributionally robust optimization to capture EU from training randomness and potential distribution shifts.
result Empirically, CreDRO outperforms existing credal methods on various tasks.