The paper provides a method to minimize regret in estimate-then-optimize decision-making.
problem Errors in estimation lead to sub-optimal decisions in data-driven decision-making.
method A novel bound on regret for smooth and unconstrained optimization problems, followed by experimental design to minimize this regret.
result A general procedure for experimental design to minimize regret resulting from estimate-then-optimize.
Data-driven model selection reduces regret in sequential decisions.
problem Optimizing model selection in stochastic environments with bandit feedback.
method Data-driven regret balancing for model selection.
result Meta-learner selects the best base learner based on actual realized regret.
Algorithm reduces long-term policy regret in ML decision-making.
problem Capturing long-term impacts of ML decisions in communities.
method Modeling communities as arms in a multi-armed bandit problem, defining policy regret as a stronger metric than external regret.
result Algorithm achieves provably sub-linear policy regret for long time horizons.
Paper presents an algorithm for optimal regret in communicating Markov decision processes.
problem Achieving optimal regret in Markov decision processes with a communicating assumption.
method The algorithm explicitly tracks the constant K(M) to learn optimally, balancing exploration, co-exploration, and exploitation.
result The algorithm achieves asymptotically optimal regret K ( M ) log ( T ) + o ( log ( T ) ) K(M) \log(T) + \mathrm{o}(\log(T)) K ( M ) log ( T ) + o ( log ( T )) for communicating Markov decision processes. New method minimizes regret in AMDP with high probability.
problem Pessimistic regret analysis in AMDP.
method Developed a new policy search method achieving optimistic regret.
result Achieved sublinear optimistic regret with high probability.
New bounds for γ γ γ -regret using modified Decision-Estimation Coefficient.
problem Statistical characterization of γ γ γ -regret for complex bandit problems. method Statistical characterization via γ γ γ -DEC, a modified Decision-Estimation Coefficient. result Upper and lower bounds for γ γ γ -regret nearly match, showing fundamental limits. Paper addresses regret minimization and inference in high-dimensional online decision-making.
problem Regret minimization and statistical inference in high-dimensional online decision-making.
method Integrates ε-greedy bandit algorithm with hard thresholding for sparse bandit parameters and debiasing method for inference.
result Achieves either O ( T 1 / 2 ) O(T^{1/2}) O ( T 1/2 ) regret or O ( T 1 / 2 ) O(T^{1/2}) O ( T 1/2 ) -consistent inference, with trade-off between exploration and exploitation. New algorithm reduces online decision-making regret with efficient LP re-solving and parallel first-order method.
problem Worse regret guarantees and high computational cost of LP-based OLP algorithms.
method Combines LP-based and first-order OLP methods, re-solving LP subproblems periodically and using parallel first-order method.
result Achieves O ( log ( T / f ) + f ) \mathscr{O}(\log (T/f) + \sqrt{f}) O ( log ( T / f ) + f ) regret, balancing computational efficiency and superior regret guarantee. dboost optimizes prediction models for convex cone problems.
problem Optimizing prediction models for decision-making.
method Gradient boosting with implicit differentiation for convex quadratic cone programming.
result dboost reduces out-of-sample decision regret.
Unified approach for non-stationary linear bandits with dynamic regret.
problem Non-stationary linear bandits with round-specific feasible actions and drifting reward models.
method Unified misspecification-reduction viewpoint, restarting algorithms with misspecification-dependent regret guarantees.
result Optimal \(T^{2/3}P_T^{1/3}\) dynamic-regret dependence for both linear bandits and contextual linear bandits.
The paper introduces new measures to quantify variability in decision tree models due to observational multiplicity.
problem The variability in decision tree models due to observational multiplicity.
method Introduces leaf regret and structural regret to decompose observational multiplicity.
result Structural regret is the primary driver of observational multiplicity, accounting for over 15 times the variability of leaf regret in some datasets.
OE2D framework reduces contextual bandits to offline regression for near-optimal regret.
problem Efficiently learning contextual bandits with large action spaces and complex reward functions.
method Offline Estimation to Decisions (OE2D) algorithm that reduces contextual bandits to offline regression.
result Near-optimal regret for contextual bandits with large action spaces and O ( log T ) O(\log T) O ( log T ) calls to an offline regression oracle. New complexity measure for interactive learning reduces regret to near-optimal levels.
problem Challenges in sample-efficient, adaptive learning algorithms for interactive decision making.
method Introduces the Decision-Estimation Coefficient and the Estimation-to-Decisions (E2D) principle.
result Unified algorithm design principle E2D achieves optimal sample-efficient learning.
Paper introduces a new G ⋆ G^\star G ⋆ regret measure for online convex optimization with smooth losses.
problem Online convex optimization with smooth losses.
method Introduces a new G ⋆ G^\star G ⋆ regret measure that depends on the cumulative squared gradient norm. result The G ⋆ G^\star G ⋆ regret can be arbitrarily sharper than existing measures when losses have vanishing curvature. As a metric to measure the performance of an online method, dynamic regret with switching cost has drawn much attention for online decision making problems. Although the sublinear regret has been provided in many previous researches, we still have little knowledge about the relation between the dynamic regret and the s…
Paper tackles non-monotonic resource utilization in sequential decision-making.
problem Sequential decision-making under uncertainty with resource constraints.
method Introduces a new MDP policy with constant regret against LP relaxation.
result Develops a learning algorithm with logarithmic regret for unknown outcome distributions.
Improved model-free reinforcement learning with decision-estimation coefficient.
problem Interactive decision making, including structured bandits and reinforcement learning.
method Combining Estimation-to-Decisions with optimistic estimation to achieve better regret bounds.
result Regret bounds for model-free reinforcement learning with value function approximation.
Paper eliminates warm-up phase for PO in linear MDPs, achieving optimal regret.
problem Costly warm-up phase in PO algorithms for linear MDPs.
method Simple contraction mechanism replaces warm-up phase.
result Achieves rate-optimal regret with improved dependence on problem parameters.
The study analyzes how covariance estimation errors affect the global minimum-variance portfolio under heavy-tailed distributions.
problem The impact of covariance estimation errors on the global minimum-variance portfolio under heavy-tailed distributions.
method Characterization of covariance-estimation error's effect on GMVP suboptimality, derivation of regret identity and bound, application to heavy-tailed returns.
result The decision geometry of GMVP regret is invariant to a (p-1)-dimensional projection of the error matrix, with invariance to the covariance-scale direction as an exact special case.
The study quantifies decision-making risks from suboptimal classifiers and proposes methods to reduce these risks.
problem Excess risk in decision-making from suboptimal probabilistic classifiers.
method Analytical expressions and upper/lower bounds for excess risk, calibration curve estimation, grouping loss estimator.
result Identifies regimes where recalibration alone or post-training is more effective.
Framework reduces contextual bandit learning to offline regression with near-optimal regret.
problem Efficient learning with large action spaces and complex reward functions.
method Offline Estimation to Decisions (OE2D) algorithm that minimizes regret with near-optimal oracle calls.
result Near-optimal regret for contextual bandits with large action spaces and O ( l o g ( T ) ) O(log(T)) O ( l o g ( T )) offline oracle calls. This thesis analyzes MACL systems with low-regret learning algorithms for sequential decision making.
problem Designing efficient learning algorithms for multi-agent cooperative systems to minimize regret.
method Analyzes and develops algorithms for cooperative multi-agent multi-armed bandit problems and online convex optimization in distributed settings.
result Presented regret lower bounds and efficient algorithms for achieving these bounds, providing guidance on communication protocols.
This paper improves online learning algorithms for LP problems, achieving better regret bounds.
problem Achieving optimal regret bounds in online linear programming.
method Develops a new framework for first-order online learning algorithms under certain error bound conditions.
result First-order learning algorithms achieve o ( T ) o(\sqrt{T}) o ( T ) regret in continuous support and O ( log T ) \mathcal{O}(\log T) O ( log T ) regret in finite support, improving over O ( T ) \mathcal{O}(\sqrt{T}) O ( T ) . New algorithm DUSA minimizes regret in structured bandits with structural information.
problem Optimal decision-making under uncertainty with structural reward information.
method DUSA algorithm exploiting convex structural information.
result Regret matches information-theoretic lower bound up to a constant factor.
Improved analysis of UCRL2 with empirical Bernstein inequality reduces exploration-exploitation regret.
problem Exploration-exploitation in communicating Markov Decision Processes.
method Analysis of UCRL2 with Empirical Bernstein inequalities (UCRL2B).
result Regret bound of O ~ ( D Γ S A T ) \widetilde{O}(\sqrt{DΓS A T}) O ( D Γ S A T ) for UCRL2B. qEUBO optimizes decision-making with noisy feedback.
problem Optimizing decision-making with noisy preference feedback.
method Introduces qEUBO as a novel acquisition function for preferential Bayesian optimization.
result qEUBO is one-step Bayes optimal and enjoys an approximation guarantee under noise.
OTSS learns personalized decision weights from logged decisions and outputs.
problem Learning context-specific decision weights from logged decisions and outputs.
method Output-targeted soft-segmentation model that deploys personalized decision-ready weight vectors.
result OTSS achieves the lowest mean regret in benchmark settings.
FRONT optimizes decisions with interference, reducing regret over time.
problem Short-sighted policies in online decision-making due to ignoring interference.
method FRONT considers long-term impacts of decisions, using exploratory and exploitative strategies.
result FRONT achieves sublinear regret in both immediate and consequential impacts.
The paper optimizes regret using covariance between costs and decisions.
problem Optimizing expected regret in decision-making problems.
method Developed derivative theory of covariance regret functional, derived Gâteaux derivative, and extended to constrained optimization.
result Gradient of covariance regret is the cost covariance matrix, with implications for portfolio optimization.
Study shows how AI model can improve decision-making with missing data.
problem Sequential decision-making with missing covariates.
method Introduced model elasticity to quantify imputation discrepancy; used statistical learning and regression for calibration.
result Calibrating pre-trained models can significantly reduce regret in decision-making.
The paper tackles robust policy learning from multiple data sources.
problem Learning a policy that generalizes across diverse settings from multiple heterogeneous data sources.
method Proposes a minimax regret optimization objective and a policy learning algorithm combining doubly robust offline policy evaluation and no-regret learning.
result Achieves minimal worst-case mixture regret up to a moderated vanishing rate of the total data across all sources.
Improved FPL algorithms for adversarial MDPs with better regret bounds.
problem Adversarial rewards and unknown transitions in MDPs.
method Refined analysis of FPL algorithms, matching current best regret bounds.
result Improved regret bounds for FPL algorithms in adversarial MDPs.
The paper addresses frequentist regret of Linear Thompson Sampling in stochastic linear bandits.
problem The frequentist regret of Linear Thompson Sampling (LinTS) is worse than its Bayesian counterpart.
method The paper proves the fundamental nature of the frequentist regret bound for LinTS and proposes a data-driven version of LinTS to achieve minimax optimal frequentist regret.
result The frequentist regret bound for LinTS is O ~ ( d d T ) \widetilde{\mathcal{O}}(d\sqrt{dT}) O ( d d T ) , which is the best possible under certain conditions. Online learning has traditionally focused on the expected rewards. In this paper, a risk-averse online learning problem under the performance measure of the mean-variance of the rewards is studied. Both the bandit and full information settings are considered. The performance of several existing policies is analyzed, an…
Paper analyzes faster convergence rates for reinforcement learning from offline data.
problem Analyzing faster convergence rates for reinforcement learning from offline data.
method Fine analysis of reinforcement learning from offline data, providing fast rates for regret convergence.
result The paper provides fast rates for the regret convergence, showing that the level of exponentiation depends on the noise in the decision-making problem.
The paper studies how noisy labels impact decision-making in machine learning.
problem The impact of noisy labels on decision-making in machine learning.
method Introducing a notion of regret, studying standard approaches, and estimating individual-level mistakes.
result Standard approaches can lead to unforeseen mistakes for individuals, revealing the need for anticipation.
The paper addresses contextual optimization problems with feedback, aiming to minimize regret.
problem Contextual optimization with feedback information.
method Characterizing the optimal minimax policy in offline setting and leveraging geometric characterization in online setting to optimize cumulative regret.
result Developed an algorithm yielding logarithmic regret bound in the online setting.
Improved privacy in RL with near-optimal regret bounds.
problem Privacy-preserving reinforcement learning in personalized decision-making systems.
method Differentially private algorithm based on LSVI-UCB++ with privacy-preserving techniques.
result Achieved a near-optimal regret bound of O(d * sqrt(H^3 * K) + H^(15/4) * d^(7/6) * K^(1/2) / ε).
This paper improves Thompson Sampling for complex decision-making problems.
problem Learning in infinite-horizon discounted decision processes with unknown parameters.
method Developed a general canonical probability space and new metrics for analyzing adaptive learning algorithms.
result Thompson Sampling achieves complete learning in complex decision-making problems.
New algorithm reduces regret in private online learning with optimal gap-dependent rate.
problem Optimal gap-dependent regret rate for private stochastic decision-theoretic online learning.
method Horizon-free pure-DP algorithm with exponential block partitioning and softmax selection.
result Explicit regret bound of 1000 ⋅ ( log K Δ min + log K ε ) 1000 \cdot (\frac{\log K}{Δ_{\min}}+\frac{\log K}{\varepsilon}) 1000 ⋅ ( Δ m i n l o g K + ε l o g K ) . Logarithmic regret for continuous-time reinforcement learning.
problem Continuous-time Markov decision processes with unknown transition probabilities and holding times.
method Upper confidence reinforcement learning, mean holding time estimation, stochastic comparison of point processes.
result Logarithmic regret bound achieved in finite time.
UCRL-CMDP algorithm optimizes RL with constraints on average costs.
problem Optimizing RL in MDPs with average cost constraints.
method Model-based RL algorithms maximizing reward while keeping costs within bounds.
result UCRL-CMDP algorithm's expected regret is upper-bounded by $T^{2\slash 3}$ .
Study minimax optimal RL in factored MDPs with bonus exploration.
problem Optimal reinforcement learning in episodic factored MDPs.
method Proposes two model-based algorithms with bonus exploration for minimax optimal regret.
result Achieves minimax optimal regret guarantees for rich factored structures.
New rule reduces exploration regret to logarithmic, improving bad episode handling.
problem Improving exploration regret in average reward MDPs.
method Replacing Doubling Trick with Vanishing Multiplicative rule in EVI-based algorithms.
result Regret is logarithmic under the new rule, significantly better than linear.
Proposes a new method for decision-aware learning in optimization.
problem Contextual linear optimization with cost prediction errors.
method Reweighing prediction error by decision regret for decision-aware predictor.
result Improves over predict-then-optimize framework for misspecified models.
New algorithm achieves optimal regret in average reward MDPs without prior bias information.
problem Achieving optimal regret in average reward MDPs with computational efficiency and without prior bias information.
method Projective Mitigated Extended Value Iteration (PMEVI) to compute bias-constrained optimal policies efficiently.
result First tractable algorithm with minimax optimal regret of O ~ ( s p ( h ∗ ) S A T ) \widetilde{\mathrm{O}}(\sqrt{\mathrm{sp}(h^*) S A T}) O ( sp ( h ∗ ) S A T ) . Optimal strategy identified for minimizing regret in fixed-budget best arm selection.
problem Minimizing expected simple regret in fixed-budget best arm selection.
method Two-Stage (TS)-Hirano-Imbens-Ridder (HIR) strategy using HIR estimator.
result TS-HIR strategy is asymptotically minimax optimal.
New framework calibrates decision robustness using inverse conformal risk control.
problem Inadequate robustness levels in decision-making due to ad hoc choices.
method Constructs valid estimators to trace miscoverage-regret Pareto frontier.
result Provides distribution-free, finite-sample guarantees on robustness levels.