Algorithm minimizes regret and converges to equilibria in Markov games.
problem Regret minimization and convergence to equilibria in general-sum Markov games under adversarial opponents.
method Decentralized algorithm that uses policy optimization and controls path length to achieve sublinear regret.
result Sublinear regret guarantees for convergence to correlated equilibrium in Markov games.
A new metric, Weighted Regret, unifies FDR and power evaluation in online multiple testing.
problem The asymmetric costs of false positives and false negatives in automated pipelines.
method Introducing Weighted Regret and Decoupled-OMT (DOMT) to unify FDR and power evaluation.
result DOMT achieves an order-optimal sublinear mitigation of threshold depletion in bursty environments.
VO Q Q Q L optimizes RL with sparse rewards using weighted bounds.
problem Sparse rewards and non-linear function approximation in RL.
method VO Q Q Q L combines Q Q Q -learning with weighted bounds for optimal regret. result Achieves asymptotically optimal regret for linear function approximation.
This paper refines the weighted strategy for non-stationary parametric bandits and MDPs, improving regret bounds.
problem Non-stationary environments with gradual drifting patterns.
method Refined analysis framework for the weighted strategy, leading to simpler and more efficient algorithms.
result Improved regret bounds for linear bandits, generalized linear bandits, and self-concordant bandits.
New aggregation strategy handles unbounded losses with regret bounds.
problem Online optimization with unbounded loss functions.
method Follow The Regularized Leader (FTRL) with φ-divergence.
result Worst regret bound for unbounded losses with alternative divergences.
This paper refines the weighted strategy for non-stationary parametric bandits, improving regret bounds.
problem Non-stationary environments with gradual drifting patterns.
method Refined analysis framework for the weighted strategy in linear and generalized linear bandits.
result A simpler weight-based algorithm with improved regret bounds compared to previous studies.
New algorithm reduces regret in RL with adversarial corruption.
problem Adversarial corruption in reinforcement learning.
method Uncertainty-weighted least-squares regression and weighted uncertainty estimator.
result Achieves regret of i l d e O ( T + ζ ) ilde{O}(\sqrt{T} + ζ) i l d e O ( T + ζ ) for contextual bandits. Algorithm learns expert weights to minimize regret in adversarial setting.
problem Learning to aggregate expert forecasts with no-regret guarantee in adversarial conditions.
method Online mirror descent algorithm for logarithmic pooling of expert forecasts.
result Achieves O ( T log T ) O(\sqrt{T} \log T) O ( T log T ) expected regret compared to best weights. Mirror descent with an entropic regularizer is known to achieve shifting regret bounds that are logarithmic in the dimension. This is done using either a carefully designed projection or by a weight sharing technique. Via a novel unified analysis, we show that these two approaches deliver essentially equivalent bounds …
New algorithm reduces regret in combinatorial causal bandits without graph structure.
problem Minimizing regret in combinatorial causal bandits without graph structure.
method Design of algorithms for binary general causal models and BGLMs without graph skeleton.
result Achieves O ( T ln T ) O(\sqrt{T}\ln T) O ( T ln T ) expected regret for causal models and O ( T 2 3 ln T ) O(T^{\frac{2}{3}}\ln T) O ( T 3 2 ln T ) for BGLMs. We consider a setting where an agent's uncertainty is represented by a set of probability measures, rather than a single measure. Measure-bymeasure updating of such a set of measures upon acquiring new information is well-known to suffer from problems; agents are not always able to learn appropriately. To deal with the…
New algorithm reduces dynamic regret in non-stationary dueling bandits using a weighted Borda score.
problem Designing algorithms with low dynamic regret in non-stationary dueling bandits.
method Introducing a novel weighted Borda score framework to analyze the Condorcet problem and establish improved bounds.
result First optimal and adaptive dynamic regret upper bound of i l d e O ( i l d e L 1 / 3 K 1 / 3 T 2 / 3 ) ilde{O}( ilde{L}^{1/3} K^{1/3} T^{2/3} ) i l d e O ( i l d e L 1/3 K 1/3 T 2/3 ) . Optimizes matching in weighted graphs with semi-bandit sampling.
problem Finding optimal pairings in weighted graphs with sequential sampling.
method Leverages rank-1 assumption on adjacency matrix to reduce sample complexity and regret.
result Achieves linear dependency in the number of vertices for sample complexity and regret.
OTSS learns personalized decision weights from logged decisions and outputs.
problem Learning context-specific decision weights from logged decisions and outputs.
method Output-targeted soft-segmentation model that deploys personalized decision-ready weight vectors.
result OTSS achieves the lowest mean regret in benchmark settings.
We investigate the use of bootstrapping in the bandit setting. We first show that the commonly used non-parametric bootstrapping (NPB) procedure can be provably inefficient and establish a near-linear lower bound on the regret incurred by it under the bandit model with Bernoulli rewards. We show that NPB with an approp…
Decentralized algorithm reduces regret and converges to Nash equilibrium in online congestion games.
problem Online congestion games with exponential action sets and strict Nash equilibria.
method CongestEXP algorithm using exponential weights method.
result CongestEXP achieves O ( k F T ) O(kF\sqrt{T}) O ( k F T ) regret bound and almost exponential convergence to strict Nash equilibrium. Optimistic algorithm reduces regret in non-stationary linear MDPs.
problem Efficient learning in non-stationary linear MDPs with evolving reward and transition.
method OPT-WLSVI, an optimistic model-free algorithm using exponential weights.
result Achieves a regret bound of O ~ ( d 5 / 4 H 2 Δ 1 / 4 K 3 / 4 ) \widetilde{\mathcal{O}}(d^{5/4}H^2 Δ^{1/4} K^{3/4}) O ( d 5/4 H 2 Δ 1/4 K 3/4 ) . Decentralized optimization on dynamic manifolds with improved regret bound.
problem Optimizing on nonstationary Riemannian manifolds in decentralized systems.
method Decentralized projected Riemannian gradient descent with weighted Frechet mean consensus.
result Achieved dynamic regret bound of O ( T ( 1 + P T ) / ( 1 − σ 2 ( W ) ) ) {\cal O}(\sqrt{T(1+P_T)}/\sqrt{(1-σ_2(W))}) O ( T ( 1 + P T ) / ( 1 − σ 2 ( W )) ) . Improved algorithm reduces regret in corrupted expert advice setting.
problem Prediction with expert advice in the presence of adversarial corruption.
method Multiplicative Weights algorithm with decreasing step sizes.
result Achieves constant regret and optimal performance in various environments.
We study the Thompson sampling algorithm in an adversarial setting, specifically, for adversarial bit prediction. We characterize the bit sequences with the smallest and largest expected regret. Among sequences of length T T T with k < T 2 k < \frac{T}{2} k < 2 T zeros, the sequences of largest regret consist of alternating zeros and …
A UCB algorithm reduces regret in cooperative multi-agent graph bandits.
problem Cooperative multi-agent decision-making on a graph with shared rewards.
method Upper Confidence Bound (UCB) algorithm for minimizing regret.
result The Multi-G-UCB algorithm achieves expected regret O ( γ N log ( T ) [ K T + D K ] ) O(γN\log(T)[\sqrt{KT} + DK]) O ( γ N log ( T ) [ K T + D K ]) . Paper addresses regret minimization and inference in high-dimensional online decision-making.
problem Regret minimization and statistical inference in high-dimensional online decision-making.
method Integrates ε-greedy bandit algorithm with hard thresholding for sparse bandit parameters and debiasing method for inference.
result Achieves either O ( T 1 / 2 ) O(T^{1/2}) O ( T 1/2 ) regret or O ( T 1 / 2 ) O(T^{1/2}) O ( T 1/2 ) -consistent inference, with trade-off between exploration and exploitation. Capacity-Constrained Online Convex Optimization with Delayed Feedback
problem Online learning with delayed feedback under a hard capacity constraint
method Reduction to a delayed and weighted OCO problem using a scheduler
result First regret guarantees for capacity-constrained OCO under convex and strongly convex losses
We present a generalization of the adversarial linear bandits framework, where the underlying losses are kernel functions (with an associated reproducing kernel Hilbert space) rather than linear functions. We study a version of the exponential weights algorithm and bound its regret in this setting. Under conditions on …
New algorithm reduces linear contextual bandit regret with adversarial corruption.
problem Linear contextual bandit with adversarial reward corruption.
method Optimism in the face of uncertainty principle, weighted ridge regression.
result Achieves nearly optimal regret for both corrupted and uncorrupted cases.
New algorithm for online learning with noisy side observations.
problem Online learning with noisy side feedback and graph-structured dependencies.
method Proposes an algorithm using a weighted directed graph to model dependencies and guarantees a regret bound of O(√α* T).
result Guarantees a regret of O(√α* T) after T rounds, where α* is the effective independence number.
UCRL-WVTR tackles long-term reinforcement learning with general approximations, achieving horizon-free and instance-dependent regret bounds.
problem Long-term reinforcement learning with general function approximations.
method UCRL-WVTR proposes a novel algorithm, UCRL-WVTR, with weighted value-targeted regression and a high-order moment estimator.
result Achieves horizon-free and instance-dependent regret bounds matching minimax lower bounds up to logarithmic factors.
Meta-strategy learns tuning parameters for online learning methods.
problem Difficulty in setting tuning parameters for online learning methods.
method Meta-learning approach to learn parameters from past tasks.
result Meta-strategy improves on learning each task in isolation.
Efficient RL for linear MDPs with unknown transitions.
problem Long planning horizons and unknown state transitions in linear mixture MDPs.
method Horizon-free algorithm using weighted least squares with variance and uncertainty awareness.
result Achieves optimal regret up to logarithmic factors.
We consider a variation on the problem of prediction with expert advice, where new forecasters that were unknown until then may appear at each round. As often in prediction with expert advice, designing an algorithm that achieves near-optimal regret guarantees is straightforward, using aggregation of experts. However, …
New approach optimizes policies in adversarial MDPs using adversarial learning.
problem Optimizing policies in adversarial Markov decision processes.
method Adversarial learning on advantage functions, extending previous reductions.
result Stronger regret criteria and performance guarantees for policy optimization.
Paper studies multiclass classifiers from binary classifiers, proving methods and demonstrating advantages.
problem Constructing efficient multiclass classifiers from binary ones.
method Two methods: one vs. all and hierarchical classification, with a new leverage-hierarchical method introduced.
result Proves upper bounds and exact formulas for multiclass regret in terms of binary regrets.
New algorithms minimize simple and cumulative regret in contextual bandits.
problem Minimizing simple and cumulative regret in contextual bandit settings.
method Proposed new algorithms using conformal arm sets (CASs).
result Near-optimal minimax guarantees for simple regret and state-of-the-art guarantees for cumulative regret.
New algorithm reduces bandit regret to log^3(T).
problem Noise model for linear stochastic bandits with vanishing noise.
method Weighted least-squares estimation, leveraging eigenvalue relation.
result Minimax regret scaling as log^3(T) for time horizon T.
End-to-end framework learns LLM routing from observational data.
problem Compounding errors in decoupled approaches and reliance on full-feedback data.
method Causal end-to-end framework minimizing decision-making regret from observational data.
result Method outperforms existing baselines across different embedding models.
Improved MMWU algorithm achieves instance-optimal regret bound for matrix LEA.
problem Matrix Learning from Expert Advice problem.
method Developed a general potential-based framework for matrix LEA, using a new Jensen's trace inequality.
result Achieved instance-optimal regret bound of O ( T ⋅ S ( X ∣ ∣ d − 1 I d ) ) O(\sqrt{T\cdot S(X||d^{-1}I_d)}) O ( T ⋅ S ( X ∣∣ d − 1 I d ) ) . New algorithms handle online prediction with bandit and delayed feedback, improving regret bounds.
problem Achieving finite bounds on surrogate regret with limited feedback.
method Proposed algorithms for bandit and delayed feedback, including inverse-weighted gradient and pseudo-inverse matrix estimators.
result Achieved improved surrogate regret bounds of O ( K T ) O(\sqrt{KT}) O ( K T ) and O ( T 2 / 3 ) O(T^{2/3}) O ( T 2/3 ) . New algorithm reduces online logistic regression regret without exponential constant.
problem Improper learning in online logistic regression with logarithmic regret.
method Regularized empirical risk minimization with surrogate losses.
result Regret scaling as O(B log(Bn)) with low computational complexity.
We consider the online version of the isotonic regression problem. Given a set of linearly ordered points (e.g., on the real line), the learner must predict labels sequentially at adversarially chosen positions and is evaluated by her total squared loss compared against the best isotonic (non-decreasing) function in hi…
New algorithms reduce label collection for online prediction with expert advice.
problem Efficiently predicting binary sequences with expert advice using fewer labels.
method Adaptive selective sampling for exponentially weighted forecasters.
result Label complexity scales roughly as the square root of the number of rounds for a scenario with a strictly better expert.
Optimal algorithm found for collaborative learning in bandits with optimal regret bounds.
problem Minimizing regret in collaborative multi-agent bandit problems.
method Proposed an algorithm with optimal regret bounds for collaborative multi-agent multi-armed bandit model.
result First algorithm with order optimal regret bounds for collaborative bandit model.
Bayesian optimization method for permutations accelerates combinatorial search.
problem Optimizing expensive-to-evaluate objectives on permutation problems.
method LAW2ORDER, a batch Bayesian optimization method based on the acquisition weighted kernel.
result LAW2ORDER achieves sublinear batch cumulative regret, demonstrating accelerated search.
Adapting policy learning for data collected from evolving systems.
problem Challenges in learning optimal policies from adaptively collected data.
method Proposes an algorithm based on generalized augmented inverse propensity weighted (AIPW) estimators to control worst-case estimation variance.
result Achieves minimax rate optimal regret guarantees even with diminishing exploration.
New lower bounds for combinatorial multi-armed bandits for general reward functions.
problem Maximizing reward in sequential decisions with sets of arms.
method Proved tight regret lower bounds for all smooth reward functions under mild assumptions.
result Lower bounds are tight up to log-factors for monotone reward functions.
Kernel ε ε ε -Greedy optimizes multi-armed bandits with covariates for sub-linear regret.
problem Optimizing multi-armed bandits with covariates in a reproducing kernel Hilbert space.
method Online weighted kernel ridge regression estimator for mean reward function estimation.
result Achieves sub-linear regret rate and optimal T \sqrt{T} T regret rate under margin condition. Proposes MRO to achieve uniformly low regret in distributionally robust learning.
problem Learning under unknown test distributions (distribution shift).
method Minimax Regret Optimization (MRO) for robust machine learning.
result MRO achieves uniformly low regret across all test distributions.
New algorithm reduces expert prediction regret for two experts.
problem Efficient prediction with two experts under fixed time constraints.
method Optimal algorithm based on stochastic calculus techniques.
result Achieves optimal regret of sqrt(T/2π) + O(1) with O(1) per-turn processing time.
New method minimizes regret in AMDP with high probability.
problem Pessimistic regret analysis in AMDP.
method Developed a new policy search method achieving optimistic regret.
result Achieved sublinear optimistic regret with high probability.