New approach uses observational data to learn policies with constraints.
problem Learning policies with application-specific constraints from observational data.
method Semiparametrically efficient estimation and algorithm for choosing whom to treat.
result Strong guarantees for the asymptotic utilitarian regret of the resulting policy.
DP-NCB algorithm ensures privacy and fairness in bandit decisions.
problem Achieving both privacy and fairness in bandit algorithms.
method Differentially Private Nash Confidence Bound (DP-NCB) framework.
result DP-NCB achieves optimal Nash regret while maintaining privacy.
Optimizes long-term social welfare in recommender systems by matching users to providers.
problem Realistic recommender systems dynamics affect all agents, not just users.
method Formulated as an optimal constrained matching problem, solved using dynamical system equilibrium selection.
result Ensures maximal social welfare with diverse viable providers, improving over myopic matching.
New framework shifts bandit algorithms from expected reward to preference metrics, optimizing mixtures of arms.
problem Traditional bandit algorithms focus on expected rewards, ignoring variability and risk.
method Introduces preference metrics (PMs) and designs algorithms to optimize mixtures of arms.
result Optimal policy selects mixtures of arms based on specific weights, not a single best arm.
New algorithm reduces unfairness in bandit problems by balancing exploration and exploitation.
problem Fairness in bandit problems where early participants can be unfairly disadvantaged.
method Introduces extsf{UCB-HARE} algorithm that balances exploration and exploitation using inverse-weighted harmonic rank schedule.
result Algorithm extsf{UCB-HARE} achieves regret matching the lower bound Ω ( σ k max ( 1 , q ) / T ) Ω(σ\sqrt{k^{\max(1,q)}/T}) Ω ( σ k m a x ( 1 , q ) / T ) for q > 1 q>1 q > 1 . This study examines how fashion consumption affects self-confidence and buying behavior in Iranian consumers.
problem Understanding the role of self-confidence in fashion buying behavior.
method A questionnaire was used to collect data from 400 consumers in Tehran's clothing market. Structural equations and factor analysis were employed to test the model.
result Interest in fashion, personal taste, utilitarianism, and new products positively impact self-confidence, and self-confidence positively impacts fashion buying behavior.
The paper proposes a new policy for optimal treatment allocation based on quantile treatment effects.
problem Optimal treatment allocation policies that target distributional welfare, especially when individuals are heterogeneous.
method The approach involves allocating treatments based on the conditional quantile of individual treatment effects (QoTE), considering both prudent and negligent policymakers.
result The proposed minimax policies are robust to model uncertainty and can be generalized to various settings.
This work analyzes how users and services adapt to reduce risk, leading to specialization.
problem Adaptation of users and services to reduce risk affects learning and performance.
method Analyzed a class of dynamics where users allocate participation and services update parameters.
result Repeated myopic updates with multiple learners lead to better outcomes than repeated risk minimization.
Study tackles RLHF with diverse human feedback, showing limitations and proposing a meta-learning approach.
problem Traditional RLHF fails to balance diverse human preferences.
method Integrates meta-learning and multiple social welfare functions to optimize diverse preferences.
result Establishes sample complexity bounds for optimizing diverse social welfare functions.
Paper proposes algorithms to minimize both dynamic and adaptive regret simultaneously.
problem Traditional regret minimization algorithms are suboptimal for changing environments.
method Developed novel online algorithms to minimize dynamic and adaptive regret simultaneously.
result Proposed algorithms minimize dynamic and adaptive regret over any interval.
New measure of policy regret shows compatibility with traditional external regret in adversarial games.
problem Incompatibility between traditional and new policy regret measures in adaptive adversaries.
method Revisited policy regret and compared it with external regret; introduced policy equilibrium.
result Policy regret and external regret are compatible in adversarial games.
The paper analyzes the sliding regret of stochastic bandit algorithms.
problem Measuring the one-shot behavior of no-regret algorithms in stochastic bandits.
method Introducing sliding regret to measure the worst pseudo-regret over a time-window.
result Randomized methods have optimal sliding regret, while index policies have the worst possible sliding regret.
This paper analyzes regret bounds for Gaussian process Thompson sampling.
problem Analyzing the performance of Gaussian process Thompson sampling (GP-TS) in Bayesian optimization.
method The paper derives several regret bounds for GP-TS, including a lower bound, upper bounds on the second moment of cumulative regret, expected lenient regret, and improved cumulative regret.
result The paper provides improved regret upper bounds for GP-TS, showing that it suffers from a polynomial dependence on 1 / δ 1/δ 1/ δ with probability δ δ δ . Efficiently minimizes regret in non-convex games with gradient-based methods.
problem Computational intractability of standard regret minimization in non-convex games.
method Defining a new notion of regret and using gradient-based optimization methods.
result Achieves optimal regret, leading to convergence to equilibrium.
New definition of regret for nonconvex online learning models.
problem Intractability of standard regret measures for nonconvex models.
method Introduced a local gradient based regret definition.
result Our definition provides more interpretable bounds for forecasting.
Online learning algorithms are designed to learn even when their input is generated by an adversary. The widely-accepted formal definition of an online algorithm's ability to learn is the game-theoretic notion of regret. We argue that the standard definition of regret becomes inadequate if the adversary is allowed to a…
Balances the regret of different algorithms in bandit and RL problems.
problem Model selection in bandit and reinforcement learning.
method Estimates and balances the empirical regrets of algorithms.
result Achieves near-optimal regret compared to the optimal base algorithm.
Paper studies how to combine regret minimizers for solving complex games.
problem Solving large-scale extensive-form games with constraints.
method Derives a calculus for constructing regret minimizers for composite convex sets.
result Local regret minimizers for simpler sets can be combined into an aggregate for composite sets.
Study Thompson Sampling in adversarial bit prediction, finding regret bounds and optimal sequences.
problem Adversarial bit prediction with varying error weights.
method Thompson Sampling, analyzing sequences with largest and smallest regret.
result Regret bounds for adversarial bit prediction sequences, including optimal and worst-case scenarios.
Data-driven model selection reduces regret in sequential decisions.
problem Optimizing model selection in stochastic environments with bandit feedback.
method Data-driven regret balancing for model selection.
result Meta-learner selects the best base learner based on actual realized regret.
Optimistic Hedge achieves optimal regret bounds in two-player zero-sum games.
problem Achieving optimal regret bounds for optimistic Hedge in two-player zero-sum games.
method Refined regret analysis and optimization problem formulation.
result Optimistic Hedge achieves O ( log m log n ) O(\sqrt{\log m \log n}) O ( log m log n ) regret bounds, matching upper and lower bounds. New algorithm reduces regret in contextual bandits.
problem Minimizing regret in contextual bandits with side information.
method Contextual-Gap algorithm for simple regret minimization.
result Established performance guarantees on simple regret.
Optimal switching regret for all segmentations in online convex optimisation.
problem Non-stationary online convex optimisation problems.
method Developed an efficient algorithm to achieve optimal switching regret on every possible segmentation.
result Achieved asymptotically optimal switching regret on every possible segmentation simultaneously.
New approach for distributed online optimization of non-convex losses with sublinear regret.
problem Regret evaluation and consensus in distributed, multi-agent systems with non-convex losses.
method Composite regret metric and consensus-based online normalized gradient (CONGD) approach for pseudo-convex losses; offline optimization oracle for general non-convex losses.
result First sublinear regret bound for general distributed online non-convex learning.
New algorithm SELECT minimizes satisficing regret in bandits.
problem Minimizing regret in bandit optimization with satisficing arms.
method SELECT algorithm for satisficing regret minimization.
result SELECT achieves constant expected satisficing regret.
The paper tackles efficient online learning by achieving minimal regret with respect to the best expert.
problem Achieving minimal regret in online learning problems where the goal is to match the lowest regret of K experts.
method A lazy form of the online subgradient algorithm is used to achieve minimal regret in 'easy' regimes.
result Minimal regret strategies exist for some 'hard' regimes, and the algorithm retains an O ( n ) O(\sqrt{n}) O ( n ) worst-case regret guarantee. Proximal online gradient minimizes dynamic regret in evolving environments.
problem Optimizing dynamic regret in online learning where the optimal solution changes over time.
method Proximal online gradient method, showing it is optimal for dynamic regret.
result Proximal online gradient matches the lower bound for dynamic regret, proving its optimality.
This paper considers the stability of online learning algorithms and its implications for learnability (bounded regret). We introduce a novel quantity called {\em forward regret} that intuitively measures how good an online learning algorithm is if it is allowed a one-step look-ahead into the future. We show that given…
Paper introduces a new G ⋆ G^\star G ⋆ regret measure for online convex optimization with smooth losses.
problem Online convex optimization with smooth losses.
method Introduces a new G ⋆ G^\star G ⋆ regret measure that depends on the cumulative squared gradient norm. result The G ⋆ G^\star G ⋆ regret can be arbitrarily sharper than existing measures when losses have vanishing curvature. Optimistic algorithms achieve logarithmic regret bounds for MDPs without diameter dependence.
problem Achieving logarithmic regret bounds for episodic MDPs without relying on diameter-like quantities.
method Novel 'clipped' regret decomposition applied to optimistic algorithms.
result Smooth interpolation between gap-dependent and minimax rates of convergence.
Paper improves adaptive regret for convex and smooth functions.
problem Online convex optimization in changing environments.
method Develops adaptive algorithms exploiting both convexity and smoothness.
result Regret bounds are comparable to worst-case results but tighter when comparators have small losses.
Bandit algorithms struggle with consistent performance and robustness.
problem Achieving consistent and robust performance in stochastic multi-armed bandit settings.
method Analyzing regret minimization trade-offs and proposing distribution-oblivious algorithms.
result Logarithmic regret is inconsistent and super-logarithmic regret is necessary for consistent learning.
New minimax theorem connects Bayesian and minimax regret in partial monitoring.
problem Minimax regret in partial monitoring with no assumptions on adversary.
method Information-theoretic tools and minimax theorem.
result Clean analysis of easy and hard finite partial monitoring with new bounds.
Paper improves worst-case regret bounds for RLSVI in reinforcement learning.
problem Minimizing regret in reinforcement learning with randomized value functions.
method Introduces a clipping variant of Thompson Sampling for RLSVI.
result Achieves a i l d e O ( H 2 S A T ) ilde{\mathrm{O}}(H^2S\sqrt{AT}) i l d e O ( H 2 S A T ) worst-case regret bound. Paper solves no-swap regret minimization for combinatorial bandits with polylogarithmic dependence on N.
problem Design efficient no-swap regret algorithms for combinatorial bandits with exponentially large action space.
method Introduces a no-swap-regret learning algorithm with polylogarithmic dependence on N and demonstrates efficient implementation.
result Achieves no-swap regret with polylogarithmic dependence on N, resolving an open problem.
New algorithms reduce dynamic regret in non-stationary RL environments.
problem Optimizing policies in environments that change over time.
method POWER and POWER++ algorithms for policy optimization with dynamic regret analysis.
result POWER++ improves dynamic regret by actively adapting to non-stationarity.
New algorithms minimize simple and cumulative regret in contextual bandits.
problem Minimizing simple and cumulative regret in contextual bandit settings.
method Proposed new algorithms using conformal arm sets (CASs).
result Near-optimal minimax guarantees for simple regret and state-of-the-art guarantees for cumulative regret.
New framework reduces minimax regret for high-dimensional data.
problem Minimizing regret in high-dimensional data with logarithmic loss.
method Developed envelope complexity framework and spike-and-tails prior.
result Achieves minimax regret within a factor of two over high-dimensional ℓ 1 \ell_1 ℓ 1 -balls. Paper explores rate-preserving reductions between Blackwell approachability and no-regret learning.
problem Tackles rate-preserving reductions between Blackwell approachability and no-regret learning.
method Studies fine-grained reductions and optimal rates of convergence.
result Shows that rate-preserving reductions do not always hold, but provides conditions for when they do.
Algorithm minimizes regret and converges to equilibria in Markov games.
problem Regret minimization and convergence to equilibria in general-sum Markov games under adversarial opponents.
method Decentralized algorithm that uses policy optimization and controls path length to achieve sublinear regret.
result Sublinear regret guarantees for convergence to correlated equilibrium in Markov games.
New algorithm achieves both static and dynamic regret optimally against an oblivious adversary for deterministic losses.
problem Achieving optimal static and dynamic regret simultaneously in adversarial bandits.
method Extends impossibility result to deterministic losses, uses negative static regret and Blackwell approachability.
result First algorithm achieving optimal static and dynamic regret simultaneously against an oblivious adversary.
Algorithm reduces regret in partially observable systems by learning dynamics and using optimistic control.
problem Minimizing regret in partially observable linear quadratic control systems with unknown dynamics.
method ExpCommit algorithm that learns model parameters and uses optimism in uncertainty.
result End-to-end sublinear regret upper bound of O ~ ( T 2 / 3 ) \tilde{\mathcal{O}}(T^{2/3}) O ~ ( T 2/3 ) for ExpCommit. New algorithm reduces regret from sqrt(T) to polylog(T) in stochastic contextual linear bandits.
problem Achieving logarithmic regret in stochastic contextual linear bandits.
method Low Regret Stochastic Contextual Bandits ( exttt{LR-SCB}) algorithm, exploiting stochastic contexts and parameter estimation.
result Logarithmic regret (polylog(T)) achieved, improving over sqrt(T) lower bound.
The paper explores dynamic regret with switching cost in online decision making.
problem The relation between dynamic regret and switching cost in online decision making.
method Investigates two classic online settings: Online Algorithms (OA) and Online Convex Optimization (OCO). Provides a new theoretical analysis framework.
result The switching cost impacts dynamic regret differently in OA and has no impact in OCO.
Optimal control methods achieve significantly smaller regret than previously thought.
problem Optimal control in linear dynamical systems with adversarial changes.
method Online gradient descent and online natural gradient methods.
result Achieves logarithmic regret scaling as O(poly(log T)) instead of O(sqrt(T)).
New algorithm reduces online learning regret in uninformed Markov games.
problem Achieving no external regret in uninformed Markov games is impossible.
method Empirical Nash-value regret, parameter-free algorithm, adaptive restart.
result Achieves O ( min { K + ( C K ) 1 / 3 , L K } ) O(\min \{\sqrt{K} + (CK)^{1/3},\sqrt{LK}\}) O ( min { K + ( C K ) 1/3 , L K }) regret bound. Study on individual regret in cooperative MAB with agents communicating over a graph.
problem Individual regret in cooperative stochastic multi-armed bandits with communication constraints.
method Analyzed COOP-SE algorithm, derived individual regret bounds under various communication constraints.
result First to show an individual regret bound in cooperative stochastic MAB independent of graph diameter.
New algorithm limits regret in changing MDPs.
problem Reinforcement learning in MDPs with time-varying rewards and transitions.
method Proposed an algorithm with performance guarantees for non-stationary policies.
result First variational regret bound for general RL setting.