New algorithm achieves both static and dynamic regret optimally against an oblivious adversary for deterministic losses.
problem Achieving optimal static and dynamic regret simultaneously in adversarial bandits.
method Extends impossibility result to deterministic losses, uses negative static regret and Blackwell approachability.
result First algorithm achieving optimal static and dynamic regret simultaneously against an oblivious adversary.
Reduces dynamic regret to static problem in RKHS.
problem Minimizing cumulative loss in online convex optimization.
method Reduces dynamic regret to static regret problem in RKHS.
result Optimal dynamic regret guarantees for linear losses and new bounds for exp-concave and improper linear regression.
Recursive least-squares algorithms often use forgetting factors as a heuristic to adapt to non-stationary data streams. The first contribution of this paper rigorously characterizes the effect of forgetting factors for a class of online Newton algorithms. For exp-concave and strongly convex objectives, the algorithms a…
Optimizes nonconvex optimization by converting it to static regret minimization.
problem Nonconvex optimization challenges in machine learning.
method Black-box online-to-nonconvex conversion with static regret minimization oracles.
result Achieves optimal convergence rates for nonconvex optimization.
In online learning, the dynamic regret metric chooses the reference (optimal) solution that may change over time, while the typical (static) regret metric assumes the reference solution to be constant over the whole time horizon. The dynamic regret metric is particularly interesting for applications such as online reco…
Dynamic regret minimization is shown equivalent to static regret minimization for linear losses.
problem Dynamic regret minimization in online convex optimization.
method Equivalence between dynamic and static regret minimization for linear losses.
result Dynamic regret minimization is equivalent to static regret minimization for linear losses.
We propose algorithms for online principal component analysis (PCA) and variance minimization for adaptive settings. Previous literature has focused on upper bounding the static adversarial regret, whose comparator is the optimal fixed action in hindsight. However, static regret is not an appropriate metric when the un…
Optimal switching regret for all segmentations in online convex optimisation.
problem Non-stationary online convex optimisation problems.
method Developed an efficient algorithm to achieve optimal switching regret on every possible segmentation.
result Achieved asymptotically optimal switching regret on every possible segmentation simultaneously.
The paper tackles minimax optimality in continuum contextual bandits with Hölder continuity.
problem Minimizing regret in a continuum of contexts with Hölder continuity.
method Proves a static-to-contextual regret conversion theorem and analyzes various dependency cases.
result Achieves minimax optimal contextual regret for convex and strongly convex bandits.
Dynamic pricing improves DeFi lending efficiency by reducing regret to logarithmic levels.
problem Static pricing mechanisms in DeFi lending protocols lead to suboptimal welfare and revenue.
method Online learning model for static and dynamic pricing models in DeFi lending.
result Adaptive supply models achieve logarithmic regret, outperforming static models.
New algorithms reduce dynamic regret in non-stationary RL environments.
problem Optimizing policies in environments that change over time.
method POWER and POWER++ algorithms for policy optimization with dynamic regret analysis.
result POWER++ improves dynamic regret by actively adapting to non-stationarity.
New algorithm reduces constraint violation to O ( T 1 / 3 ) O(T^{1/3}) O ( T 1/3 ) while maintaining O ( T ) O(\sqrt{T}) O ( T ) regret.
problem Minimizing static regret and cumulative constraint violation in constrained online convex optimization.
method Proposes an algorithm that achieves O ( T ) O(\sqrt{T}) O ( T ) regret and O ( T 1 / 3 ) O(T^{1/3}) O ( T 1/3 ) cumulative constraint violation. result Shows that O ( T 1 / 3 ) O(T^{1/3}) O ( T 1/3 ) cumulative constraint violation is achievable with O ( T ) O(\sqrt{T}) O ( T ) regret. Online learning is a powerful tool for analyzing iterative algorithms. However, the classic adversarial setup sometimes fails to capture certain regularity in online problems in practice. Motivated by this, we establish a new setup, called Continuous Online Learning (COL), where the gradient of online loss function cha…
To deal with changing environments, a new performance measure -- adaptive regret, defined as the maximum static regret over any interval, was proposed in online learning. Under the setting of online convex optimization, several algorithms have been successfully developed to minimize the adaptive regret. However, existi…
Near-logarithmic regret per switch achieved for mixable/exp-concave losses.
problem Online optimization of mixable loss functions with dynamic environments.
method Online mixture framework using static solvers and hyper-expert creations.
result Near-logarithmic regret per switch with sub-polynomial complexity.
New approach reduces unconstrained linear bandits to simpler optimization problems.
problem Unconstrained linear bandits problem.
method Perturbation-based approach combined with comparator-adaptive OLO algorithms.
result First high-probability guarantees for both static and dynamic regret in unconstrained linear bandits.
We consider a multi-armed bandit problem in a setting where each arm produces a noisy reward realization which depends on an observable random covariate. As opposed to the traditional static multi-armed bandit problem, this setting allows for dynamically changing rewards that better describe applications where side inf…
Paper proposes algorithms to minimize both dynamic and adaptive regret simultaneously.
problem Traditional regret minimization algorithms are suboptimal for changing environments.
method Developed novel online algorithms to minimize dynamic and adaptive regret simultaneously.
result Proposed algorithms minimize dynamic and adaptive regret over any interval.
We study stochastic multi-armed bandits with many players. The players do not know the number of players, cannot communicate with each other and if multiple players select a common arm they collide and none of them receive any reward. We consider the static scenario, where the number of players remains fixed, and the d…
This paper considers distributed online optimization with time-varying coupled inequality constraints. The global objective function is composed of local convex cost and regularization functions and the coupled constraint function is the sum of local convex functions. A distributed online primal-dual dynamic mirror des…
Improved regret bounds for online convex optimization under stochastic and adversarial settings.
problem Interpolating between stochastic and adversarial online convex optimization.
method Optimistic online mirror descent (OMD) for the Stochastically Extended Adversarial (SEA) model.
result Established new regret bounds for various function classes.
In this paper, we consider the problem of prediction with expert advice in dynamic environments. We choose tracking regret as the performance metric and develop two adaptive and efficient algorithms with data-dependent tracking regret bounds. The first algorithm achieves a second-order tracking regret bound, which impr…
New analysis shows how temporal variability affects online learning performance.
problem Understanding the impact of temporal variability on online learning performance.
method Careful regret analysis and adaptive algorithm development.
result Proved a novel static regret bound that depends on temporal variability.
New algorithm learns and unlearns from streaming data efficiently.
problem Continuous learning and unlearning from production data streams.
method Translated batch unlearning techniques to online setting using regret, sample complexity, and deletion capacity.
result Achieved logarithmic regret bound of O ( ln T ) \mathcal{O}(\ln{T}) O ( ln T ) for online unlearning. New method tackles online DR-submodular maximization with improved regret guarantees.
problem Online maximization of non-monotone DR-submodular functions over down-closed convex sets.
method 1/e-linearization through exponential reparametrization, surrogate potential, and reduction to online linear optimization.
result Achieves O ( T 1 / 2 ) O(T^{1/2}) O ( T 1/2 ) static regret with single gradient query per round, improving state of the art. Two algorithms for linear contextual bandits with rare updates achieve optimal regret and efficiency.
problem Linear contextual bandits with infrequent parameter updates.
method Two practical algorithms with O ( log log T ) O(\log\log T) O ( log log T ) updates, BLCE-G and BLCE. result Minimax-optimal regret with low computational complexity.
This paper tackles near-optimal adversarial RL with switching costs, providing algorithms and matching lower bounds.
problem Adversarial RL with switching costs, where loss distribution can be non-stationary or adversarial.
method Developed novel switching-reduced algorithms with matching lower bounds for known and unknown transition functions.
result Achieved near-optimal performance in adversarial RL with switching costs, matching theoretical lower bounds.
New algorithms reduce dynamic regret in online MDPs with changing losses.
problem Online MDPs with adversarial loss changes and known transitions.
method Dynamic regret measure, novel ensemble algorithms for three models.
result Provably optimal dynamic regret bounds for episodic SSP, improved bounds for predictable environments.
New approach for distributed online optimization of non-convex losses with sublinear regret.
problem Regret evaluation and consensus in distributed, multi-agent systems with non-convex losses.
method Composite regret metric and consensus-based online normalized gradient (CONGD) approach for pseudo-convex losses; offline optimization oracle for general non-convex losses.
result First sublinear regret bound for general distributed online non-convex learning.
Causality graphs are routinely estimated in social sciences, natural sciences, and engineering due to their capacity to efficiently represent the spatiotemporal structure of multivariate data sets in a format amenable for human interpretation, forecasting, and anomaly detection. A popular approach to mathematically for…
Online learning is a powerful tool for analyzing iterative algorithms. However, the classic adversarial setup sometimes fails to capture certain regularity in online problems in practice. Motivated by this, we establish a new setup, called Continuous Online Learning (COL), where the gradient of online loss function cha…
New algorithm reduces dynamic regret in time-varying movement costs.
problem Dynamic regret in online convex optimization with time-varying movement costs.
method Introduced a novel algorithm for time-varying movement costs, achieving comparator-adaptive dynamic regret bound.
result Established first comparator-adaptive dynamic regret bound of O ~ ( ( M 2 + M P T ) ( T + ∑ t λ t ) ) \widetilde{\mathcal{O}}(\sqrt{(M^2+MP_T)(T+\sum_t λ_t)}) O ( ( M 2 + M P T ) ( T + ∑ t λ t ) ) . Improved COCO algorithms with better constraint control.
problem Achieving small regret and constraint violation in online convex optimization.
method Simple projection-based algorithm leveraging self-contraction geometry.
result Exponential improvement in cumulative constraint violation for strongly convex losses.
Unified framework for analyzing online convex optimization across various settings.
problem Analyzing online convex optimization in different settings and feedback types.
method Unified framework allowing systematic proposal and analysis of meta-algorithms.
result Comparable regret bounds for various feedback types and adversary types.
Paper proposes OPF policy for fair resource allocation with sublinear regret.
problem Fair resource allocation in an online setting against an unrestricted adversary.
method Online Proportional Fair (OPF) policy achieving approximate sublinear regret.
result OPF policy achieves c α c_α c α -approximate sublinear regret with c α ≤ 1.445 c_α \leq 1.445 c α ≤ 1.445 . Universal online optimization for dynamic environments using uniclass prediction.
problem Online optimization in changing environments with dynamic regret.
method Reduces dynamic online optimization to uniclass prediction problem, allowing control over dynamic regret bounds.
result First paper with state-of-the-art dynamic regret guarantees for general convex cost functions.
New method for online meta-learning reduces dynamic regret in changing environments.
problem Learning new tasks quickly from limited data in dynamic settings.
method Established dynamic regret analysis using generalized adaptive gradient methods.
result Logarithmic local dynamic regret with dependence on total iterations and learner parameters.
Optimistic Mirror Descent framework improves bidding strategies in non-stationary first-price auctions.
problem Optimizing bidding strategies in non-stationary first-price auctions.
method Introducing Optimistic Mirror Descent (OMD) framework with novel optimism configuration.
result Minimax-optimal dynamic regret rates achieved for non-stationary first-price auctions.
We consider a setting where an agent's uncertainty is represented by a set of probability measures, rather than a single measure. Measure-bymeasure updating of such a set of measures upon acquiring new information is well-known to suffer from problems; agents are not always able to learn appropriately. To deal with the…
Study of multidimensional control problems with reflection controls.
problem Solving control problems with reflection controls in multidimensional settings.
method Gradient descent algorithm for polytope approximations, data-driven domain estimator, episodic learning algorithm.
result Data-driven solutions for unknown diffusion dynamics with sublinear regret.
Optimal online linear regression in dynamic environments using discounted Vovk-Azoury-Warmuth forecaster.
problem Achieving optimal performance in dynamic online linear regression without prior knowledge.
method Developed a discounted variant of the Vovk-Azoury-Warmuth forecaster to achieve optimal dynamic regret guarantees.
result Achieved dynamic regret of the form $O\left(d\log(T)\vee \sqrt{dP_{T}^γ(\vec{u})T}
ight)$ , with a learnable discount factor.
PCGS-TF uses a Transformer to adaptively control expert switching in non-stationary environments.
problem Static regret is insufficient for strictly online prediction in non-stationary settings.
method Policy-Controlled Generalized Share (PCGS) with a Transformer as an update controller.
result PCGS-TF achieves the lowest dynamic regret in non-stationary families and expert pools.
Avare improves optimization and sampling with adaptive importance sampling.
problem Improving convergence rate of stochastic gradient-based algorithms.
method Adaptive importance sampling with decreasing step-sizes.
result Achieves dynamic regret bounds of O ( T 2 / 3 ) \mathcal{O}(T^{2/3}) O ( T 2/3 ) and O ( T 5 / 6 ) \mathcal{O}(T^{5/6}) O ( T 5/6 ) . This study develops a dynamic inverse optimization framework to recover hidden, time-varying preferences from observed allocation trajectories.
problem The gap between classical optimization theory and real-world practice, especially in the presence of drift and shocks.
method Dynamic inverse optimization framework using a drift-aware estimator grounded in convex analysis and online learning theory.
result Sharp static and dynamic regret bounds for the framework, demonstrating its responsiveness to gradual drift and sudden shocks.
OMD and DA perform similarly in static settings but OMD is inferior under dynamic learning rates.
problem Proving and understanding the performance difference between OMD and DA under dynamic learning rates.
method Introducing stabilization to OMD and modifying its convergence analysis.
result OMD with stabilization and DA have the same performance guarantees under dynamic learning rates.
A learning-based algorithm optimizes admission control in a queuing system.
problem Optimizing admission decisions in a queuing system with unknown parameters.
method Proposes a learning-based dispatching algorithm to minimize regret compared to optimal policies.
result Achieves optimal regret bounds for different scenarios of unknown parameters.
The paper addresses treatment recommendation problems by optimizing distributional characteristics.
problem Optimizing treatment recommendations based on distributional targets.
method Characterizes the problem's difficulty and proposes near-optimal policies.
result Characterizes the difficulty of the problem and proposes near-regret optimal policies.
Algorithm minimizes control regret for non-stationary LQR systems.
problem Control of non-stationary LQR systems with unknown dynamics.
method Adaptive non-stationarity detection and OLS estimator with small bias.
result Achieves optimal dynamic regret of $ ilde{\mathcal{O}}\left(V_T^{2/5}T^{3/5}
ight)$ .