Memory-limited learning tackles adversarial bandits with reduced storage.
problem Adversarial bandit problem with limited memory storage.
method Hierarchical learning policy with sublinear memory requirement.
result Established sublinear regret bounds for weak and shifting regrets.
Study online learning in unknown Markov games with sublinear regret.
problem Online learning in unknown Markov games with unobservable opponents.
method Introduced an algorithm achieving sublinear regret against the minimax value.
result First sublinear regret bound for unknown Markov games, independent of action spaces size.
New approach for distributed online optimization of non-convex losses with sublinear regret.
problem Regret evaluation and consensus in distributed, multi-agent systems with non-convex losses.
method Composite regret metric and consensus-based online normalized gradient (CONGD) approach for pseudo-convex losses; offline optimization oracle for general non-convex losses.
result First sublinear regret bound for general distributed online non-convex learning.
Online learning is a powerful tool for analyzing iterative algorithms. However, the classic adversarial setup sometimes fails to capture certain regularity in online problems in practice. Motivated by this, we establish a new setup, called Continuous Online Learning (COL), where the gradient of online loss function cha…
Online learning algorithms are designed to learn even when their input is generated by an adversary. The widely-accepted formal definition of an online algorithm's ability to learn is the game-theoretic notion of regret. We argue that the standard definition of regret becomes inadequate if the adversary is allowed to a…
LIBO optimizes repeated bandit tasks without prior knowledge or regret.
problem Optimizing repeated bandit tasks without prior knowledge or regret.
method LIBO sequentially meta-learns a kernel to adapt to the environment and solve tasks with the latest estimate.
result LIBO achieves sublinear lifelong regret, converging to oracle performance as more tasks are solved.
New algorithm achieves sublinear regret in CMDPs without error cancellations.
problem Safety constraints in reinforcement learning with error cancellations.
method Model-based primal-dual algorithm for CMDPs with multiple constraints.
result Achieves sublinear regret without error cancellations.
GP-UCB resolves sublinear regret for kernelized bandits.
problem Minimizing regret in kernelized bandit problems.
method Using a new regularization technique for kernel ridge estimators, improving GP-UCB's sublinear regret rate.
result GP-UCB achieves nearly optimal sublinear regret for the Matérn kernel.
LaPSRL achieves optimal regret for isoperimetric RL distributions.
problem Designing RL algorithms with sublinear regret for non-log-concave distributions.
method Posterior Sampling (PSRL) and Langevin sampling (LaPSRL) for isoperimetric distributions.
result LaPSRL achieves order-optimal regret and subquadratic complexity.
Paper proposes OPF policy for fair resource allocation with sublinear regret.
problem Fair resource allocation in an online setting against an unrestricted adversary.
method Online Proportional Fair (OPF) policy achieving approximate sublinear regret.
result OPF policy achieves c α c_α c α -approximate sublinear regret with c α ≤ 1.445 c_α \leq 1.445 c α ≤ 1.445 . Two-stage mechanism designs reduce regret in recommender systems with stochastic covariates.
problem Designing effective recommender systems with user covariates sampled online.
method Two-stage algorithm integrating incentivized exploration with offline learning methods.
result Achieves sublinear regret while maintaining incentive compatibility.
Two new algorithms reduce online kernel regression's computational cost while maintaining optimal regret bounds.
problem Trade-off between regret and computational cost in online kernel regression.
method AOGD-ALD and NONS-ALD algorithms dynamically maintain nearly orthogonal basis to approximate kernel mapping and control approximate error.
result Achieves nearly optimal regret bounds at sublinear computational complexity.
Greedy algorithm achieves sublinear regret for various distributions.
problem Efficient performance of greedy algorithms in linear contextual bandit problems.
method Introduced Local Anti-Concentration (LAC) condition to ensure sublinear regret.
result Greedy algorithm achieves O ( poly log T ) O(\operatorname{poly} \log T) O ( poly log T ) cumulative expected regret. Sequential screening and dynamic regret in multi-armed bandits with arriving arms
problem Sequential experimentation with expanding arm set
method UCB-AA with preliminary screening
result Regret bounds depend on arrival process
Algorithm achieves logarithmic regret with sublinear hints.
problem Online linear optimization with limited hints.
method Using logarithmic hints to improve regret from sqrt(T) to log(T).
result O(log T) regret with O(sqrt(T)) hints, and O(sqrt(T)) regret with o(sqrt(T)) hints.
New framework guides resource usage to achieve sublinear regret in adversarial settings.
problem Achieving sublinear regret in online decision making with changing reward and cost distributions.
method General primal-dual methods guided by spending plans that ensure balanced resource usage.
result Achieves sublinear regret with respect to spending plans that balance resource usage.
The paper develops methods for time-varying constrained online convex optimization.
problem Time-varying loss and constraint functions in online convex optimization.
method Model-based augmented Lagrangian methods (MALM) for time-varying and delayed feedback.
result Sublinear regret and constraint violation for both time-varying and delayed feedback scenarios.
Algorithm reduces regret in partially observable systems by learning dynamics and using optimistic control.
problem Minimizing regret in partially observable linear quadratic control systems with unknown dynamics.
method ExpCommit algorithm that learns model parameters and uses optimism in uncertainty.
result End-to-end sublinear regret upper bound of O ~ ( T 2 / 3 ) \tilde{\mathcal{O}}(T^{2/3}) O ~ ( T 2/3 ) for ExpCommit. Study online control of unknown time-varying systems with negative and positive results.
problem Online control of time-varying systems with unknown dynamics.
method Algorithmic upper bounds and lower bounds for different policy classes.
result Sublinear adaptive regret bounds for Disturbance Response policies.
New algorithms for constrained online optimization with memory and predictions.
problem Control of constrained dynamical systems and scheduling with reconfiguration budgets.
method Proposed algorithms achieving sublinear regret and constraint violation under time-varying constraints, both with and without predictions.
result First algorithms achieving sublinear regret and constraint violation in constrained online optimization with memory.
Sublinear LSVI via LSH reduces runtime to sublinear in actions.
problem Efficiently estimating value functions in reinforcement learning with sublinear runtime.
method Formulated as approximate maximum inner product search, used LSH to solve with sublinear time complexity.
result Sublinear runtime while maintaining LSVI's regret.
New algorithm improves game learning with randomised optimism.
problem Learning in matrix games with unknown payoffs and bandit feedback.
method Integrates evolutionary algorithms into bandit framework for randomised optimism.
result Achieves sublinear regret, outperforming classical methods.
The paper connects discrete choice models to multi-armed bandit algorithms with sublinear regret bounds.
problem Optimizing user choices in a multi-armed bandit setting.
method Establishes connections between discrete choice models and multi-armed bandit algorithms, providing sublinear regret bounds and novel algorithms.
result Sublinear regret bounds for a family of algorithms, including the Exp3 algorithm.
GP-PSRL achieves sublinear regret for continuous control with unbounded state space.
problem Analyzing regret bounds for GP-PSRL in continuous control with unbounded state space.
method Recursive application of Borell-Tsirelson-Ibragimov-Sudakov inequality and chaining method.
result Sublinear regret bound of O ~ ( H γ T T ) \widetilde{\mathcal{O}}(H\sqrt{γ_TT}) O ( H γ T T ) for GP-PSRL. New algorithm tackles delayed feedback in Lipschitz bandits with sublinear regret.
problem Delayed feedback in Lipschitz bandits.
method Design of algorithms for bounded and unbounded stochastic delays.
result Sublinear regret guarantees for both bounded and unbounded delays.
New controller reduces regret in non-stochastic control with adversarial perturbations.
problem Non-stochastic control with adversarial perturbations and partially observed states.
method Denoised observations and online gradient descent.
result Sublinear regret bounds, optimal for known and unknown systems.
Algorithm minimizes regret and converges to equilibria in Markov games.
problem Regret minimization and convergence to equilibria in general-sum Markov games under adversarial opponents.
method Decentralized algorithm that uses policy optimization and controls path length to achieve sublinear regret.
result Sublinear regret guarantees for convergence to correlated equilibrium in Markov games.
New algorithms reduce private bandit regret to nearly non-private levels.
problem Differentially private adversarial bandits and expert advice.
method Conversion of non-private algorithms to private, new algorithms for bandits and expert advice.
result Improved regret bounds for private bandits, sublinear for small ε.
Posterior sampling-based EI achieves sublinear regret bounds for expensive function optimization.
problem Theoretical analysis of expected improvement (EI) in Bayesian optimization.
method Randomized posterior sampling of EI.
result Achieves sublinear Bayesian cumulative regret bounds.
New algorithm reduces prediction error in online learning without knowing base measure.
problem Smoothed online learning without knowledge of base measure.
method R-Cover algorithm based on recursive coverings.
result First algorithm to guarantee sublinear regret for agnostic smoothed online learning without prior knowledge of base measure.
New method reduces linear regret in high-dimensional bandit problems.
problem Heavy spectral tails in streaming matrices lead to linear regret in sketch-based linear bandits.
method Dyadic Block Sketching, a multi-scale matrix sketching approach.
result Achieves sublinear regret bounds without prior knowledge of streaming matrix properties.
Boosts weak online learners to strong ones with sublinear regret.
problem Online learning agnostic setting without strong guarantees.
method Reduction to online convex optimization, boosting via marginally-better-than-trivial regret guarantees.
result First agnostic online boosting algorithm with sublinear regret.
New algorithm reduces control error in systems with changing dynamics.
problem Online control of systems with time-varying linear dynamics.
method Introduces adaptive regret metric and a novel meta-algorithm.
result First adaptive regret bound for online convex optimization with memory.
Paper establishes no-regret property for practical EGO optimization.
problem No theoretical bounds on cumulative regret for practical EGO.
method Introduced practical EGO with a positive nugget, analyzed its regret bounds.
result Practical EGO is a no-regret algorithm with sublinear regret bounds.
New algorithm optimizes functions in Matérn kernel RKHS with noisy feedback.
problem Optimizing functions in RKHS of Matérn kernel with noisy bandit feedback.
method π-GP-UCB algorithm with guaranteed sublinear regret for all ν > 1 and d ≥ 1.
result First practical approach with guaranteed sublinear regret for all ν > 1 and d ≥ 1.
Study online RL with mismatched dynamics, achieving sublinear regret.
problem Exploration challenges in online RL with mismatched training and deployment dynamics.
method Introduce supremal visitation ratio, propose efficient algorithm with f f f -divergence. result Achieves sublinear regret in online RMDPs with optimal dependence on supremal visitation ratio and interaction episodes.
New approach tackles resource constraints in bandit problems with weakly adaptive algorithms.
problem Maximizing rewards while adhering to general long-term constraints.
method Weakly adaptive primal and dual regret minimizers.
result Achieves sublinear constraints violations and competitive ratios in both stochastic and adversarial settings.
Online learning is a powerful tool for analyzing iterative algorithms. However, the classic adversarial setup sometimes fails to capture certain regularity in online problems in practice. Motivated by this, we establish a new setup, called Continuous Online Learning (COL), where the gradient of online loss function cha…
Algorithm learns multiple tasks with minimal planning, achieving near-optimal performance.
problem Learning multiple tasks efficiently in a reinforcement learning setting.
method UCB Lifelong Value Distillation (UCBlvd) algorithm with structural assumption for shared exploration.
result Sublinear regret bound of i l d e O ( ( d 3 + d ′ d ) H 4 K ) ilde{\mathcal{O}}(\sqrt{(d^3+d^\prime d)H^4K}) i l d e O ( ( d 3 + d ′ d ) H 4 K ) with O ( d H log ( K ) ) \mathcal{O}(dH\log(K)) O ( d H log ( K )) planning calls. New algorithm controls systems with unknown, changing losses.
problem Control systems with adversarial perturbations and unknown loss function.
method Efficient sublinear regret algorithm for bandit convex optimization with memory.
result Achieves efficient control with sublinear regret in the presence of unknown, changing losses.
New framework finds periodic policies in reset-free MDPs with sublinear regret.
problem Reset-free reinforcement learning with unknown dynamics and terminal law constraints.
method Periodic framework, periodic policies, periodic regret.
result First non-asymptotic guarantees for reset-free learning in multi-agent settings.
Algorithm maximizes revenue-risk by estimating price impact kernel and optimizing control problems.
problem Maximizing revenue-risk in a risky asset liquidation with unknown price impact.
method Alternates exploration and exploitation phases, uses novel kernel estimation and stability results.
result Sublinear regret achieved with high probability.
New algorithms achieve small prediction regret for learning from overlapping groups.
problem Online multi-group learning with fairness applications.
method Oracle-efficient algorithms for groups not explicitly enumerated.
result Sublinear regret in various settings.
Study shows how repetition affects learning in bandit settings, providing algorithms with sublinear regret.
problem Effect of persistence of engagement on learning in stochastic multi-armed bandit settings.
method Novel algorithms that achieve sublinear regret under temporal constraints.
result Additive effect of priming on regret upper bound, matching popular algorithms in absence of priming.
A new federated algorithm reduces regret in X-armed bandit problems.
problem Collaborative optimization of heterogeneous local objectives.
method Fed-PNE algorithm using hierarchical partitioning and weak smoothness.
result Achieves sublinear cumulative regret with minimal communication.
Paper tackles stochastic k k k -submodular bandits with full feedback, achieving sublinear regret.
problem Online optimization of k k k -submodular functions with full-bandit feedback. method Proposes online algorithms for various k k k -submodular stochastic combinatorial multi-armed bandit problems. result Achieves sublinear α α α -regret bounds for multiple k k k -submodular stochastic combinatorial multi-armed bandit problems. New algorithm reduces regret in noisy context bandits.
problem Online decision-making with noisy context predictions.
method Extends classical statistics measurement error model to online decision-making.
result Achieves sublinear regret guarantees under mild conditions.
Algorithm reduces regret in safe Bayesian optimization with monotonicity constraints.
problem Sequentially maximize unknown function with safety constraints.
method Sequential algorithms using Gaussian processes with safety constraints modeled as monotonicity.
result Sublinear regret achieved for expanding safe region and finding optimal s s s .