Online learning is a powerful tool for analyzing iterative algorithms. However, the classic adversarial setup sometimes fails to capture certain regularity in online problems in practice. Motivated by this, we establish a new setup, called Continuous Online Learning (COL), where the gradient of online loss function cha…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Sublinear Dynamic Regret”
Sequential screening and dynamic regret in multi-armed bandits with arriving arms
problem Sequential experimentation with expanding arm set
method UCB-AA with preliminary screening
result Regret bounds depend on arrival process
New approach for distributed online optimization of non-convex losses with sublinear regret.
problem Regret evaluation and consensus in distributed, multi-agent systems with non-convex losses.
method Composite regret metric and consensus-based online normalized gradient (CONGD) approach for pseudo-convex losses; offline optimization oracle for general non-convex losses.
result First sublinear regret bound for general distributed online non-convex learning.
New setup for continuous online learning improves understanding of imitation learning.
problem Challenges in capturing regularity in online problems.
method Continuous Online Learning (COL) setup, focusing on continuous gradient changes.
result Fundamental equivalence between sublinear dynamic regret and solving certain EPs.
Study online control of unknown time-varying systems with negative and positive results.
problem Online control of time-varying systems with unknown dynamics.
method Algorithmic upper bounds and lower bounds for different policy classes.
result Sublinear adaptive regret bounds for Disturbance Response policies.
New algorithm reduces control error in systems with changing dynamics.
problem Online control of systems with time-varying linear dynamics.
method Introduces adaptive regret metric and a novel meta-algorithm.
result First adaptive regret bound for online convex optimization with memory.
Algorithm reduces regret in partially observable systems by learning dynamics and using optimistic control.
problem Minimizing regret in partially observable linear quadratic control systems with unknown dynamics.
method ExpCommit algorithm that learns model parameters and uses optimism in uncertainty.
result End-to-end sublinear regret upper bound of for ExpCommit.
Efficient algorithm controls unknown systems with adversarial perturbations.
problem Controlling unknown linear systems with adversarial perturbations and convex losses.
method Measures regret against an optimal linear policy, provides efficient algorithm with sublinear regret bound.
result First efficient algorithm with sublinear regret bound of T^{2/3}.
Study online RL with mismatched dynamics, achieving sublinear regret.
problem Exploration challenges in online RL with mismatched training and deployment dynamics.
method Introduce supremal visitation ratio, propose efficient algorithm with -divergence.
result Achieves sublinear regret in online RMDPs with optimal dependence on supremal visitation ratio and interaction episodes.
New controller reduces regret in non-stochastic control with adversarial perturbations.
problem Non-stochastic control with adversarial perturbations and partially observed states.
method Denoised observations and online gradient descent.
result Sublinear regret bounds, optimal for known and unknown systems.
New algorithm controls systems with unknown, changing losses.
problem Control systems with adversarial perturbations and unknown loss function.
method Efficient sublinear regret algorithm for bandit convex optimization with memory.
result Achieves efficient control with sublinear regret in the presence of unknown, changing losses.
Two new algorithms reduce online kernel regression's computational cost while maintaining optimal regret bounds.
problem Trade-off between regret and computational cost in online kernel regression.
method AOGD-ALD and NONS-ALD algorithms dynamically maintain nearly orthogonal basis to approximate kernel mapping and control approximate error.
result Achieves nearly optimal regret bounds at sublinear computational complexity.
New algorithms for constrained online optimization with memory and predictions.
problem Control of constrained dynamical systems and scheduling with reconfiguration budgets.
method Proposed algorithms achieving sublinear regret and constraint violation under time-varying constraints, both with and without predictions.
result First algorithms achieving sublinear regret and constraint violation in constrained online optimization with memory.
This paper considers distributed online optimization with time-varying coupled inequality constraints. The global objective function is composed of local convex cost and regularization functions and the coupled constraint function is the sum of local convex functions. A distributed online primal-dual dynamic mirror des…
Memory-limited learning tackles adversarial bandits with reduced storage.
problem Adversarial bandit problem with limited memory storage.
method Hierarchical learning policy with sublinear memory requirement.
result Established sublinear regret bounds for weak and shifting regrets.
Study online learning in unknown Markov games with sublinear regret.
problem Online learning in unknown Markov games with unobservable opponents.
method Introduced an algorithm achieving sublinear regret against the minimax value.
result First sublinear regret bound for unknown Markov games, independent of action spaces size.
Study local exploration on dynamic graphs with time-varying edges.
problem Learning optimal actions in a network with changing connections.
method Local explore-then-commit algorithms under a structural condition ensuring intrinsic walk stability.
result Sublinear expected regret for reward-aware strategies.
The paper explores dynamic regret with switching cost in online decision making.
problem The relation between dynamic regret and switching cost in online decision making.
method Investigates two classic online settings: Online Algorithms (OA) and Online Convex Optimization (OCO). Provides a new theoretical analysis framework.
result The switching cost impacts dynamic regret differently in OA and has no impact in OCO.
Revisiting Matrix Sketching in Linear Bandits: Achieving Sublinear Regret via Dyadic Block Sketchingcs.LG
New method reduces linear regret in high-dimensional bandit problems.
problem Heavy spectral tails in streaming matrices lead to linear regret in sketch-based linear bandits.
method Dyadic Block Sketching, a multi-scale matrix sketching approach.
result Achieves sublinear regret bounds without prior knowledge of streaming matrix properties.
Supplier learns to price contracts against a learning retailer.
problem Designing data-driven pricing policies for a supplier facing a learning retailer.
method Connecting to non-stationary online learning, proposing dynamic pricing policies for discrete and continuous demand.
result Supplier's pricing policies lead to sublinear regret bounds under various retailer learning policies.
Online learning algorithms are designed to learn even when their input is generated by an adversary. The widely-accepted formal definition of an online algorithm's ability to learn is the game-theoretic notion of regret. We argue that the standard definition of regret becomes inadequate if the adversary is allowed to a…
New algorithms for stochastic linear bandits with heavy-tailed payoffs achieve nearly optimal regret.
problem Stochastic linear bandits with heavy-tailed payoffs.
method Median of means and dynamic truncation.
result Sublinear regret bound of for .
We consider online forecasting problems for non-convex machine learning models. Forecasting introduces several challenges such as (i) frequent updates are necessary to deal with concept drift issues since the dynamics of the environment change over time, and (ii) the state of the art models are non-convex models. We ad…
Algorithm learns multiple tasks with minimal planning, achieving near-optimal performance.
problem Learning multiple tasks efficiently in a reinforcement learning setting.
method UCB Lifelong Value Distillation (UCBlvd) algorithm with structural assumption for shared exploration.
result Sublinear regret bound of with planning calls.
New framework finds periodic policies in reset-free MDPs with sublinear regret.
problem Reset-free reinforcement learning with unknown dynamics and terminal law constraints.
method Periodic framework, periodic policies, periodic regret.
result First non-asymptotic guarantees for reset-free learning in multi-agent settings.
LIBO optimizes repeated bandit tasks without prior knowledge or regret.
problem Optimizing repeated bandit tasks without prior knowledge or regret.
method LIBO sequentially meta-learns a kernel to adapt to the environment and solve tasks with the latest estimate.
result LIBO achieves sublinear lifelong regret, converging to oracle performance as more tasks are solved.
New algorithm achieves sublinear regret in CMDPs without error cancellations.
problem Safety constraints in reinforcement learning with error cancellations.
method Model-based primal-dual algorithm for CMDPs with multiple constraints.
result Achieves sublinear regret without error cancellations.
GP-UCB resolves sublinear regret for kernelized bandits.
problem Minimizing regret in kernelized bandit problems.
method Using a new regularization technique for kernel ridge estimators, improving GP-UCB's sublinear regret rate.
result GP-UCB achieves nearly optimal sublinear regret for the Matérn kernel.
LaPSRL achieves optimal regret for isoperimetric RL distributions.
problem Designing RL algorithms with sublinear regret for non-log-concave distributions.
method Posterior Sampling (PSRL) and Langevin sampling (LaPSRL) for isoperimetric distributions.
result LaPSRL achieves order-optimal regret and subquadratic complexity.
New algorithm balances personalization and statistical validity in MRTs.
problem Optimizing decisions in nonstationary settings with habituation and recovery.
method ROGUE-TS Thompson Sampling with probability clipping.
result Achieves lower regret and maintains high statistical power.
Paper proposes OPF policy for fair resource allocation with sublinear regret.
problem Fair resource allocation in an online setting against an unrestricted adversary.
method Online Proportional Fair (OPF) policy achieving approximate sublinear regret.
result OPF policy achieves -approximate sublinear regret with .
Efficiently controls unknown linear systems with black-box interactions.
problem Controlling an unknown linear dynamical system from black-box interactions.
method First efficient algorithm with sublinear regret, using robust system identification.
result Resolves open problem on stochastic LQR and black-box LQR control.
Two-stage mechanism designs reduce regret in recommender systems with stochastic covariates.
problem Designing effective recommender systems with user covariates sampled online.
method Two-stage algorithm integrating incentivized exploration with offline learning methods.
result Achieves sublinear regret while maintaining incentive compatibility.
Universal preconditioning reduces sequential prediction regret.
problem Improving sequential prediction performance.
method Convolve target sequence with orthogonal polynomial coefficients.
result First sublinear and hidden-dimension-independent regret bounds.
Greedy algorithm achieves sublinear regret for various distributions.
problem Efficient performance of greedy algorithms in linear contextual bandit problems.
method Introduced Local Anti-Concentration (LAC) condition to ensure sublinear regret.
result Greedy algorithm achieves cumulative expected regret.
Algorithm achieves logarithmic regret with sublinear hints.
problem Online linear optimization with limited hints.
method Using logarithmic hints to improve regret from sqrt(T) to log(T).
result O(log T) regret with O(sqrt(T)) hints, and O(sqrt(T)) regret with o(sqrt(T)) hints.
New RL algorithm achieves sublinear regret and constraint violation without simulators.
problem Maximizing reward under utility constraints in large-scale systems.
method Model-free, simulator-free algorithm using LSVI-UCB with primal-dual optimization and soft-max policy.
result Achieves regret and constraint violation bounds.
New framework guides resource usage to achieve sublinear regret in adversarial settings.
problem Achieving sublinear regret in online decision making with changing reward and cost distributions.
method General primal-dual methods guided by spending plans that ensure balanced resource usage.
result Achieves sublinear regret with respect to spending plans that balance resource usage.
The paper develops methods for time-varying constrained online convex optimization.
problem Time-varying loss and constraint functions in online convex optimization.
method Model-based augmented Lagrangian methods (MALM) for time-varying and delayed feedback.
result Sublinear regret and constraint violation for both time-varying and delayed feedback scenarios.
Proposes H-UCRL for efficient model-based RL with sublinear regret.
problem Greedy policy exploration in model-based RL ignores epistemic uncertainty.
method Reparameterizes plausible models, hallucinates control, augments input space, solves with greedy planners.
result H-UCRL achieves provably sublinear regret for Gaussian Process models.
Sublinear LSVI via LSH reduces runtime to sublinear in actions.
problem Efficiently estimating value functions in reinforcement learning with sublinear runtime.
method Formulated as approximate maximum inner product search, used LSH to solve with sublinear time complexity.
result Sublinear runtime while maintaining LSVI's regret.
New algorithm improves game learning with randomised optimism.
problem Learning in matrix games with unknown payoffs and bandit feedback.
method Integrates evolutionary algorithms into bandit framework for randomised optimism.
result Achieves sublinear regret, outperforming classical methods.
The paper connects discrete choice models to multi-armed bandit algorithms with sublinear regret bounds.
problem Optimizing user choices in a multi-armed bandit setting.
method Establishes connections between discrete choice models and multi-armed bandit algorithms, providing sublinear regret bounds and novel algorithms.
result Sublinear regret bounds for a family of algorithms, including the Exp3 algorithm.
Algorithm learns interference network and optimizes treatment allocation for unknown network effects.
problem Adaptive experimentation under unknown network interference.
method Thompson sampling algorithm with Gibbs sampler for joint learning of interference network and treatment allocation.
result Proves a Bayesian regret bound and achieves sublinear regret in real-world applications.
The paper addresses online prediction in marginally stable systems with bounded perturbations.
problem Online prediction in marginally stable linear dynamical systems with adversarial or stochastic perturbations.
method The online least-squares algorithm is used to achieve sublinear regret, with a refined regret analysis and a structural lemma.
result The online least-squares algorithm achieves sublinear regret, with polynomial dependence on the system's parameters.
GP-PSRL achieves sublinear regret for continuous control with unbounded state space.
problem Analyzing regret bounds for GP-PSRL in continuous control with unbounded state space.
method Recursive application of Borell-Tsirelson-Ibragimov-Sudakov inequality and chaining method.
result Sublinear regret bound of for GP-PSRL.
New algorithm tackles delayed feedback in Lipschitz bandits with sublinear regret.
problem Delayed feedback in Lipschitz bandits.
method Design of algorithms for bounded and unbounded stochastic delays.
result Sublinear regret guarantees for both bounded and unbounded delays.
Algorithm minimizes regret and converges to equilibria in Markov games.
problem Regret minimization and convergence to equilibria in general-sum Markov games under adversarial opponents.
method Decentralized algorithm that uses policy optimization and controls path length to achieve sublinear regret.
result Sublinear regret guarantees for convergence to correlated equilibrium in Markov games.