Randomized exploration methods are more statistically efficient than optimistic methods in reinforcement learning.
problem Comparing and contrasting optimistic and randomized exploration methods in reinforcement learning.
method Analytic examples to compare optimistic and randomized approaches.
result Randomized approaches are more statistically efficient than optimistic approaches.
Proposes Optimistic Pessimistically Initialised Q-Learning (OPIQ) for better exploration in RL.
problem Pessimistic initialisation of Q-values in deep RL leads to poor exploration performance.
method Augments pessimistically initialised Q-values with count-based bonuses to ensure optimism.
result OPIQ outperforms non-optimistic DQN variants in hard exploration tasks.
Combines online learning algorithms to achieve better performance.
problem Improving online learning algorithms with varying guarantees.
method Adding iterates of two parameter-free algorithms to create a new algorithm with improved regret.
result Generates efficient algorithms that adapt to multiple norms and maintain dimension-free guarantees.
Characterizes concept classes for optimistic online learning.
problem Understanding minimal assumptions for online learnability.
method Investigates two questions about concept classes' learnability.
result Characterizes all concept classes for optimistically universal online learnability.
New algorithms reduce forecasting errors by leveraging optimistic learning and hinting.
problem Real-time climate forecasting with delayed feedback.
method Developed optimistic online learning algorithms that reduce regret under delayed feedback.
result Optimistic learning algorithms achieve optimal regret guarantees under delayed feedback.
Optimistic NPG improves policy optimization in online RL with efficient sample complexity.
problem Limited theoretical understanding of policy optimization, especially in online RL.
method Combines natural policy gradient with optimistic policy evaluation.
result Achieves optimal dimension dependence sample complexity for learning near-optimal policies.
Optimistic likelihoods improve classification accuracy by considering nearby distributions.
problem Evaluating likelihoods of nominal distributions estimated from data, which can be inaccurate.
method Use ambiguity sets and geodesic/standard convex optimization to compute optimistic likelihoods.
result Optimistic likelihoods lead to better classification performance.
Unified framework for model- and value-optimistic reinforcement learning.
problem Designing and analyzing algorithms for episodic reinforcement learning.
method Lagrangian duality for a unified view of optimism.
result Unified algorithms with both efficient implementation and probabilistic analysis.
New method minimizes regret in AMDP with high probability.
problem Pessimistic regret analysis in AMDP.
method Developed a new policy search method achieving optimistic regret.
result Achieved sublinear optimistic regret with high probability.
Proposes H-UCRL for efficient model-based RL with sublinear regret.
problem Greedy policy exploration in model-based RL ignores epistemic uncertainty.
method Reparameterizes plausible models, hallucinates control, augments input space, solves with greedy planners.
result H-UCRL achieves provably sublinear regret for Gaussian Process models.
Novel approach to universal online learning for bounded losses, closing open problems.
problem Characterizing processes for universal online learning under non-i.i.d. conditions.
method Characterization of processes admitting strong and weak universal learning, introduction of optimistically universal learning rule.
result Introduction of a novel 1NN algorithm that is optimistically universal for bounded losses.
Solves open problem on universally consistent online learning with unbounded losses.
problem Open problem on universally consistent online learning with unbounded losses.
method Constructs random measurable partitions of the instance space.
result Simple memorization rule is optimistically universal for any unbounded loss.
Optimistic Hedge achieves optimal regret bounds in two-player zero-sum games.
problem Achieving optimal regret bounds for optimistic Hedge in two-player zero-sum games.
method Refined regret analysis and optimization problem formulation.
result Optimistic Hedge achieves O ( log m log n ) O(\sqrt{\log m \log n}) O ( log m log n ) regret bounds, matching upper and lower bounds. Optimistic bounds for multi-output learning using self-bounding Lipschitz condition.
problem Learning vector-valued functions from supervised data.
method Introducing self-bounding Lipschitz condition and proving optimistic bounds using local Rademacher complexity and Srebro's inequality.
result Minimax optimal generalization bounds for multi-output learning, up to logarithmic factors.
Omega method mitigates noise in stochastic game optimization.
problem Noise sensitivity and convergence issues in stochastic game optimization.
method Omega method incorporates EMA of historic gradients in its update rule.
result Omega method outperforms optimistic gradient method in stochastic games.
Bayesian classifier improves robustness with optimistic score ratio.
problem Limited information on class-conditional distribution.
method Optimistic score ratio for robust binary classification.
result Bayesian classifier using optimistic score ratio is robust and computationally tractable.
Accelerates policy optimization in RL with optimistic and adaptive updates.
problem Improving policy optimization methods in reinforcement learning.
method Integrates foresight into policy improvement step via optimistic and adaptive updates.
result Designs an optimistic policy gradient algorithm, adaptive via meta-gradient learning.
We show that the optimistic limits of the colored Jones polynomials of the hyperbolic knots coincide with the optimistic limits of the Kashaev invariants modulo 4 π 2 4π^2 4 π 2 .
Study optimizes linear regression analysis for high-dimensional settings.
problem Understanding high-dimensional linear regression with interpolation and regularization.
method Localized uniform convergence analysis of optimistic rates for linear regression.
result Recover guarantees for ridge and LASSO regression under random designs.
Efficiently solves exploration-exploitation in LQR using Lagrangian relaxation.
problem Exploration-exploitation dilemma in linear quadratic regulator (LQR) setting.
method Relax optimistic optimization into a constrained extended LQR problem, then solve using Riccati equations.
result Computes ε ε ε -optimistic controller efficiently with O ( log ( 1 / ε ) ) O\big(\log(1/ε)\big) O ( log ( 1/ ε ) ) Riccati equations. New method for optimistic planning in MDPs using regularization.
problem Optimistic planning in infinite-horizon discounted MDPs.
method Regularized dynamic programming for approximate value iteration.
result Achieves near-optimal statistical guarantees in learning policies.
Optimistic PPO variant solves linear MDPs with improved regret bound.
problem Understanding theoretical limits of PPO in linear MDPs.
method Proposes an optimistic variant of PPO for episodic adversarial linear MDPs with full-information feedback.
result Establishes a i l d e O ( d 3 / 4 H 2 K 3 / 4 ) ilde{\mathcal{O}}(d^{3/4}H^2K^{3/4}) i l d e O ( d 3/4 H 2 K 3/4 ) regret bound. The optimistic limit is the mathematical formulation of the classical limit which is a physical method to expect the actual limit by using saddle point method of certain potential function. The original optimistic limit of the Kashaev invariant was formulated by Yokota, and a modified formulation was suggested by the a…
Optimistic estimate predicts best fitting performance of nonlinear models.
problem Evaluating the potential of nonlinear models in fitting.
method Proposes an optimistic estimate to quantify the smallest sample size for fitting nonlinear models.
result Predicts specific subsets of targets that can be fitted at overparameterization.
Kernel-based function approximation improves reinforcement learning performance.
problem Average reward reinforcement learning in infinite horizon settings.
method Optimistic algorithm based on kernel ridge regression.
result No-regret performance guarantees and confidence intervals for kernel-based predictions.
The paper explores optimistic robust optimization for machine learning problems.
problem Addressing uncertainty in machine learning models using optimistic robust optimization.
method Develops optimistic robust optimization techniques for machine learning problems, including robust linear programming and sparsity-inducing regularization.
result Optimistic robust optimization can provide new interpretations and solutions for existing machine learning challenges.
New algorithm reduces regret in sequential decision-making problems.
problem Balancing exploration and exploitation in online sequential decision problems.
method Variational Bayesian optimistic sampling (VBOS) for optimizing policies.
result VBOS achieves i l d e O ( A T ) ilde O(\sqrt{AT}) i l d e O ( A T ) Bayesian regret for stochastic multi-armed bandits. New learning rules achieve optimal sample complexity for weakly supervised classification.
problem Learning from label proportions in weakly supervised settings.
method Debiased proportional square loss and EasyLLP learning rule.
result Achieves optimal sample complexity in both realizable and agnostic settings.
Paper proves convergence of optimistic policy iteration for stochastic shortest path problems.
problem Optimizing policies in stochastic shortest path problems.
method Analyzes optimistic policy iteration algorithm with Monte Carlo and TD(λ) methods.
result Proves convergence of the algorithm under specific conditions.
Paper proposes Adaptive DDPG for better stock portfolio allocation.
problem Challenges in finding optimal stock portfolio allocation in dynamic stock markets.
method Adaptive Deep Deterministic Reinforcement Learning (Adaptive DDPG) incorporating optimistic or pessimistic reinforcement learning.
result Adaptive DDPG outperforms traditional and baseline strategies in investment return and Sharpe ratio.
Optimistic RL algorithms are simplified for deep RL with competitive performance.
problem Achieving accurate optimism in model-based RL for large-scale problems.
method Interpreting scalable optimistic model-based algorithms as solving a tractable noise augmented MDP.
result Competitive regret bound of i l d e O ( ∣ S ∣ H ∣ A ∣ T ) ilde{\mathcal{O}}( |\mathcal{S}|H\sqrt{|\mathcal{A}| T } ) i l d e O ( ∣ S ∣ H ∣ A ∣ T ) for Gaussian noise augmentation. Optimistic algorithms achieve logarithmic regret bounds for MDPs without diameter dependence.
problem Achieving logarithmic regret bounds for episodic MDPs without relying on diameter-like quantities.
method Novel 'clipped' regret decomposition applied to optimistic algorithms.
result Smooth interpolation between gap-dependent and minimax rates of convergence.
Develops ODRPO to improve RL algorithms with better performance and stability.
problem RL algorithms converge to sub-optimal solutions due to limited policy representation.
method Integrates DRO approach to solve trust region constrained optimization problem without parameterizing policies.
result Achieves globally optimal policy update and higher sample efficiency.
Enhances optimization algorithms to achieve faster convergence rates.
problem Minimizing smooth convex functions efficiently.
method Extends optimistic learning to achieve faster convergence rates.
result Achieves a rate of O ( 1 / T 2 ) O(1/T^2) O ( 1/ T 2 ) for optimization. Study on learning without i.i.d. assumptions, focusing on universally consistent function learning.
problem Develop a theory of learning without i.i.d. assumptions.
method Study universally consistent function learning under general stochastic processes, focusing on optimistically universal learning rules.
result Optimistically universal learning rules exist in the self-adaptive learning setting.
Develops new algorithms for solving root-finding problems in large-scale settings.
problem Solving nonlinear equations in large-scale settings.
method Randomized block-coordinate optimistic gradient algorithms.
result Achieves convergence rates of O ( 1 / k ) \mathcal{O}(1/k) O ( 1/ k ) and O ( 1 / k 2 ) \mathcal{O}(1/k^2) O ( 1/ k 2 ) for root-finding problems. Simple algorithm gives optimal regret bounds for reinforcement learning.
problem Optimizing regret in reinforcement learning for Markov decision processes.
method Optimistic algorithm with regret bound analysis based on mixing time.
result First optimal regret bounds with i l d e O ( t m m i x S A T ) ilde{O}(\sqrt{t_{
m mix} SAT}) i l d e O ( t mmi x S A T ) after T T T steps. Algorithm converges to Nash equilibria in competitive games.
problem Finding Nash equilibria in decentralized, competitive Markov games.
method Decentralized Optimistic Gradient Descent/Ascent with a critic.
result Converges to the set of Nash equilibria under self-play.
Paper tackles constrained bandit problems with a new learning framework.
problem Optimizing a black-box reward function subject to a black-box constraint function over a continuous space.
method Rectified Pessimistic-Optimistic Learning (RPOL) framework, incorporating optimistic and pessimistic GP bandit learning.
result RPOL achieves sublinear regret and minimal cumulative constraint violation.
Algorithm finds best mixture of training datasets for improved validation performance.
problem Learning from mixture distributions with covariate shift.
method Combines SGD with optimistic tree search and model re-use over mixture space.
result Proves simple regret guarantees for recovering optimal mixture.
New algorithm reduces reinforcement learning regret to sqrt(d^3T).
problem Efficient reinforcement learning with generalized linear function approximation.
method Optimistic closure assumption for generalized linear functions.
result Proved regret bound of O(sqrt(d^3T)).
New algorithm reduces reinforcement learning regret to sqrt(T) without strong dynamics assumptions.
problem Infinite-horizon average-reward reinforcement learning with linear MDPs.
method Approximate by discounted-reward MDPs and apply optimistic value iteration.
result Achieves O(sqrt(T)) regret with polynomial complexity.
OMD improves training GANs by avoiding limit cycling.
problem Limit cycling behavior in training GANs.
method Optimistic Mirror Decent (OMD) for training Wasserstein GANs.
result OMD addresses limit cycling in WGANs and converges to an equilibrium.
Optimistic method adapted for faster convex-concave min-max problems.
problem Solving convex-concave min-max optimization problems efficiently.
method Adaptive, line search-free second-order methods combining optimistic updates and second-order information.
result Achieves optimal convergence rate without line search or backtracking.
The paper analyzes bias in sample means of multi-armed bandits.
problem Analyzing bias in sample means of multi-armed bandits.
method Decoupling three sources of bias: sampling, stopping, choosing; using optimism to capture monotonic behaviors.
result Optimistic sampling induces negative bias, while optimistic stopping and choosing induce positive bias.
New RL algorithm optimizes policies with bandit feedback, matching previous bounds.
problem Optimizing policies with unknown transitions and bandit feedback.
method Optimistic Trust Region Policy Optimization (TRPO) algorithm.
result Sub-linear regret bounds for both stochastic and adversarial rewards.
New RL algorithm tackles online robust MDPs with uncertainty.
problem Developing robust reinforcement learning models for real-world environments.
method Proposes a robust optimistic policy optimization algorithm for online robust MDPs.
result Establishes the first regret bound for online robust MDPs.
This paper develops a model of reference-dependent assessment of subjective beliefs in which loss-averse people optimally choose the expectation as the reference point to balance the current felicity from the optimistic anticipation and the future disappointment from the realisation. The choice of over-optimism or over…