New algorithms improve contextual bandit performance by adapting to problem difficulty.
problem Improving contextual bandit performance on problems with varying difficulty.
method Introducing complexity measures and oracle-efficient algorithms.
result Achieves optimal instance-dependent regret bounds for rich policy classes.
Paper establishes first instance-dependent lower bound for PAC reinforcement learning.
problem Identifying near-optimal policies in tabular MDPs with minimal samples.
method Proposes instance-dependent lower bound for sample complexity.
result Lower bound closely matches PEDEL algorithm's sample complexity.
New reinforcement learning algorithm achieves instance-optimal sample complexity.
problem Achieving low regret and identifying optimal policies in reinforcement learning.
method A novel planning-based algorithm that explicitly accounts for state visitation distributions.
result The proposed algorithm attains nearly minimax optimal sample complexity, improving over worst-case bounds.
Algorithm extsc{Pedel} learns near-optimal policies efficiently on specific problems.
problem Learning near-optimal policies in linear MDPs with minimal samples.
method Online experiment design to focus exploration on relevant directions.
result Achieves instance-dependent complexity, outperforming minimax-optimal algorithms.
BeGIN benchmarks GNNs for instance-dependent label noise in graphs.
problem Instance-dependent label noise in graph data.
method BeGIN introduces a benchmark with various noise types and evaluates noise-handling strategies across GNN architectures.
result Challenges of instance-dependent noise, especially LLM-based corruption, and the importance of node-specific parameterization.
We consider PAC-learning a good item from k k k -subsetwise feedback information sampled from a Plackett-Luce probability model, with instance-dependent sample complexity performance. In the setting where subsets of a fixed size can be tested and top-ranked feedback is made available to the learner, we give an algorithm w…
This work improves the convergence theory of diffusion models for generating samples from complex distributions.
problem Improving theoretical understanding of diffusion models, particularly their convergence analysis.
method Developed an instance-dependent convergence rate that adapts to the smoothness of target distributions.
result Established an iteration complexity of min { d , d 2 / 3 L 1 / 3 , d 1 / 3 L } ε − 2 / 3 \min\{d,d^{2/3}L^{1/3},d^{1/3}L\}\varepsilon^{-2/3} min { d , d 2/3 L 1/3 , d 1/3 L } ε − 2/3 for generating high-quality samples. Paper tackles instance-dependent label noise by approximating it with part-dependent noise.
problem Learning with instance-dependent label noise is challenging.
method Approximate instance-dependent label noise with part-dependent noise. Use transition matrices for parts to model noise.
result Method outperforms state-of-the-art approaches for instance-dependent label noise.
A new SSL method uses instance-dependent thresholds to improve accuracy.
problem Improving semi-supervised learning by better selecting confident unlabeled instances.
method Proposes instance-dependent thresholds that vary based on the ambiguity and error rates of pseudo-labels for each unlabeled instance.
result Demonstrates that instance-dependent thresholds provide a probabilistic guarantee for correct pseudo-labels.
UCRL-WVTR tackles long-term reinforcement learning with general approximations, achieving horizon-free and instance-dependent regret bounds.
problem Long-term reinforcement learning with general function approximations.
method UCRL-WVTR proposes a novel algorithm, UCRL-WVTR, with weighted value-targeted regression and a high-order moment estimator.
result Achieves horizon-free and instance-dependent regret bounds matching minimax lower bounds up to logarithmic factors.
New algorithm optimally evaluates policies with linear approximations.
problem Policy evaluation with linear function approximation.
method Accelerated, variance-reduced fast temporal difference algorithm (VRFTD).
result VRFTD matches both deterministic and stochastic lower bounds.
Paper bounds PAC RL sample complexity in deterministic MDPs.
problem Identify ε-optimal policy with high probability.
method Proposes nearly matching upper and lower bounds on sample complexity, introduces deterministic return gap, uses graph-theoretical concepts and maximum-coverage exploration.
result First nearly matching upper and lower bounds on sample complexity for PAC RL in deterministic MDPs.
New methods improve temporal difference learning for policy evaluation in Markov decision processes.
problem Improving temporal difference learning for policy evaluation in Markov decision processes.
method Introduced variance-reduced forms of stochastic approximation to achieve non-asymptotic, instance-dependent optimality.
result Temporal difference learning is strictly suboptimal, but variance-reduced forms achieve optimality up to logarithmic factors.
The research proposes a stopping rule for reinforcement learning algorithms based on instance-dependent confidence.
problem Dramatic variation in convergence rates of reinforcement learning algorithms due to problem structure.
method Develops instance-dependent confidence regions and a data-dependent stopping rule for MDP policy evaluation and optimal value estimation.
result Proposes a stopping rule that adapts to the instance-specific difficulty of the problem, allowing for early termination.
New CTRL algorithm adapts to varying problem difficulty.
problem Adapting to varying levels of problem difficulty in CTRL.
method MLE with a general function approximator, estimating state marginal density.
result Regret bound scales with reward variance and measurement resolution, independent of measurement strategy.
New RLHF algorithm identifies optimal policies from human feedback without explicit reward inference.
problem Training large language models with human feedback without reward inference.
method Model-free RLHF algorithm B S A D \mathsf{BSAD} BSAD that identifies optimal policies directly from human preference. result Provable, instance-dependent sample complexity i l d e O ( c M S A 3 H 3 M log 1 δ ) ilde{\mathcal{O}}(c_{\mathcal{M}}SA^3H^3M\log\frac{1}δ) i l d e O ( c M S A 3 H 3 M log δ 1 ) . CORES2 removes noisy labels by sieving out corrupted examples.
problem Instance-dependent label noise degrades DNN performance.
method CORES2 (COnfidence REgularized Sample Sieve) progressively sieves out corrupted examples.
result CORES2 provides theoretical guarantees for filtering out corrupted examples.
Study optimizes solving fixed-point equations using subspace search.
problem Solving linear fixed point equations in Hilbert spaces.
method Linear stochastic approximation scheme with Polyak--Ruppert averaging.
result Established optimal approximation factor for temporal difference learning methods.
Paper proposes a universal probabilistic model for handling instance-dependent label noise.
problem Instance-dependent label noise in data quality challenges DNN training robustness.
method Categorizes instances into confusing and unconfusing, proposes a probabilistic model.
result Significant improvements in robustness over state-of-the-art methods on various datasets.
No communication allows optimal instance-dependent regret guarantees in multi-player bandits.
problem Achieving optimal instance-dependent regret in multi-player multi-armed bandits without communication.
method Characterization of Pareto optimal trade-offs and development of an algorithm.
result Achieving optimal instance-dependent regret requires strict sub-optimality in other regimes.
Papers learn from data to make decisions without interacting, improving on previous methods.
problem Achieving optimal decision-making from offline data with non-linear function approximation.
method Pessimistic Nonlinear Least-Square Value Iteration (PNLSVI) with three innovative components.
result Achieves minimax optimal instance-dependent regret for non-linear function approximation.
A method uses confidence scores to handle noisy labels for each instance.
problem Learning with noisy labels where each instance's label can randomly change.
method Introduces confidence-scored instance-dependent noise (CSIDN) to estimate transition distributions for each instance.
result Demonstrates the utility and effectiveness of CSIDN through experiments with synthetic and real-world noise.
A method to approximate instance-dependent label noise using instance-confidence embedding.
problem Real-world label noise that depends on individual instances.
method Variational approximation with instance embedding to capture instance-specific label corruption.
result ICE method effectively approximates instance-dependent noise and detects ambiguous instances.
Study optimal and instance-dependent guarantees for solving linear equations with Markovian data.
problem Approximately solving linear fixed point equations with Markovian data.
method Non-asymptotic bounds and instance-dependent characterizations for stochastic approximation.
result Instance-optimality of the averaged SA estimator and matching upper and lower bounds.
New algorithm tackles heavy-tailed rewards in RL with instance-dependent regret bounds.
problem Efficient algorithms for RL with heavy-tailed rewards in large state-action spaces.
method Design of \textsc{Heavy-OFUL} for heavy-tailed linear bandits and \textsc{Heavy-LSVI-UCB} for RL with linear function approximation.
result First instance-dependent regret bounds for heavy-tailed rewards in RL with linear function approximation.
Open problem: fixed-budget best arm identification complexity.
problem Understanding the complexity of identifying the best arm in a fixed budget setting.
method Analyzing existing results and conjectures in the fixed-confidence setting.
result Open questions remain about the fixed-budget setting.
New algorithm reduces contextual bandit identification to argmax calls.
problem Best-arm identification in stochastic contextual bandits.
method Instance-optimal PAC algorithm using argmax oracle calls.
result First instance-dependent PAC sample complexity for contextual bandits.
We study the problem of hypothesis testing between two discrete distributions, where we only have access to samples after the action of a known reversible Markov chain, playing the role of noise. We derive instance-dependent minimax rates for the sample complexity of this problem, and show how its dependence in time is…
CODA uses a new dropout technique inspired by constructivism learning to improve deep learning performance.
problem Existing dropout methods fail to differentiate among instances, leading to overfitting.
method CODA incorporates structural information and uses a Bayesian nonparametric method to create a better dropout technique.
result CODA outperformed other state-of-the-art dropout techniques on 5 real-world datasets.
Improved stochastic optimization outperforms standard methods.
problem Optimizing smooth, strongly convex functions with noisy data.
method Variance reduction strategy called VISOR.
result VISOR achieves optimal sample complexity and oracle complexity.
New method for PU learning with instance-dependent propensity scores.
problem Learning from positive and unlabeled data with instance-dependent labeling.
method Empirical risk minimization of joint risk function, alternating optimization of posterior probability and propensity score.
result The method achieves comparable or better performance than state-of-the-art methods.
VRPG algorithm optimizes convex constraints with non-asymptotic guarantees.
problem Stochastic convex optimization under convex constraints.
method Natural variance reduced proximal gradient (VRPG) algorithm.
result VRPG achieves local minimax lower bound up to constants and log factor of N N N . The study analyzes batched methods for early stopping in stochastic multi-armed bandits.
problem Early stopping in stochastic multi-armed bandits with fixed confidence.
method Instance-dependent lower bounds and a general batched algorithm with upper bounds.
result Upper and lower bounds on the number of batches and sample complexity.
Two algorithms tackle heavy-tailed rewards in reinforcement learning with linear function approximation.
problem Online sequential decision-making with heavy-tailed rewards.
method AdaOFUL and VARA algorithms for linear stochastic bandits and MDPs, using modified adaptive Huber regression.
result Achieved state-of-the-art and variance-aware regret bounds for heavy-tailed rewards.
New algorithm reduces regret and constraint violation in online convex optimization with complex constraints.
problem Online convex optimization with multiple functional constraints and a simple constraint set.
method Instance-dependent bound using online primal-dual mirror-prox algorithm in general normed spaces.
result Achieves an O(√V*(T)) regret and O(1) constraint violation, improving over previous works.
This paper identifies and estimates the label noise transition matrix without ground truth labels.
problem Learning with noisy labels and identifying the noise transition matrix.
method Building on Kruskal's identifiability results, the paper characterizes the identifiability of the label noise transition matrix for the generic case at the instance level.
result The necessity of multiple noisy labels in identifying the noise transition matrix for the generic case at the instance level.
New algorithm identifies best arm in semiparametric bandits with near optimal efficiency.
problem Fixed-confidence Best Arm Identification in semiparametric bandits with unknown baseline shift.
method Phase-elimination algorithm based on orthogonalized regression design.
result Nearly optimal high-probability sample-complexity upper bound established.
Algorithm reduces regret in multi-player bandits with unknown collision rewards.
problem Reducing regret in multi-player multi-armed bandits with unknown collision rewards.
method Proposes an algorithm that combines a modified successive elimination strategy with a communication protocol to estimate suboptimality gaps and coordinate among players.
result Achieves logarithmic regret for the problem when collision reward is unknown.
Study contextual online pricing with biased offline data, achieving optimal regret bounds.
problem Contextual online pricing with biased offline data.
method Identify δ 2 δ^2 δ 2 to measure data bias, use OFU policy and robust variant for unknown bias. result Achieve minimax-optimal regret bounds for contextual pricing.
Optimizes regret distribution in stochastic bandits for risk balance.
problem Balancing regret expectation and tail risk in stochastic bandits.
method Characterizes optimal regret tail probability for any threshold, proposes new policies.
result Discovers an intrinsic gap in optimal tail rate based on time horizon uncertainty.
New algorithm reduces regret for kernelized bandits by adapting to specific problem instances.
problem Efficiently learning the optimizer of an unknown function in RKHS with noisy oracle.
method Instance-dependent regret analysis and a new minimax near-optimal algorithm.
result New algorithm achieves better performance on specific problem instances.
New algorithms for model selection in linear bandits adapt to instance complexity.
problem Adapting to the instance-dependent complexity of the true model in linear bandits.
method Design of algorithms in fixed confidence and fixed budget settings, leveraging experimental design and selection-validation procedures.
result Near instance optimal guarantees for model selection in linear bandits.
New Q-learning method achieves optimal sample complexity for average-reward problems.
problem Challenges in achieving optimal sample complexity for average-reward Q-learning.
method Synchronous and asynchronous Q-learning with a new contraction principle.
result Optimal O ~ ( ε − 2 ) \widetilde{O}(\varepsilon^{-2}) O ( ε − 2 ) sample complexity guarantees. Algorithm identifies nearest mode in noisy data.
problem Identifying the point with the minimum k-th nearest neighbor distance in unknown multivariate probability density.
method Sequential learning algorithm using noisy oracle queries to adaptively decide which points to query.
result Upper bounds on query complexity show significant improvement over baselines.
Study non-asymptotic BPI guarantees for online RL.
problem Identify optimal policy in MDP with high confidence.
method Non-asymptotic sample complexity guarantees for NaS algorithm.
result Sample complexity depends on MDP connectivity and curvature.
Estimates Markov chain mixing time from a single trajectory.
problem Estimating mixing time of Markov chains from a single trajectory.
method Contraction with respect to total variation, inspired by Wolfer's contraction coefficient.
result Improved confidence intervals and instance-dependent rates for estimating Markov chains.
New bounds explain neural network generalization by considering local Lipschitz properties.
problem Existing bounds fail to account for initialization and SGD biases.
method Optimal transport interpretation of generalization problem.
result Instance-dependent bounds that depend on local Lipschitz regularity.
RL-LOW algorithm achieves exponential simple regret in offline RLHF with pairwise comparisons.
problem Offline reinforcement learning from human feedback with pairwise comparisons.
method Proposes RL-LOW algorithm to minimize simple regret with exponential convergence.
result Achieves an exponential form of simple regret of \(\exp ( - Ω(n/H) )\).