Algorithm learns to switch control among agents in a team.
problem Learning to switch control among reinforcement learning agents.
method 2-layer Markov decision process, upper confidence bounds, shared confidence bounds.
result Sublinear total regret with shared confidence bounds.
UCB-TQL learns from multiple tasks with shared dynamics and adapts to task-specific variations.
problem Transfer reinforcement learning with composite MDPs where tasks share core dynamics but have sparse differences.
method UCB-TQL, a novel transfer RL algorithm for composite MDPs.
result Achieved a regret bound of i l d e O ( e H 5 N ) ilde{O}(\sqrt{eH^5N}) i l d e O ( e H 5 N ) that scales independently of the ambient dimension. CRB tackles rising rewards in combinatorial online learning.
problem Rising rewards in combinatorial online learning.
method CRB framework and CRUCB algorithm.
result Empirical and theoretical validation of CRUCB's effectiveness.
Algorithm aggregates rewards from multiple players to learn related tasks in online bandit learning.
problem Learning related but slightly different tasks in an online setting with heterogeneous feedback.
method RobustAgg ( ε ) (ε) ( ε ) algorithm that aggregates rewards from different players. result Achieves instance-dependent regret guarantees and nearly matching lower bounds.
New tighter confidence bounds for sequential kernel regression.
problem Quantifying uncertainty in sequential learning algorithms.
method Martingale tail inequalities and conic programming.
result New confidence bounds are tighter than existing ones.
Efficiently estimates Cox model coefficients without sharing data.
problem Privacy and ownership concerns in multi-center biomedical studies.
method Communication-efficient iterative distributed algorithms for estimation and inference.
result Achieves convergence rate of full-sample estimator with minimal iterations.
The design of personalized incentives or recommendations to improve user engagement is gaining prominence as digital platform providers continually emerge. We propose a multi-armed bandit framework for matching incentives to users, whose preferences are unknown a priori and evolving dynamically in time, in a resource c…
A new differentiable UCB algorithm for linear bandits learns adaptive confidence bounds.
problem Inability of UCB to strike optimal exploration-exploitation due to confidence bounds.
method Proposes a differentiable linear bandit algorithm and a gradient estimator for learning adaptive confidence bounds.
result Achieves a i l d e O ( β ^ d T ) ilde{\mathcal{O}}(\hatβ\sqrt{dT}) i l d e O ( β ^ d T ) upper bound of T T T -round regret. The paper honors Lai's contributions to multi-armed bandits and establishes new regret bounds.
problem Improving regret bounds in multi-armed bandit problems.
method Establishes non-asymptotic regret bounds for upper confidence bound indices.
result New regret bounds match Lai-Robbins lower bound.
The paper develops adaptive confidence intervals for Efron's Gaussian two-groups model with unknown contamination.
problem Developing robust uncertainty quantification for Efron's Gaussian two-groups model with unknown contamination fraction.
method The approach involves Fourier-based certification procedures to find minimax-optimal adaptive confidence intervals.
result The minimax-optimal length of adaptive confidence intervals is polynomially worse than when contamination fraction is known.
The paper extends confidence sequences for infinite variance data.
problem Addressing confidence sequences for distributions with infinite variance.
method Establishing lower bounds and deriving tight confidence sequences for relaxed bounded p t h p^{th} p t h -moment distributions. result Derived confidence sequences are tighter than those using Dubins-Savage inequality.
Improved confidence bounds for linear logistic model with applications to bandits.
problem Improving confidence bounds for linear logistic model.
method Self-concordant analysis of the logistic loss to avoid dependence on worst-case variance.
result Significant improvement in confidence bounds, avoiding dependence on 1 / κ 1/κ 1/ κ . Research aims to improve confidence intervals for RKHS elements in online learning.
problem Improper confidence intervals lead to suboptimal regret bounds in kernel-based bandit and reinforcement learning.
method Formalizes the open problem of online confidence intervals in RKHS and reviews existing results.
result Identifies the online nature of observation points as the main challenge for tight confidence intervals.
Near-optimal confidence intervals for bounded data.
problem Online inference for sequential decision problems like A/B testing.
method Utilizing Bentkus' concentration results to improve on existing methods.
result Near-optimal confidence intervals confirmed favorable in synthetic and practical applications.
A new framework tightens risk measure confidence bounds.
problem Improving confidence bounds for various risk measures.
method Distribution optimization framework with two estimation schemes based on concentration bounds.
result Consistently tighter confidence bounds compared to previous methods.
Note improves confidence bounds for random variables.
problem Improving confidence bounds for random variables with unbounded ranges and different distributions.
method PAC-Bayes-ification of a derived confidence bound.
result Streamlined proofs for existing results.
Improved algorithms for stochastic linear bandits using tighter confidence sequences.
problem Stochastic linear bandits with improved worst-case regret guarantees.
method Novel tail bound for adaptive martingale mixtures to construct tighter confidence sequences.
result Linear bandit algorithm achieves competitive worst-case regret.
Paper presents robust confidence sequences for means with known moment bounds and arbitrary corruption.
problem Tackles robustness to outliers and adversarial corruptions in mean estimation.
method Designs new robust exponential supermartingales to create confidence sequences.
result Achieves optimal width and shows smaller margin of error compared to fixed-time robust methods.
The paper finds a fundamental trade-off between confidence and efficiency in transductive conformal prediction.
problem The challenge is to balance confidence and efficiency in predicting multiple data points.
method The authors derive a strict finite-sample bound and introduce a practical algorithm to approach this bound.
result Any non-trivial confidence level leads to exponential growth in prediction set size, with a linear scaling in the number of samples.
New methods improve uncertainty in machine learning predictions for asset returns.
problem Uncertainty in machine learning predictions for asset returns.
method Developed new methods to construct forecast confidence intervals for expected returns from neural networks.
result Neural network forecasts of expected returns have the same asymptotic distribution as classic nonparametric methods, enabling standard error calculation.
This paper improves GP-UCB by using a shifted exponential distribution for confidence parameters.
problem Theoretical confidence parameter in GP-UCB increases with iterations, leading to large values.
method Introduced IRGP-UCB, a randomized variant of GP-UCB using a shifted exponential distribution for confidence parameters.
result IRGP-UCB achieves sub-linear regret without increasing the confidence parameter.
Improved online confidence bounds for multinomial logistic models in bandits.
problem Achieving optimal regret in multinomial logistic bandits with bounded parameters and outcomes.
method Deriving an improved online confidence bound and proposing OFU-MNL++ and OFU-MN 2 ^2 2 L algorithms. result Achieved variance-dependent optimal regret for MNL bandits.
The paper presents a method to compute trusted confidence bounds for LECs in CPS.
problem Non-transparent predictions of LECs make CPS safety challenging.
method Inductive Conformal Prediction (ICP) and Triplet Network architecture.
result Efficient real-time computation of trusted confidence bounds.
Develops confidence bounds for off-policy evaluation in contextual bandits.
problem Evaluating policies that were not used to collect data.
method Martingale analysis for non-asymptotic, non-parametric, and valid confidence sequences.
result Empirically tight bounds on failure probability and width.
New algorithms optimize multiple tasks with shared similarities, reducing regret.
problem Optimizing multiple objectives with shared similarities in non-parametric Bayesian optimization.
method Developed two novel BO algorithms using multi-task kernels and random scalarizations.
result Derived worst-case regret bounds capturing inter-task similarities.
In the field of reinforcement learning there has been recent progress towards safety and high-confidence bounds on policy performance. However, to our knowledge, no practical methods exist for determining high-confidence policy performance bounds in the inverse reinforcement learning setting---where the true reward fun…
CONTINA provides adaptive confidence intervals for traffic demand prediction.
problem Uncertainty in future traffic demand predictions and the need for valid confidence intervals.
method Adaptive confidence interval method that adjusts based on deployment errors.
result Valid confidence intervals with shorter lengths and theoretical coverage guarantee.
The paper improves confidence regions for band-limited functions using tighter norm bounds and majority voting.
problem Constructing reliable confidence regions for band-limited functions from noisy data.
method Improved norm bounds using Hoeffding's inequality and empirical Bernstein bound, majority voting to aggregate intervals.
result Confidence intervals retain their simultaneous coverage guarantee even when aggregated from random subsamples.
This paper improves multi-class calibration methods using mutual information maximization-based binning.
problem Calibration of deep neural network predictions, especially for small prior classes.
method I-Max concept for binning, shared class-wise calibration strategy.
result Improves multi-class ranking and calibration performance using a small calibration set.
Paper improves confidence intervals and variance estimation for deep learning models.
problem Improving confidence intervals and variance estimation in deep learning models.
method Residual-based framework for conditional variance estimation; robust bootstrap procedure for confidence intervals.
result First non-asymptotic bounds for variance estimation using ReLU networks.
The estimation of probabilities of default (PDs) for low default portfolios by means of upper confidence bounds is a well established procedure in many financial institutions. However, there are often discussions within the institutions or between institutions and supervisors about which confidence level to use for the…
We provide a pointwise confidence bound for non-linear least-squares with fixed design.
problem Confidence estimation in non-linear ℓ 2 \ell^2 ℓ 2 -regularized least squares. method Pointwise confidence bound for local minimizers, using weighted norm involving inverse-Hessian.
result The proposed confidence bound scales with the test input's similarity to the training data.
Paper improves regret bounds for Gaussian process upper confidence bound in Bayesian optimization.
problem Minimizing regret in Gaussian process bandit optimization.
method Gaussian process upper confidence bound (GP-UCB) algorithm with refined analysis.
result Achieves O ( T ln 2 T ) O(\sqrt{T \ln^2 T}) O ( T ln 2 T ) cumulative regret under squared exponential kernel. Paper provides new bounds for risk aggregation and sharing.
problem Quantitative risk management and robust risk aggregation with dependence uncertainty.
method Established new inequality for RVaR, derived extended convolution bounds, and analyzed risk sharing for averaged quantiles.
result Extended convolution bounds for robust risk aggregation and risk sharing, providing sharpness conditions and explicit expressions.
For an autonomous agent, executing a poor policy may be costly or even dangerous. For such agents, it is desirable to determine confidence interval lower bounds on the performance of any given policy without executing said policy. Current methods for exact high confidence off-policy evaluation that use importance sampl…
Proposes a method to compute valid lower confidence bounds for multiple models selected based on their performance.
problem Model selection and evaluation in machine learning.
method Interprets model selection as a simultaneous inference problem, uses bootstrap tilting and maxT-type multiplicity correction.
result Yields valid lower confidence bounds that are at least as good as standard approaches and reliably reach nominal coverage probability.
Paper improves CI and CS for bounded means using betting and mixtures.
problem Estimating means of bounded random variables.
method Composite nonnegative martingales, testing by betting, method of mixtures.
result Empirically outperforms existing CI and CS methods.
Improved regret bounds for logistic bandits via novel confidence set construction.
problem Dependencies in parameter space for logistic bandits, especially when S ≥ d S \geq d S ≥ d . method Regret-to-confidence-set conversion (R2CS) to construct convex confidence sets.
result Strict improvement in regret bound w.r.t. S S S in logistic bandits. This chapter reviews statistical tools for reinforcement learning.
problem Applying RL algorithms in healthcare and ride-sharing platforms.
method Statistical inference tools for RL, including hypothesis testing and confidence interval construction.
result Highlighting the value of statistical inference in RL for both communities.
The paper improves confidence ellipsoids for ridge regression with PAC bounds.
problem Uncertainty quantification in ridge regression for insufficiently exciting inputs.
method Extension of SPS EOA algorithm to ridge regression with PAC bounds.
result Explicitly shows how regularization parameter affects region sizes and provides tighter bounds.
We propose an estimator and confidence interval for computing the value of a policy from off-policy data in the contextual bandit setting. To this end we apply empirical likelihood techniques to formulate our estimator and confidence interval as simple convex optimization problems. Using the lower bound of our confiden…
Improved Bayesian optimisation method using randomised Gaussian process UCB.
problem Improving performance in Bayesian optimisation.
method Developed a modified Gaussian process upper confidence bound (GP-UCB) acquisition function.
result The method achieves better performance than GP-UCB in various problems.
WS-KDE provides robust confidence bounds for stochastic functions.
problem Optimizing time-consuming black-box functions with stochastic outputs.
method Wilson Score Kernel Density Estimation (WS-KDE) for Bayesian optimization.
result WS-KDE provides reliable confidence bounds for any stochastic function.
Confidence measures for the generalization error are crucial when small training samples are used to construct classifiers. A common approach is to estimate the generalization error by resampling and then assume the resampled estimator follows a known distribution to form a confidence set [Kohavi 1995, Martin 1996,Yang…
This note gives a short, self-contained, proof of a sharp connection between Gittins indices and Bayesian upper confidence bound algorithms. I consider a Gaussian multi-armed bandit problem with discount factor γ γ γ . The Gittins index of an arm is shown to equal the γ γ γ -quantile of the posterior distribution of the arm'…
Biondi et al. (2012) develop an analytical model to examine the emergent dynamic properties of share market price formation over time, capable to capture important stylized facts. These latter properties prove to be sensitive to regulatory regimes for fundamental information provision, as well as to market confidence c…
Bayesian methods improve drug discovery experiment design.
problem Optimizing drug screening experiments in high-dimensional data.
method Bayesian inference and optimisation with upper confidence bound algorithms, Thompson sampling, and sparse tree search.
result Sparse tree search techniques outperform other methods in drug toxicity screening.
Low-rank framework for task-specific LLM ranking from sparse comparisons.
problem Challenges in reliable task-specific ranking of LLMs under sparse, imbalanced comparisons.
method Low-rank modeling of task-by-model ability matrix, max-norm accurate estimator, task-wise top-K recovery guarantees, uncertainty quantification framework.
result Improves sample efficiency and produces tighter, better-calibrated ranking certificates.