This paper studies Thompson sampling's arm-pull dynamics and inference, revealing key differences from UCB algorithms.
problem Understanding the precise arm-pull dynamics in Thompson sampling algorithms.
method Developed new approaches to analyze the arm-pull count process and noise processes, including inverse process and reparametrization methods.
result Arm-pull count is asymptotically deterministic only for suboptimal or unique optimal arms, revealing a unifying principle of stability.
Develops a framework for clustering and distribution matching with bandit feedback.
problem Clustering and distribution matching problems with limited feedback.
method General framework using K K K -armed bandit model, Track-and-Stop method, and Frank--Wolfe algorithm. result Average number of arm pulls matches lower bound, with asymptotic convergence to fundamental limit.
UCB-V algorithm improves on UCB for MAB problems with variance estimates.
problem Optimizing arm selection in MAB problems with variance information.
method Asymptotic and high probability analysis of UCB-V algorithm.
result UCB-V can exhibit instability in arm-pulling rates but achieves refined regret bounds.
We extend the classic multi-armed bandit (MAB) model to the setting of noncompliance, where the arm pull is a mere instrument and the treatment applied may differ from it, which gives rise to the instrument-armed bandit (IAB) problem. The IAB setting is relevant whenever the experimental units are human since free will…
New MAB model incentivizes user arm-pulling with self-reinforcing preferences.
problem Balancing exploration and exploitation in recommender systems with incentivized user preferences.
method Proposes a new MAB model with random arm selection and two policies: At-Least- n n n Explore-Then-Commit and UCB-List. result Achieves O ( l o g T ) O(log T) O ( l o g T ) expected regret and O ( l o g T ) O(log T) O ( l o g T ) expected payment over a time horizon T T T . UCB algorithms improve on bandit problems with precise regret analysis and adaptive inference.
problem Understanding the performance and statistical inference of UCB algorithms in multi-armed bandit problems.
method Deterministic characterization of arm pulls and precise regret analysis.
result UCB algorithms' maximal regret deviates from minimax regret by a logarithmic factor, and the Lai-Robbins formula is exact only under specific conditions.
Motivated by drug design, we consider the best-arm identification problem in generalized linear bandits. More specifically, we assume each arm has a vector of covariates, there is an unknown vector of parameters that is common across the arms, and a generalized linear model captures the dependence of rewards on the cov…
New algorithm tackles resource allocation in multi-armed bandits to balance speed and throughput.
problem Balancing speed and throughput in stochastic multi-armed bandits with limited resources.
method Proposes an algorithm that trades off between information accumulation and throughput.
result Upper bounds the time taken to find the best arm with a given target success probability.
Algorithm identifies Pareto set in bandits with contaminated feedback.
problem Identifying Pareto set in multi-objective bandits with adversarial contamination.
method Sample median-based multi-objective adaptive elimination algorithm.
result Sample complexity bound that depends on contamination probability.
Learning preferences implicit in the choices humans make is a well studied problem in both economics and computer science. However, most work makes the assumption that humans are acting (noisily) optimally with respect to their preferences. Such approaches can fail when people are themselves learning about what they wa…
UCB algorithm provides stable sample means for sequential data.
problem Challenges in inferential tasks with sequential data.
method Stability property of UCB algorithm for multiarmed bandit problems.
result UCB algorithm ensures asymptotically normal sample means.
A new algorithm improves sample complexity for thresholding in Monte Carlo Tree Search.
problem Determining if the root node value of a tree is at least a given threshold.
method Developed a δ-correct sequential sampling algorithm based on the Track-and-Stop strategy.
result Ratio-based modification of D-Tracking strategy reduces sample complexity and computational cost.
This paper studies active learning in the context of robust statistics. Specifically, we propose a variant of the Best Arm Identification problem for \emph{contaminated bandits}, where each arm pull has probability ε \varepsilon ε of generating a sample from an arbitrary contamination distribution instead of the true und…
Study quantile reward identification with 1-bit feedback constraints.
problem Best arm identification with quantile reward and 1-bit communication.
method Proposes an algorithm using noisy binary search for quantile reward estimation.
result Derives upper and lower bounds on sample complexity for 1-bit feedback.
The paper tackles a bandit problem with infinitely many arms per group, aiming to identify the group with the highest quantile reward.
problem Max-quantile group bandit problem with infinitely many arms per group.
method Two-step algorithm: first request arms from each group, then apply a finite-arm max-quantile bandit algorithm.
result Characterization of instance-dependent and worst-case regret, with matching lower bounds.
Optimizing rewards under budget constraints with correlated costs and rewards.
problem Maximizing total expected reward under a budget constraint on total cost with correlated and potentially heavy-tailed cost-reward pairs.
method Proposes algorithms exploiting correlation between cost and reward via linear minimum mean-square error estimation to achieve tight regret bounds.
result Achieves O ( log B ) O(\log B) O ( log B ) regret for a budget B > 0 B>0 B > 0 under certain moment conditions. New algorithms identify best arm with less pulls, adapting to arm covariances.
problem Best arm identification under dependent and correlated arm distributions.
method Adaptive algorithms estimating arm covariances to minimize pulls.
result Substantial improvement in best arm identification over standard setting.
New algorithm finds high-reward combinatorial sets with fewest pulls.
problem Finding high-reward combinatorial sets with unknown individual arm rewards.
method Successive acceptance and elimination based on combinatorial structure.
result Algorithm requires minimal combinatorial oracle calls, making it practical for large problems.
A new algorithm for fair decision-making in bandit problems with biased feedback.
problem Fair allocation of resources across sensitive groups in bandit problems with biased feedback.
method Proposes a novel algorithm that learns a societal bias term to mitigate unfairness in multi-armed bandit problems.
result The proposed algorithm achieves a theoretical bound on regret and demonstrates fairness in synthetic and real-world data.
New algorithm identifies optimal actions in large reward spaces efficiently.
problem Finding the best action from a large set of options with minimal trials.
method GenTS-Explore algorithm for real-valued combinatorial pure exploration.
result Achieves optimal sample complexity for large action sets.
New method identifies cluster representatives with minimal pulls.
problem Identifying cluster representatives in multi-armed bandits.
method Fixed confidence approach using confidence intervals.
result Sample complexity matches theoretical lower bound.
Paper optimizes clustering with bandit feedback, reducing errors.
problem Online clustering with bandit feedback, minimizing errors.
method Instance-dependent information-theoretic lower bound, BOC algorithm with adaptive stopping rule.
result BOC algorithm matches lower bound asymptotically and significantly outperforms non-adaptive baseline.
New algorithms improve stopping time for best arm identification.
problem Efficiently identifying the best alternative in experiments.
method Proposed algorithms with exponential-tailed stopping time.
result Proved that some algorithms never stop, leading to new methods.
New algorithms for streaming bandits with limited memory.
problem Optimizing decisions in a stream of uncertain outcomes with limited memory.
method Developed algorithms for minimizing regret and identifying the best arm under bounded memory constraints.
result Upper and lower bounds on sample complexity for best-arm identification algorithms.
A bandit algorithm reduces regret in noisy, communication-constrained feedback.
problem Distributed stochastic multi-armed bandit with noisy, communication-constrained feedback.
method Proposes a multi-phase bandit algorithm, UE-UCB++, that matches an information-theoretic lower bound.
result Matches an information-theoretic lower bound of Ω(√(KT/σ²)) on the minimax regret.
New study shows feedback doesn't improve MAB performance over erasure channels.
problem Distributed multi-armed bandit problem with arm erasure channels.
method Analyzed MAB with feedback and erasure channels, proving no improvement in regret.
result Regret lower bound of Ω(√KT + K / (1 - ε)) matches no-feedback upper bounds.
A federated learning algorithm tackles linear bandits with adversarial actions, achieving optimal regret bounds.
problem Federated linear bandits with finite adversarial action sets.
method FedSupLinUCB algorithm, extending SupLinUCB and OFUL principles.
result Achieves a total regret of i l d e O ( d T ) ilde{O}(\sqrt{d T}) i l d e O ( d T ) , matching minimax lower bound and being order-optimal. New algorithm CROP achieves asymptotic optimality with bounded regret.
problem Optimistic algorithms fail to achieve asymptotic instance-dependent regret optimality.
method CRush Optimism with Pessimism (CROP) algorithm that eliminates optimistic hypotheses.
result CROP achieves constant-factor asymptotic optimality and bounded regret.
In this paper, we introduce a distributed version of the classical stochastic Multi-Arm Bandit (MAB) problem. Our setting consists of a large number of agents n n n that collaboratively and simultaneously solve the same instance of K K K armed MAB to minimize the average cumulative regret over all agents. The agents can co…
Opt-BBAI identifies the best arm with minimal batches and pulls, optimizing both sample and batch complexity.
problem Batched best arm identification (BBAI) problem, aiming to minimize policy switches and resource usage.
method Proposed Opt-BBAI algorithm, achieving near-optimal sample and batch complexity in non-asymptotic settings.
result First algorithm to achieve near-optimal sample and batch complexity in non-asymptotic settings.
New algorithm tackles multi-armed bandit with arbitrary delays and general bounded losses.
problem Scale-free adversarial multi-armed bandit with arbitrary feedback delays.
method SFD-INF combines convex combination trick and doubling/skipping technique.
result Achieves adaptive regret bounds for non-negative and general scale-free losses.
New model for multi-armed bandits with growing arms.
problem Balancing exploration and exploitation in a growing set of arms.
method Introduces Ballooning Multi-Armed Bandits (BL-MAB) and analyzes existing algorithms.
result Achieves sub-linear regret under certain conditions.
PAC best arm identification with a deadline, improving efficiency over sequential methods.
problem Identifying an optimal arm under a fixed time constraint.
method Elastic Batch Racing (EBR) algorithm for ( ε , δ ) (ε, δ) ( ε , δ ) -PAC best arm identification under a deadline. result EBR is optimal with respect to two hardness results and outperforms sequential methods.
New algorithm uses machine learning to predict rewards for decision-making problems.
problem Sequential decision-making under uncertainty with scarce online data.
method Machine Learning-Assisted Upper Confidence Bound (MLA-UCB) algorithm.
result Proves to improve cumulative regret even with biased surrogate rewards.
New algorithm tackles batched stochastic linear bandits with 1-bit communication constraints.
problem Stochastic linear bandits with 1-bit communication constraints.
method Phased-elimination algorithms based on G-optimal designs and 1-bit mean estimation.
result Achieves near-optimal regret bounds for broad scaling regimes.
Paper shows leafwise cohomological expression for dynamical zeta functions.
problem Analyzing dynamical zeta functions on foliated dynamical systems.
method Leafwise cohomological approach.
result Leafwise cohomological expression of dynamical zeta functions.
Study on 2-valued dynamics on complex plane, showing some dynamics can't be group actions.
problem Whether 2-valued dynamics can be defined by the action of a 2-valued group.
method Construction of examples of dynamics that are or are not group actions.
result Some 2-valued dynamics on complex plane cannot be defined by the action of a 2-valued group.
The paper studies dynamic star-shaped risk measures and their representation.
problem Representing dynamic star-shaped risk measures and their properties.
method Representation theorems for dynamic monetary and star-shaped risk measures.
result Dynamic star-shaped risk measures can be represented as the lower envelope of a family of dynamic convex risk measures.
Study circles to understand dynamics and rigidity in homogeneous spaces.
problem Understanding dynamics and rigidity in infinite-volume homogeneous spaces.
method Addressing four questions about circle packings.
result Highlighting the interplay between dynamics, geometry, and rigidity.
Paper connects dynamics of mechanical systems to Reeb dynamics.
problem Understanding dynamics in mechanical systems with Poisson structures.
method Using Jacobi bundle metrics and linear Poisson structures.
result Extends classical results on Reeb dynamics to mechanical systems.
Two heuristics solve dynamic multiple travelling salesmen problems.
problem Dynamic routing with unknown customers.
method Balanced dynamic closest vehicle heuristic and balanced dynamic assignment vehicle heuristic.
result Continuous approximation models for strategic dynamic routing.
In this paper we present a theoretical framework for studying coherent acceptability indices in a dynamic setup. We study dynamic coherent acceptability indices and dynamic coherent risk measures, and we establish a duality between them. We derive a representation theorem for dynamic coherent risk measures in terms of …
DOODL learns shared spectral dynamics across related dynamical systems.
problem Learning independent dynamical operators for each system limits discovery of shared structure.
method DOODL learns a dictionary of characteristic spectral dynamics on a manifold of related systems.
result DOODL achieves errors one to two orders of magnitude lower than independent operator estimation methods.
We propose a new class of mappings, called Dynamic Limit Growth Indices, that are designed to measure the long-run performance of a financial portfolio in discrete time setup. We study various important properties for this new class of measures, and in particular, we provide necessary and sufficient condition for a Dyn…
SPICE estimates sparse linear dynamic networks without hyperparameters.
problem Estimating topology and dynamics of sparse linear dynamic networks.
method SPICE (Sparse Iterative Covariance Estimation) method in an iterative framework.
result Directly reveals the underlying topology of the network.
In this paper we present a theoretical framework for determining dynamic ask and bid prices of derivatives using the theory of dynamic coherent acceptability indices in discrete time. We prove a version of the First Fundamental Theorem of Asset Pricing using the dynamic coherent risk measures. We introduce the dynamic …
Dynamical-VAE learns causal dynamics from POMDPs using future information.
problem Learning accurate state representations from partial observations in POMDPs.
method Dynamical Variational Auto-Encoder (DVAE) with hindsight framework.
result DVAE uncovers causal graph more effectively than history-based methods.
Most real world phenomena such as sunlight distribution under a forest canopy, minerals concentration, stock valuation, exhibit nonstationary dynamics i.e. phenomenon variation changes depending on the locality. Nonstationary dynamics pose both theoretical and practical challenges to statistical machine learning algori…