A new algorithm balances exploration and exploitation in online decision-making.
problem Balancing exploration and exploitation in online decision-making.
method Proposed C 4 C^4 C 4 -UCB algorithm incorporating conservative mechanism. result Proved n-step upper regret bound for two situations.
We revisit the proof by Qin et al. (2014) of bounded regret of the C 2 ^2 2 UCB contextual combinatorial bandit. We demonstrate an error in the proof of volumetric expansion of the moment matrix, used in upper bounding a function of context vector norms. We prove a relaxed inequality that yields the originally-stated regre…
New algorithms for neural bandits learn from context and arm features.
problem Learning from context and arm features in combinatorial bandit problems.
method Proposed algorithms CN-UCB and CN-TS using deep neural networks.
result Achieved regret guarantees for the first time in combinatorial neural bandits.
New algorithm for contextual combinatorial bandits with probabilistic arm triggering.
problem Optimizing decisions in dynamic environments with probabilistic arm availability.
method C^2-UCB-T and VAC^2-UCB algorithms with TPM and VM conditions.
result Achieved improved regret bounds for contextual combinatorial bandits.
Improved sample complexity for contextual combinatorial semi-bandits with sparse rewards.
problem Optimizing decisions in contexts with many possible actions and sparse rewards.
method Developed an algorithm for ( ε , δ ) (ε,δ) ( ε , δ ) -PAC variant of contextual combinatorial semi-bandits with improved sample complexity. result Achieved an ε ε ε -optimal policy with a sample complexity of i l d e O ( ( p o l y ( K / m ) + s m / ε 2 ) log ( ∣ Π ∣ / δ ) ) ilde{O}((poly(K/m)+sm/ε^2) \log(|Π|/δ)) i l d e O (( p o l y ( K / m ) + s m / ε 2 ) log ( ∣Π∣/ δ )) . New algorithm reduces regret in both adversarial and stochastic contexts.
problem Contextual combinatorial semi-bandits with adversarial and corrupted stochastic regimes.
method Follow-the-Regularized-Leader (FTRL) framework with Shannon entropy regularizer, accelerated by Karush-Kuhn-Tucker conditions.
result Achieves O ~ ( T ) \widetilde{\mathcal{O}}(\sqrt{T}) O ( T ) regret in adversarial and O ~ ( ln T ) \widetilde{\mathcal{O}}(\ln T) O ( ln T ) regret in corrupted stochastic regimes. Improved regret bounds for contextual combinatorial semi-bandits with linear payoffs.
problem Maximizing rewards in decision-making problems with feature vectors and constraints.
method Proposed C^2UCB algorithm and modified reward estimates for general constraints.
result Optimal regret bounds of C^2UCB algorithm and modified algorithm for various constraints.
New algorithm for fair ranking in contextual bandits with concave rewards.
problem Fair ranking in recommendation systems.
method Geometric interpretation of CBCR as optimization, Frank-Wolfe analyses.
result First algorithm with provably vanishing regret for CBCR.
A new method for decision-focused learning reduces computational cost.
problem Efficiently solving combinatorial problems with uncertain parameters.
method Reframed as cost-sensitive multi-output regression, with novel loss components.
result Comparable downstream task quality with reduced computational cost.
New framework uses tree ensembles for contextual bandits.
problem Optimizing decisions in dynamic environments with contextual information.
method Adapts Upper Confidence Bound and Thompson Sampling to tree ensemble methods.
result Tree ensemble methods outperform traditional methods in regret minimization and runtime.
Algorithm optimizes bandit decisions with changing action sets using Gaussian processes.
problem Optimizing decisions in a bandit problem with time-varying action sets.
method Proposes an algorithm called O'CLOK-UCB using Gaussian processes to handle changing action sets and contexts.
result Achieves regret bound of i l d e O ( λ ∗ ( K ) K T γ K T ( ∪ t ≤ T X t ) ) ilde{O}(\sqrt{λ^*(K)KTγ_{KT}(\cup_{t\leq T}\mathcal{X}_t)} ) i l d e O ( λ ∗ ( K ) K T γ K T ( ∪ t ≤ T X t ) ) with high probability. A new online learning problem, CAB, tackles matching platforms to maximize user satisfaction.
problem Maximizing matches in a matching platform can lead to dissatisfaction and churn.
method Developed CAB, an online learning problem that maximizes arm satisfaction, and analyzed algorithms like UCB and Thompson sampling.
result CAB-UCB achieves higher cumulative satisfaction than baselines in experiments.
The problem of multi-armed bandits (MAB) asks to make sequential decisions while balancing between exploitation and exploration, and have been successfully applied to a wide range of practical scenarios. Various algorithms have been designed to achieve a high reward in a long term. However, its short-term performance m…
Online learning with one-sided feedback aims to maximize accuracy while ensuring fairness.
problem Maximizing accuracy in online learning with limited feedback and ensuring fairness.
method Extending the framework of Bechavod et al. (2020) to incorporate dynamic panels of auditors, reducing the problem to a contextual combinatorial semi-bandit, and leveraging Exp2 and Context-Semi-Bandit-FTPL algorithms.
result Multi-criteria no regret guarantees for accuracy and fairness are provided.
New algorithm eliminates arms to minimize regret in complex bandit problems.
problem Minimizing regret in combinatorial bandit problems with explicit exploration.
method Introduces a novel arm elimination scheme that partitions arms into three categories and incorporates explicit exploration.
result Achieves near-optimal regret in combinatorial multi-armed and linear contextual bandit problems.
New method for evaluating and learning in complex decision-making scenarios.
problem Evaluating and learning from policies in contextual combinatorial bandits with high bias and variance.
method Factored action space decomposition and importance sampling-based estimator (OPCB).
result OPCB achieves superior performance in OPE and OPL compared to conventional methods.
A new algorithm balances global reward and group constraints in federated multi-armed bandits.
problem Maximizing global reward while protecting client privacy in federated learning.
method Combinatorial contextual bandit with group constraints, using a two-output Gaussian process.
result TCGP-UCB incurs low regret, balancing super arm reward and group reward constraints.
We propose a new framework for designing estimators for off-policy evaluation in contextual bandits. Our approach is based on the asymptotically optimal doubly robust estimator, but we shrink the importance weights to minimize a bound on the mean squared error, which results in a better bias-variance tradeoff in finite…
New model for personalized online advertising with multi-user interaction.
problem Realistic online advertising scenarios with multiple users interacting simultaneously.
method Introduces Multi-User Contextual Cascading Bandit (MCCB) model and proposes UCBBP and AUCBBP algorithms.
result Proves UCBBP and AUCBBP achieve optimal regret bounds for multi-user context.
DMNL bandits optimize assortment choices balancing relevance and diversity.
problem Balancing relevance-driven choice with within-assortment diversity.
method Augments MNL choice probabilities with a submodular diversity function, proposing a white-box UCB-based algorithm.
result Achieves at least a ( 1 − 1 e + 1 ) (1-\frac{1}{e+1}) ( 1 − e + 1 1 ) -approximate regret bound of $ ilde{O}\left(d \sqrt{T/K}
ight)$ . Paper tackles combinatorial reinforcement learning with preference feedback.
problem Modeling long-term user engagement in scenarios like recommender systems and online advertising.
method Assumes a contextual MNL preference model with linear mean utilities and approximates item values. Proposes MNL-VQL algorithm.
result Achieves nearly minimax-optimal regret for linear MDPs with preference feedback.
New algorithms reduce sample complexity for multiclass contextual bandits.
problem Designing efficient algorithms for multiclass contextual bandits with sparse rewards.
method Two complementary approaches: decision-estimation coefficient analysis and low-variance exploration.
result Achieved optimal sample complexity bounds for multiclass contextual bandits.
New insights into multi-armed bandits with budget constraints.
problem Multi-armed bandits with supply/budget constraints.
method Characterization of logarithmic regret rates, simple regret, and reduction to other bandit problems.
result Full characterization of logarithmic, instance-dependent regret rates for BwK.
Master-slave architecture tackles combinatorial multi-armed bandits with diversity constraints.
problem Solving top- K K K combinatorial multi-armed bandits with non-linear feedback and diversity constraints. method Master-slave architecture with six slave models, teacher learning, and policy co-training.
result Significantly outperforms existing algorithms in synthetic and real datasets.
A novel bandit problem with context-dependent rewards and blocking.
problem Contextual bandit problem with blocking.
method Online bipartite matching, UCB algorithm, delayed exploitation, opportunistic subsampling.
result Guaranteed O ( log T ) \mathcal{O}(\log T) O ( log T ) -regret in bandit setting. Authors use exotic 4-manifolds to model quantum computing and measurements.
problem Modeling quantum computing and measurements using exotic space-time structures.
method Choose the fundamental group of an exotic 4-manifold R 4 R^4 R 4 to encode quantum states and measurements. result Exotic R 4 R^4 R 4 models encode a variety of standard contextual geometries in quantum measurements. Contextualized ML learns context-dependent effects using deep learning.
problem Learning heterogeneous and context-dependent effects in data.
method Applying deep learning to the meta-relationship between contextual information and context-specific parametric models.
result Unified framework for cluster analysis and cohort modeling.
IDS improves reinforcement learning with contextual information.
problem Optimizing IDS for contextual reinforcement learning.
method Investigated contextual bandit problems and proposed a computationally-efficient IDS.
result Contextual IDS outperforms conditional IDS by considering future contexts.
IDS optimizes regret in stochastic partial monitoring with linear rewards.
problem Optimizing decision-making in uncertain environments with linear rewards.
method Information Directed Sampling (IDS) for stochastic partial monitoring.
result Achieves optimal regret rates in all observable game regimes.
SADCBO optimizes contextual variables by balancing relevance and cost.
problem Optimizing contextual variables with varying costs and unknown relevance.
method Adaptive selection of relevant contextual variables using sensitivity analysis and early stopping.
result Consistent improvement in optimization across various examples.
Interactive recommender systems that enable the interactions between users and the recommender system have attracted increasing research attentions. Previous methods mainly focus on optimizing recommendation accuracy. However, they usually ignore the diversity of the recommendation results, thus usually results in unsa…
The bundle approach and n-contextuality reveal quantum model contextuality.
problem Understanding contextuality in quantum models using topology.
method Using the bundle approach, we describe contextuality as the non-existence of global sections in the measure bundle. We introduce n-contextuality to explore model dependence on scenario topology.
result Quantum theory and GHZ models exhibit all levels of n-contextuality, showing contextuality is related to holonomy group non-triviality.
Proposes a neural network for contextual regression.
problem Improving model efficiency and interpretability in regression with contextual features.
method Simple contextual neural network (SCtxtNN) that separates context identification from context-specific regression.
result SCtxtNN achieves lower excess mean squared error and more stable performance than feed-forward neural networks.
Paper tackles domain adaptation for contextual bandits with sub-linear regret.
problem Adapting contextual bandit algorithms across domains with distribution shift.
method Learn a bandit model for the target domain using feedback from the source domain.
result Sub-linear regret bound maintained across domains.
Transformers model contextual relations using probabilistic measures, revealing their expressive power.
problem Lack of clear understanding of Transformer's ability to model contextual relations.
method Introduced a measure-theoretic framework connecting softmax attention and entropy-regularized optimal transport.
result Transformer architectures can approximate arbitrary contextual relations, and the choice of normalization affects how these relations are represented.
Paper solves stochastic contextual linear bandits using linear bandit algorithms.
problem Stochastic contextual linear bandits with unknown context distribution.
method Establishes a reduction framework to convert to linear bandit problems.
result Achieves nearly optimal regret bound of O ( d T log T ) O(d\sqrt{T\log T}) O ( d T log T ) . The study explores whether model selection guarantees apply to contextual bandits.
problem Applying model selection guarantees to contextual bandits.
method Investigates whether similar guarantees for model selection in statistical learning can be extended to contextual bandit learning.
result Initial findings suggest that model selection guarantees may not directly apply to contextual bandits.
Faster algorithm reduces contextual bandit regret with fewer offline regression calls.
problem Optimizing reward in contextual bandits with unknown functions.
method Designing a simple algorithm with O ( log T ) {O}(\log T) O ( log T ) offline regression calls. result Achieves statistically optimal regret with minimal offline calls.
Efficiently handles contextual bandits with diffusion models.
problem Challenges in online decision-making with contextual bandits.
method Leverage pre-trained diffusion models as priors to capture action dependencies.
result Developed an algorithm for efficient posterior approximation.
This study analyzes how RNNs process context in sentiment analysis.
problem Understanding how recurrent neural networks process context in sentiment analysis.
method Developed methods to reverse engineer RNNs, identifying contextual effects and quantifying their strength and timescale.
result Identified inputs that induce contextual effects and quantified their properties.
Thompson Sampling bounds for contextual bandits with sub-Gaussian rewards.
problem Improving the performance of Thompson Sampling in contextual bandits with sub-Gaussian rewards.
method Proved comprehensive bounds on Thompson Sampling expected cumulative regret based on mutual information and lifted information ratio for sub-Gaussian rewards.
result Explicit regret bounds for various contextual bandit scenarios.
CADR estimator improves inference for contextual bandit data.
problem Valid inference on contextual bandit data.
method CADR estimator for policy value, addressing adaptive data collection challenges.
result CADR provides correct coverage of confidence intervals.
New algorithm learns optimal exploration parameters for contextual bandits.
problem Learning optimal exploration in contextual bandits.
method Proposes two algorithms that learn optimal exploration parameters online based on context and reward.
result Demonstrates improved performance in learning optimal exploration compared to traditional methods.
The paper tackles learning varying DAG structures based on contextual features.
problem Learning a single DAG for the entire population from observational data.
method A neural network that maps contextual features to a weighted adjacency matrix of a DAG, with a projection layer to ensure acyclicity.
result The new approach can recover context-specific DAGs where existing methods fail.
A method for unsupervised contextual anomaly detection is proposed using a cross-linked pair of Variational Auto-Encoders for assigning a normality score to an observation. The method enables a distinct separation of contextual from behavioral attributes and is robust to the presence of anomalous or novel contextual at…
A framework for auto-tuning hyper-parameters in contextual bandit algorithms.
problem Auto-tuning hyper-parameters in real-time for contextual bandit algorithms.
method Proposes a Syndicated Bandits framework to learn multiple hyper-parameters dynamically.
result Achieves optimal regret bounds under certain scenarios and handles multiple contextual bandit algorithms.
New algorithm improves learning efficiency in multi-task contextual bandits.
problem Improving learning efficiency in multi-task contextual bandits.
method Alternating projected gradient descent (GD) and minimization estimator for low-rank feature matrix recovery.
result Proved regret bound for multi-task learning algorithm.
We consider the stochastic linear (multi-armed) contextual bandit problem with the possibility of hidden simple multi-armed bandit structure in which the rewards are independent of the contextual information. Algorithms that are designed solely for one of the regimes are known to be sub-optimal for the alternate regime…