New algorithms reduce slate bandit regret for large slates, outperforming existing methods.
problem Non-separable reward functions in slate bandits with many slates.
method Design of algorithms with sub-linear regret.
result Sub-linear regret with respect to the time horizon for large number of slates.
A new estimator reduces variance in slate bandit OPE.
problem Large action spaces in slate bandits cause high variance in OPE.
method Develops Latent IPS (LIPS) to optimize slate abstractions for low variance and bias.
result LIPS substantially outperforms existing estimators in scenarios with non-linear rewards and large slate spaces.
Efficient algorithms for contextual slate bandits with limited adaptivity.
problem Contextual slate bandit problem with limited adaptivity.
method Proposed B-SlateGLinCB and RS-SlateGLinCB algorithms for batched and rarely-switching settings.
result Achieved regret bounds of O(Nd^(3/2)√T) and O(Nd√T) under diversity assumption.
New estimator reduces risk in slate bandits by leveraging Bayes risk criterion.
problem Evaluating slate policies using logged data when policies factorize over slots.
method Developed a new estimator using a control variate approach, showing risk improvement over existing methods.
result The new estimator has lower risk than the pseudoinverse estimator in slate bandit problems.
A new recommender system uses slates and Thompson Sampling to improve diversity and click rates.
problem Recommending relevant content to users in a dynamic environment.
method Variational Bayesian Recurrent Neural Net, Thompson Sampling, hierarchical prior, in-slate Thompson Sampling.
result Improved click rates and diversity in recommended slates.
Generative model learns to create coherent slates from prompts.
problem Generating coherent slates from natural language prompts.
method DMSG, a generative framework based on diffusion models.
result DMSG outperforms baselines in relevance and diversity.
Scalable model for slate recommendation learns reward probabilities.
problem Scalable personalized slate recommendation in large action spaces.
method Probabilistic Rank and Reward (PRR) model combining reward, interaction, and rank.
result PRR outperforms existing methods and is scalable to large action spaces.
A deep reinforcement learning approach for slate re-ranking in e-commerce.
problem Improving user satisfaction in e-commerce by optimizing the ranking of items in a slate.
method Generator and Critic approach, using reinforcement learning and a Full Slate Critic model.
result The Generator and Critic approach significantly outperforms existing methods in slate evaluation and efficiency.
A new dataset tracks user interactions and click responses in online marketplaces.
problem Lack of exposure data in recommender systems datasets.
method Proposes a novel dataset including slates and click responses, allowing more accurate likelihood models.
result Models using exposure data show more natural likelihood, reducing bias towards previously exposed items.
This paper studies the evaluation of policies that recommend an ordered set of items (e.g., a ranking) based on some context---a common scenario in web search, ads, and recommendation. We build on techniques from combinatorial bandits to introduce a new practical estimator that uses logged data to estimate a policy's p…
The conventional solution to the recommendation problem greedily ranks individual document candidates by prediction scores. However, this method fails to optimize the slate as a whole, and hence, often struggles to capture biases caused by the page layout and document interdepedencies. The slate recommendation problem …
We learn hierarchical slate representations for collaborative filtering.
problem Building models for recommendation systems with hierarchical slates.
method Learning low-dimensional embeddings of hierarchical slates using recursive composition rules.
result Improved recommendation system performance on a real-world dataset.
Most practical recommender systems focus on estimating immediate user engagement without considering the long-term effects of recommendations on user behavior. Reinforcement learning (RL) methods offer the potential to optimize recommendations for long-term user engagement. However, since users are often presented with…
New algorithms learn MNL weights efficiently for any slate size.
problem Efficiently learn weights for MNL models given query access.
method Two algorithms: adaptive and non-adaptive, with specific query complexities.
result Optimal query complexities for both adaptive and non-adaptive cases.
This paper optimizes slate decision systems for large action spaces.
problem Optimizing large-scale decision systems with arbitrary reward functions.
method A policy optimization framework with a novel relaxation of decision functions.
result Demonstrates the effectiveness of the proposed method on large action spaces.
Ranking is a central task in machine learning and information retrieval. In this task, it is especially important to present the user with a slate of items that is appealing as a whole. This in turn requires taking into account interactions between items, since intuitively, placing an item on the slate affects the deci…
New method for evaluating sequential recommendations with lower variance.
problem Evaluating good sequences of music, video, news, and e-commerce recommendations.
method Proposes a new counterfactual estimator for sequential reward interactions with lower variance and asymptotic unbiasedness.
result Our method outperforms existing methods in bias and data efficiency for sequential track recommendations.
We present a deep neural-network model for lifelong learning inspired by several forms of neuroplasticity. The neural network develops continuously in response to signals from the environment. In the beginning, the network is a blank slate with no nodes at all. It develops according to four rules: (i) expansion, which …
We summarize the potential impact that the European Union's new General Data Protection Regulation will have on the routine use of machine learning algorithms. Slated to take effect as law across the EU in 2018, it will restrict automated individual decision-making (that is, algorithms that make decisions based on user…
Researchers show mixtures of ranking models are generally identifiable.
problem Understanding when and how parameters of mixtures of ranking models can be uniquely determined.
method Algebraic geometry framework applied to verify the number of solutions in polynomial systems.
result Popular mixtures of ranking models with two components are generically identifiable.
New algorithms tackle RKHS bandits with reduced complexity and improved performance.
problem Adversarial and stochastic RKHS bandit problems with high computational complexity.
method Combining approximation theory with misspecified linear bandit methods.
result First general algorithm for adversarial RKHS bandit problem.
Unified formulation bridges adversarial and nonstationary bandits.
problem Handling time-varying reward distributions in multi-armed bandit problems.
method Unified oracle that switches between adversarial and nonstationary bandit oracles based on window size.
result Optimal regret achieved with matching lower bound.
Paper tackles LDP bandits learning with improved results and sub-linear regret.
problem Contextual bandits learning with LDP privacy constraints.
method Simple black-box reduction frameworks for context-free bandits, extended to GLB.
result First result for BCO with multi-point feedback under LDP, sub-linear regret for GLB.
Paper solves stochastic contextual linear bandits using linear bandit algorithms.
problem Stochastic contextual linear bandits with unknown context distribution.
method Establishes a reduction framework to convert to linear bandit problems.
result Achieves nearly optimal regret bound of O ( d T log T ) O(d\sqrt{T\log T}) O ( d T log T ) . Graph-Triggered Bandits unify rested and restless bandits with graph-defined arm interactions.
problem Modeling sequential decision-making problems with evolving arm rewards.
method Graph-Triggered Bandits (GTBs) framework that generalizes rested and restless bandits using a graph.
result Rested and restless bandits are special cases of GTBs for suitable graphs.
New definition resolves ambiguity in non-stationary bandit classification.
problem Ambiguity in classifying non-stationary bandits using existing definitions.
method Introducing a formal definition that resolves ambiguity and provides a unified approach.
result Unified approach applicable to both Bayesian and frequentist formulations, resolves classification issues.
Unified approach for non-stationary and clustered bandits.
problem Solving non-stationary and clustered bandits with overlapping solutions.
method Test of homogeneity for seamless integration of non-stationary and clustered bandits.
result Unified solution framework for change detection and cluster identification.
A framework for auto-tuning hyper-parameters in contextual bandit algorithms.
problem Auto-tuning hyper-parameters in real-time for contextual bandit algorithms.
method Proposes a Syndicated Bandits framework to learn multiple hyper-parameters dynamically.
result Achieves optimal regret bounds under certain scenarios and handles multiple contextual bandit algorithms.
Stochastic multi-armed bandits form a class of online learning problems that have important applications in online recommendation systems, adaptive medical treatment, and many others. Even though potential attacks against these learning algorithms may hijack their behavior, causing catastrophic loss in real-world appli…
New algorithm learns optimal exploration parameters for contextual bandits.
problem Learning optimal exploration in contextual bandits.
method Proposes two algorithms that learn optimal exploration parameters online based on context and reward.
result Demonstrates improved performance in learning optimal exploration compared to traditional methods.
Investigates sequential problems on graph structures and large action spaces.
problem Sequential decision-making on graph structures and large action spaces.
method Spectral bandits, side observations, influence maximization, kernel bandits, polymatroid bandits, function optimization, infinitely many-arms bandits.
result Contributions to graph and structured bandits.
Study on indexability of restless multi-armed bandits and rollout policy performance.
problem Maximizing discounted rewards in finite state restless multi-armed bandit problems.
method Decouple the problem into single-armed restless bandits, analyze using value iteration, and compare with Whittle index policy.
result Demonstrates conditions for indexability and compares performance of index policy and rollout policy.
Optimal algorithm for identifying best arm in stochastic linear bandits with fixed confidence.
problem Identifying the best arm in stochastic linear bandits with fixed confidence.
method Extending an algorithm designed for Best Arm Identification to the ε ε ε -Thresholding Bandit Problem (TBP). result Asymptotically optimal algorithm for TBP.
A meta-UCB method combines stochastic bandit algorithms.
problem Combining multiple stochastic bandit algorithms efficiently.
method Meta-UCB procedure solving an N-armed bandit problem.
result Final regret depends only on the best base algorithm's regret.
New method for contextual bandits with corrupted context.
problem Contextual bandits with corrupted context in online settings.
method Combining contextual bandit and multi-armed bandit approaches.
result Improved learning from all iterations, including corrupted ones.
A new framework for structured bandits using influence diagrams and variational Thompson sampling.
problem Complex statistical dependencies in structured bandit problems.
method Influence diagram framework, variational Thompson sampling, tracking structured posterior distribution.
result Empirically evaluated algorithms perform as well as or better than existing baselines.
New algorithm for nonstationary multi-armed bandits with optimal performance.
problem Nonstationary multi-armed bandits with changing model parameters over time.
method Adaptive Resetting Bandit (ADR-bandit) algorithm using adaptive windowing techniques.
result ADR-bandit achieves nearly optimal performance in both abrupt and gradual changes.
First robust bandit algorithm for contextual bandits with sub-linear regret.
problem Vulnerability of linear contextual bandit algorithms to adversarial attacks.
method Proposes a robust bandit algorithm for stochastic linear contextual bandits under fully adaptive and omniscient attacks.
result Sub-linear regret under various attacks without requiring attack information.
Unified framework for high-dimensional bandit problems with low-dimensional structures.
problem Stochastic high-dimensional bandit problems with low-dimensional structures.
method Proposed a simple unified algorithm and a general analysis framework for the regret upper bound.
result Unified algorithm achieves comparable regret bounds in various high-dimensional bandit problems.
Survey and compare PAC-Bayes bounds for bandit problems.
problem Designing and evaluating bandit algorithms with strong performance guarantees.
method PAC-Bayes bounds applied to bandit problems.
result PAC-Bayes bounds useful for offline bandit algorithms, but loose for online algorithms.
New algorithms reduce regret for convex bandits with small comparator norms.
problem Optimizing in bandit convex optimization with varying comparator norms.
method Developed algorithms using techniques from full-information setting and new gradient estimators.
result Regret bounds are small when comparator norm is small.
Paper tackles domain adaptation for contextual bandits with sub-linear regret.
problem Adapting contextual bandit algorithms across domains with distribution shift.
method Learn a bandit model for the target domain using feedback from the source domain.
result Sub-linear regret bound maintained across domains.
New model tackles causal bandits with dependent variables.
problem Understanding reward-maximizing interventions in causal networks with dependent variables.
method Introduces hierarchical causal bandit model with a contextual variable capturing interactions among variables.
result Derives nearly matching regret bounds for binary context in causal bandits with dependent arms.
We consider the problem of best arm identification in a variant of multi-armed bandits called linked bandits. In a single interaction with linked bandits, multiple arms are played sequentially until one of them receives a positive reward. Since each interaction provides feedback about more than one arm, the sample comp…
New dynamic allocation methods for multi-armed bandit models.
problem Dynamic allocation problems in multi-armed bandit models.
method New types of dynamic allocation problems and proofs for Gittins index decomposition.
result New proofs for Gittins index decomposition and related results.
The paper tackles lifelong learning in multi-armed bandits, aiming to minimize average regret over multiple tasks.
problem Minimizing average regret in multi-armed bandits over multiple tasks.
method Confidence interval tuning of UCB algorithms and greedy algorithms applied to a bandit over bandit approach.
result Empirical improvement over previous work in the mortal bandit problem.
New algorithm for partially observable contexts in finance.
problem Decision making based on partially observable, correlated market information.
method EMKF-Bandit algorithm integrating system identification, filtering, and bandit algorithms.
result Sub-linear regret under conditions on filtering.
New meta-learning approach for bandit policies that achieve high average reward.
problem Designing bandit policies that balance between worst-case and Bayesian assumptions.
method Differentiable parameterized policies optimized using policy gradients.
result Proposed algorithm achieves low regret and is practical for various bandit problems.