New algorithms for efficient learning with long-term rewards in contextual bandits.
problem Efficient learning with long-term rewards in contextual bandits.
method Proposes new algorithms leveraging sparsity to discover dependence patterns and arm parameters.
result Regret upper bounds for data-poor and data-rich regimes, showing improved sample complexity.
New algorithms ensure fair selection in combinatorial semi-bandit with unrestricted delays.
problem Fair selection in stochastic combinatorial semi-bandit with delayed feedback.
method Introduced merit-based fairness constraints and new bandit algorithms for reward and fairness.
result Achieved sublinear expected reward and fairness regrets with dependence on delay distribution quantiles.
New models for bandit problems with fidelity rewards are introduced and analyzed.
problem Fidelity rewards in bandit problems to incentivize loyalty.
method Two models (loyalty-points and subscription) for fidelity rewards; stochastic and adversarial settings considered.
result Sublinear regret bounds for some models, worst case lower bounds for others.
We consider reinforcement learning in input-driven environments, where an exogenous, stochastic input process affects the dynamics of the system. Input processes arise in many applications, including queuing systems, robotics control with disturbances, and object tracking. Since the state dynamics and rewards depend on…
SS-SARSA tackles recovering bandits by treating rounds as states.
problem Real-world rewards vary with the elapsed rounds since last pull.
method Proposes SS-SARSA, a new RL algorithm treating rounds as states.
result Achieves efficient learning with asymptotic convergence to optimal policy.
Off-policy evaluation for MNAR rewards in MDPs
problem Off-policy evaluation in MDPs with MNAR rewards
method Formalizing a reward-dependent propensity model and using future states as shadow variables
result Proposed an Fitted-Q-Evaluation-style estimator that propagates recovered rewards while allowing target policies to depend on past missingness indicators
New algorithms improve efficiency in learning from personalized rewards.
problem Learning from personalized rewards in recommendation systems.
method Developed provably efficient algorithms with sublinear regret for context-dependent feedback.
result Introduced a Lipschitz reward estimator that improves generalization performance.
New algorithm tackles nonstationary linear bandits with latent dynamics.
problem Nonstationary bandit problem with latent states and unknown dynamics.
method Explore-then-commit algorithm with exploration and commitment phases.
result Achieves ildeO(T2/3) regret. The paper tackles fair sequential decision making with biased linear bandit feedback.
problem Fair sequential decision making with biased linear bandit feedback.
method Phased elimination algorithm to correct unfair evaluations, establishing upper bounds on regret.
result The worst-case regret is smaller than O(κ∗1/3log(T)1/3T2/3). New findings on universal learning in contextual bandits with adversarial rewards.
problem Learning in contextual bandits with time-varying, adversarial rewards.
method Characterization of learnable processes and necessary/sufficient conditions for universal learning.
result Optimistic universal learning for contextual bandits with adversarial rewards is impossible in general.
New algorithm achieves consistent learning from context in bandit problems.
problem Learning from context in bandit problems with non-i.i.d. contexts.
method Optimistically universal learning rule balancing generalization and personalization.
result Achieves universal consistency for large classes of non-i.i.d. contexts.
New algorithm reduces regret for single-index bandits to nearly optimal.
problem Optimizing rewards from unknown projections of high-dimensional contexts.
method Two-phase algorithm: estimate projection direction, reduce to 1D bandit, use UCB.
result Achieved optimal regret of ildeO(T2/3). Improved regret for single-index bandits with optimal algorithm.
problem Optimizing rewards from unknown one-dimensional projections of high-dimensional contexts.
method Two-phase algorithm: first estimate projection direction, then discretize and use UCB.
result Achieved optimal regret of ildeO(T2/3). A new imitation learning method uses random search for simple policies, outperforming complex models.
problem Complexity and reward dependency issues in imitation learning.
method Derivative-free optimization with simple linear policies and random search.
result The proposed method achieves competitive performance on MuJoCo locomotion tasks without a direct reward signal.
Many sequential decision-making tasks require choosing at each decision step the right action out of the vast set of possibilities by extracting actionable intelligence from high-dimensional data streams. Most of the times, the high-dimensionality of actions and data makes learning of the optimal actions by traditional…