arXiv research

Algorithm provides online learning guarantees against general comparators in full and bandit feedback.

Recommender systems recommend items more accurately by analyzing users' potential interest on different brands' items. In conjunction with users' rating similarity, the presence of users' implicit feedbacks like clicking items, viewing items specifications, watching videos etc. have been proved to be helpful for learni…

With the rise of social media like Twitter and of software distribution platforms like app stores, users got various ways to express their opinion about software products. Popular software vendors get user feedback thousandfold per day. Research has shown that such feedback contains valuable information for software de…

This thesis tackles learning reward functions from human comparative feedback.

Unified framework for analyzing online convex optimization across various settings.

We present a study on reinforcement learning (RL) from human bandit feedback for sequence-to-sequence learning, exemplified by the task of bandit neural machine translation (NMT). We investigate the reliability of human bandit feedback, and analyze the influence of reliability on the learnability of a reward estimator,…

New algorithm for recommending best arms with aggregated feedback.

Study of reinforcement learning with additional feedback observations.

A survey is performed of various Multi-Armed Bandit (MAB) strategies in order to examine their performance in circumstances exhibiting non-stationary stochastic reward functions in conjunction with delayed feedback. We run several MAB simulations to simulate an online eCommerce platform for grocery pick up, optimizing …

FP uses random projections to train networks without feedback, achieving comparable performance to backpropagation.

New algorithm reduces dynamic regret in time-varying movement costs.

CAFL breaks feedback loops in recommender systems using causal inference.

LoCo-RLHF models diverse human feedback with contextual information.

Simulation-based training (SBT) is gaining popularity as a low-cost and convenient training technique in a vast range of applications. However, for a SBT platform to be fully utilized as an effective training tool, it is essential that feedback on performance is provided automatically in real-time during training. It i…

Paper studies CLO with partial feedback, improving decision-making in uncertain contexts.

New algorithms handle online prediction with bandit and delayed feedback, improving regret bounds.

New method learns from subgroup feedback in complex systems.

We present online boosting algorithms for multiclass classification with bandit feedback, where the learner only receives feedback about the correctness of its prediction. We propose an unbiased estimate of the loss using a randomized prediction, allowing the model to update its weak learners with limited information. …

New algorithm for active learning from feedback coding.

Stochastic structured prediction under bandit feedback follows a learning protocol where on each of a sequence of iterations, the learner receives an input, predicts an output structure, and receives partial feedback in form of a task loss evaluation of the predicted structure. We present applications of this learning …

Study the tradeoffs of bandit feedback in multiclass classification.

Interactive Machine Learning is concerned with creating systems that operate in environments alongside humans to achieve a task. A typical use is to extend or amplify the capabilities of a human in cognitive or physical ways, requiring the machine to adapt to the users' intentions and preferences. Often, this takes the…

We formulate and study a novel multi-armed bandit problem called the qualitative dueling bandit (QDB) problem, where an agent observes not numeric but qualitative feedback by pulling each arm. We employ the same regret as the dueling bandit (DB) problem where the duel is carried out by comparing the qualitative feedbac…

Unified LP framework for offline reward learning from human demonstrations and feedback.

Anomaly detectors are often used to produce a ranked list of statistical anomalies, which are examined by human analysts in order to extract the actual anomalies of interest. Unfortunately, in realworld applications, this process can be exceedingly difficult for the analyst since a large fraction of high-ranking anomal…

Robot motions in the presence of humans should not only be feasible and safe, but also conform to human preferences. This, however, requires user feedback on the robot's behavior. In this work, we propose a novel approach to leverage the user's brain signals as a feedback modality in order to decode the judgment of rob…

Proposes a new theoretical framework for PbRL that requires less human feedback.

The paper proposes a new method for learning reward models from ordinal feedback, improving upon binary feedback.

Paper finds algorithms with both low regret and high exploitation.

A new one-point feedback scheme improves ZO algorithms for black-box optimization.

This paper explores a new form of the linear bandit problem in which the algorithm receives the usual stochastic rewards as well as stochastic feedback about which features are relevant to the rewards, the latter feedback being the novel aspect. The focus of this paper is the development of new theory and algorithms fo…

Study online learning with off-policy feedback in adversarial bandit problems.

Two coding schemes designed for different settings are compared.

RANDomized-exploration policy Optimization via Multiple Importance Sampling with Truncation (RANDOMIST) for PO with mediator feedback.

We consider the problem of learning to play a repeated multi-agent game with an unknown reward function. Single player online learning algorithms attain strong regret bounds when provided with full information feedback, which unfortunately is unavailable in many real-world scenarios. Bandit feedback alone, i.e., observ…

Sign-based algorithms (e.g. signSGD) have been proposed as a biased gradient compression technique to alleviate the communication bottleneck in training large neural networks across multiple workers. We show simple convex counter-examples where signSGD does not converge to the optimum. Further, even when it does conver…

This work focuses on dynamic regret of online convex optimization that compares the performance of online learning to a clairvoyant who knows the sequence of loss functions in advance and hence selects the minimizer of the loss function at each step. By assuming that the clairvoyant moves slowly (i.e., the minimizers c…

This work compares human feedback methods for reward learning in bandits.

SLHF uses sequential game theory to optimize preferences from human feedback.

New techniques improve distributed training with compressed gradients.

The paper addresses learner privacy in convex optimization with feedback.

New Feedback Transformer architecture improves model performance by exposing past representations to future.

Improved regret bounds for bandits with expert advice.

We present an approach to interactive-predictive neural machine translation that attempts to reduce human effort from three directions: Firstly, instead of requiring humans to select, correct, or delete segments, we employ the idea of learning from human reinforcements in form of judgments on the quality of partial tra…

Method tackles uncertainty in reward models for LLMs from heterogeneous human feedback.

SetRank tackles collaborative ranking from implicit feedback using setwise Bayesian approach.

Paper introduces a new framework for optimizing non-convex functions.

QS-BO optimizes functions using only rank-based feedback.