arXiv research

Estimates mode from partial feedback, improving AI learning pipelines.

RLHF fails when humans only partially observe, leading to inflated or overjustified feedback.

We propose a generalization of the best arm identification problem in stochastic multi-armed bandits (MAB) to the setting where every pull of an arm is associated with delayed feedback. The delay in feedback increases the effective sample complexity of standard algorithms, but can be offset if we have access to partial…

Develops a new model for RLHF accounting for partially observed states and intermediate feedback.

New protocol for online learning with partial feedback, extending classical methods.

Paper studies CLO with partial feedback, improving decision-making in uncertain contexts.

New algorithm minimizes expert selection regret in partial bandit feedback.

Paper characterizes minimax regret rates for online ranking with top-k feedback.

We use online convex optimization (OCO) for setpoint tracking with uncertain, flexible loads. We consider full feedback from the loads, bandit feedback, and two intermediate types of feedback: partial bandit where a subset of the loads are individually observed and the rest are observed in aggregate, and Bernoulli feed…

We investigate contextual online learning with nonparametric (Lipschitz) comparison classes under different assumptions on losses and feedback information. For full information feedback and Lipschitz losses, we design the first explicit algorithm achieving the minimax regret rate (up to log factors). In a partial feedb…

The paper tackles combinatorial pure exploration with various feedback structures and proposes efficient algorithms.

SA-PEF improves federated learning efficiency by correcting gradient mismatches.

We study the adversarial multi-armed bandit problem where partial observations are available and where, in addition to the loss incurred for each action, a \emph{switching cost} is incurred for shifting to a new action. All previously known results incur a factor proportional to the independence number of the feedback …

A control system $\dot{q} = f(q,u)$ is said to be trivializable if there exists local coordinates in which the system is feedback equivalent to a control system of the form $\dot{q} = f(u)$. In this paper we characterize trivializable control systems and control systems for which, up to a feedback transformation, $f$ a…

This paper analyzes error feedback in compressed federated learning for non-convex optimization problems.

We consider the problem of online multiclass classification with partial feedback, where an algorithm predicts a class for a new instance in each round and only receives its correctness. Although several methods have been developed for this problem, recent challenging real-world applications require further performance…

New algorithms for efficient learning with partial information, reducing regret.

A new RL method improves revenue management with delayed feedback.

We present an approach to interactive-predictive neural machine translation that attempts to reduce human effort from three directions: Firstly, instead of requiring humans to select, correct, or delete segments, we employ the idea of learning from human reinforcements in form of judgments on the quality of partial tra…

New algorithm for online learning with noisy side observations.

New Thompson sampling algorithm for stochastic partial monitoring achieves logarithmic regret.

New framework learns from partial feedback in multi-label tasks.

Study cost-effective fairness audits with partial feedback, improving over random exploration.

We study an online decision making problem where on each round a learner chooses a list of items based on some side information, receives a scalar feedback value for each individual item, and a reward that is linearly related to this feedback. These problems, known as contextual semibandits, arise in crowdsourcing, rec…

Learning to rank is an important problem in machine learning and recommender systems. In a recommender system, a user is typically recommended a list of items. Since the user is unlikely to examine the entire recommended list, partial feedback arises naturally. At the same time, diverse recommendations are important be…

Proposes neural delay differential equations for stable system identification with partially observed states.

Reduces user feedback needed for accurate recommender systems.

Partial-monitoring games constitute a mathematical framework for sequential decision making problems with imperfect feedback: The learner repeatedly chooses an action, opponent responds with an outcome, and then the learner suffers a loss and receives a feedback signal, both of which are fixed functions of the action a…

Unified framework for expert selection with bandit and lower-bound feedback.

Stochastic structured prediction under bandit feedback follows a learning protocol where on each of a sequence of iterations, the learner receives an input, predicts an output structure, and receives partial feedback in form of a task loss evaluation of the predicted structure. We present applications of this learning …

Increased public interest in healthy lifestyles has motivated the study of algorithms that encourage people to follow a healthy diet. Applying collaborative filtering to build recommendation systems in domains where only implicit feedback is available is also a rapidly growing research area. In this report we combine t…

We study online learning when partial feedback information is provided following every action of the learning process, and the learner incurs switching costs for changing his actions. In this setting, the feedback information system can be represented by a graph, and previous works studied the expected regret of the le…

Bandit structured prediction describes a stochastic optimization framework where learning is performed from partial feedback. This feedback is received in the form of a task loss evaluation to a predicted output structure, without having access to gold standard structures. We advance this framework by lifting linear ba…

New algorithm for multi-armed bandits with delayed, partially observed rewards.

Algorithm learns to bid optimally in repeated first-price auctions with censored feedback.

Framework identifies population quantities from MNAR feedback using weak shadow variables from pretrained models.

Partial monitoring is a general model for sequential learning with limited feedback formalized as a game between two players. In this game, the learner chooses an action and at the same time the opponent chooses an outcome, then the learner suffers a loss and receives a feedback signal. The goal of the learner is to mi…

Adaptive MAB algorithms handle composite, anonymous feedback without reward interval knowledge.

IDS algorithm optimizes sequential decisions in various monitoring settings.

Enhanced feedback model improves sample-efficiency in POMDPs.

FedSGM tackles constrained federated learning with unified framework.

Conventional collaborative filtering techniques treat a top-n recommendations problem as a task of generating a list of the most relevant items. This formulation, however, disregards an opposite - avoiding recommendations with completely irrelevant items. Due to that bias, standard algorithms, as well as commonly used …

Algorithm learns NE in imperfect information games with imperfect feedback.

A new method steers Gaussian distributions with minimal effort.

We study an online classification problem with partial feedback in which individuals arrive one at a time from a fixed but unknown distribution, and must be classified as positive or negative. Our algorithm only observes the true label of an individual if they are given a positive classification. This setting captures …

Paper tackles robustness in reward learning with partial identifiability.

New method targets vaccines for new variants using Thompson sampling.

Paper tackles dueling bandits with delayed feedback, revealing preference bias.