arXiv research

Study on sample complexity for pure exploration in feedback graph settings.

Study analyzes feedback complexity for sparse feature retrieval in deep networks.

Neural algorithms optimize arm selection with human preference feedback for complex reward functions.

New algorithms handle online prediction with bandit and delayed feedback, improving regret bounds.

Easy conditions found for simplifying complex systems.

A new one-point feedback scheme improves ZO algorithms for black-box optimization.

Study quantile reward identification with 1-bit feedback constraints.

Feedback improves ESN performance by 30-60% across various tasks.

The paper tackles combinatorial pure exploration with various feedback structures and proposes efficient algorithms.

Algorithm identifies Copeland winners in dueling bandits with ternary feedback.

Advocates a local feedback approach for RL in unknown systems.

Proposes a new theoretical framework for PbRL that requires less human feedback.

We propose a generalization of the best arm identification problem in stochastic multi-armed bandits (MAB) to the setting where every pull of an arm is associated with delayed feedback. The delay in feedback increases the effective sample complexity of standard algorithms, but can be offset if we have access to partial…

This work studies reinforcement learning in the Sim-to-Real setting, in which an agent is first trained on a number of simulators before being deployed in the real world, with the aim of decreasing the real-world sample complexity requirement. Using a dynamic model known as a rich observation Markov decision process (R…

New method for identifying best designs in vector optimization with uncertain feedback.

New method learns from subgroup feedback in complex systems.

We introduce the probably approximately correct (PAC) \emph{Battling-Bandit} problem with the Plackett-Luce (PL) subset choice model--an online learning framework where at each trial the learner chooses a subset of $k$ arms from a fixed set of $n$ arms, and subsequently observes a stochastic feedback indicating prefere…

Develops a new model for RLHF accounting for partially observed states and intermediate feedback.

Sharp sample complexity for multiclass PAC learning with bandit feedback.

Paper tackles binary feedbacks in contextual search learning.

The paper addresses learner privacy in convex optimization with feedback.

Modeling HFT interactions reveals market instability.

To widen their accessibility and increase their utility, intelligent agents must be able to learn complex behaviors as specified by (non-expert) human users. Moreover, they will need to learn these behaviors within a reasonable amount of time while efficiently leveraging the sparse feedback a human trainer is capable o…

Current algorithms for deep learning probably cannot run in the brain because they rely on weight transport, where forward-path neurons transmit their synaptic weights to a feedback path, in a way that is likely impossible biologically. An algorithm called feedback alignment achieves deep learning without weight transp…

Deep Reinforcement Learning has enabled the control of increasingly complex and high-dimensional problems. However, the need of vast amounts of data before reasonable performance is attained prevents its widespread application. We employ binary corrective feedback as a general and intuitive manner to incorporate human …

New robust algorithms improve learning with feature feedback.

We consider stochastic multi-armed bandit problems with complex actions over a set of basic arms, where the decision maker plays a complex action rather than a basic arm in each round. The reward of the complex action is some function of the basic arms' rewards, and the feedback observed may not necessarily be the rewa…

We present online boosting algorithms for multiclass classification with bandit feedback, where the learner only receives feedback about the correctness of its prediction. We propose an unbiased estimate of the loss using a randomized prediction, allowing the model to update its weak learners with limited information. …

Simple algorithms identify best items or full rankings from choice-based feedback.

We present online boosting algorithms for multilabel ranking with top-k feedback, where the learner only receives information about the top k items from the ranking it provides. We propose a novel surrogate loss function and unbiased estimator, allowing weak learners to update themselves with limited information. Using…

Study tackles RLHF with diverse human feedback, showing limitations and proposing a meta-learning approach.

Paper analyzes online reinforcement learning with outcome-based feedback, providing efficient algorithms and fundamental limits.

A new algorithm for identifying the best arm in linear feedback with safety constraints.

The paper proposes a new method for learning reward models from ordinal feedback, improving upon binary feedback.

Online learning with delayed feedback has received increasing attention recently due to its several applications in distributed, web-based learning problems. In this paper we provide a systematic study of the topic, and analyze the effect of delay on the regret of online learning algorithms. Somewhat surprisingly, it t…

Improved PAC learning algorithm for multiclass classification with bandit feedback.

The paper tackles efficient change point detection with limited samples.

Social media systems rely on user feedback and rating mechanisms for personalization, ranking, and content filtering. However, when users evaluate content contributed by fellow users (e.g., by liking a post or voting on a comment), these evaluations create complex social feedback effects. This paper investigates how ra…

UCFE benchmarks LLMs in financial tasks with human feedback.

Algorithm minimizes regret in predictive models influenced by their own predictions.

Improves reinforcement learning for complex tasks with sparse feedback.

This paper explores a new form of the linear bandit problem in which the algorithm receives the usual stochastic rewards as well as stochastic feedback about which features are relevant to the rewards, the latter feedback being the novel aspect. The focus of this paper is the development of new theory and algorithms fo…

Algorithm identifies Pareto set in bandits with contaminated feedback.

The paper tackles Nash-regret minimization in congestion games with bandit feedback.

Top-k Combinatorial Bandits generalize multi-armed bandits, where at each round any subset of $k$ out of $n$ arms may be chosen and the sum of the rewards is gained. We address the full-bandit feedback, in which the agent observes only the sum of rewards, in contrast to the semi-bandit feedback, in which the agent obse…

Improved error feedback method reduces communication complexity in distributed training.

This paper solves the multiple reference model problem in RLHF with exact solutions and sample complexity guarantees.

Interactive IL algorithm queries noisy expert for feedback, reducing sample size.