The paper tackles PAC ranking with subset-wise feedback, achieving optimal sample complexity.
problem Probably Approximately Correct (PAC) ranking of items with subset-wise preference feedback.
method Adaptive subset-wise preference feedback, Plackett-Luce model, pivot trick for score estimates.
result Achieves optimal sample complexity for PAC ranking with subset-wise feedback.
New algorithms minimize regret in combinatorial online learning with relative feedback.
problem Minimizing regret in online learning with subset-wise relative preference feedback.
method Instance-dependent and order-optimal regret algorithms for two settings: bounded size subsets and fixed size subsets.
result Regret bounds of O ( n m ln T ) O(\frac{n}{m} \ln T) O ( m n ln T ) and O ( n k ln T ) O(\frac{n}{k} \ln T) O ( k n ln T ) for respective settings. Optimal sample complexity for learning Plackett-Luce models.
problem PAC-learning good items from subsetwise feedback in Plackett-Luce models.
method Algorithm based on a wrapper around a PAC winner-finding algorithm, adapting to instance hardness.
result Optimal instance-dependent sample complexity for best arm identification.
New insights into cascade feedback linearization of control systems.
problem Obtaining a cascade feedback linearization for invariant control systems.
method Introducing truncated versions of operators from the calculus of variations to prove new theorems.
result Established new geometry and foundational theorems for future work.
Paper proposes a combined model for better recommendation by integrating explicit and implicit feedbacks.
problem Improve recommendation accuracy by considering both explicit and implicit feedbacks.
method Developed three models (RHC-PMF, RV-PMF, RHCV-PMF) that incorporate users' explicit and implicit feedbacks for better rating prediction.
result RHCV-PMF model outperforms other models in cold start scenarios for both users and items.
New method learns from either positive or negative feedback alone.
problem Limited applicability of existing preference optimization methods in scenarios with only unpaired feedback.
method Decouples learning from positive and negative feedback, using expectation-maximization (EM) to optimize probability of positive outcomes and explicitly incorporate negative examples.
result Stable learning from negative feedback alone demonstrated.
User preferences for items can be inferred from either explicit feedback, such as item ratings, or implicit feedback, such as rental histories. Research in collaborative filtering has concentrated on explicit feedback, resulting in the development of accurate and scalable models. However, since explicit feedback is oft…
This paper improves image retrieval accuracy through novel relevance feedback methods.
problem Improving image retrieval accuracy in Content-Based Image Retrieval (CBIR).
method Novel addition to feature re-weighting and classification techniques, focusing on 0-th iteration improvement.
result Significantly improved retrieval accuracy from relevance feedback.
Study evaluates new models using human feedback from another model.
problem Evaluate a new model using human feedback collected for another model.
method Formalize problem, propose model-based and model-free estimators, analyze unbiasedness, and empirically evaluate.
result Proposed estimators can predict absolute values, rank, and optimize evaluated policies.
Develops a new model for RLHF accounting for partially observed states and intermediate feedback.
problem Lack of models for partially observed states and intermediate feedback in RLHF.
method PORRL model with cardinal and dueling feedback methods.
result Demonstrates improved learning and alignment with new model-based and model-free methods.
Classifier learns to ignore unreliable feedback from end users.
problem Improving classifier performance by filtering unreliable feedback.
method Modeling end users as autonomous agents, periodically retraining classifier with filtered feedback.
result Classifier can identify and filter out unreliable feedback, improving performance.
Paper characterizes minimax regret rates for online ranking with top-k feedback.
problem Analyzing online ranking with partial feedback.
method Developed techniques from partial monitoring to characterize minimax regret rates.
result Full characterization of minimax regret rates for Precision@n.
Study on sample complexity for pure exploration in feedback graph settings.
problem Sample complexity of pure exploration in online learning with feedback graphs.
method Derive instance-specific lower bounds and present asymptotically optimal algorithm TaS-FG.
result TaS-FG is asymptotically optimal and efficient across different graph configurations.
Study of online learning with noisy feedback in adversarial settings.
problem Adversarial online learning with noisy feedback.
method Models of adversarial online learning with binary losses xored by noise, considering both constant and variable noise rates.
result Tight regret bounds for learning with noise in adversarial online learning.
One-bit feedback suffices for a bandit problem's optimal strategy.
problem Optimal strategy for multi-armed bandit problem with limited feedback.
method Coding and decoding schemes for one-bit feedback to mimic full-reward feedback.
result Regret ratio approaches 1 with one-bit feedback.
Advocates a local feedback approach for RL in unknown systems.
problem Finding optimal feedback laws in unknown nonlinear dynamical systems.
method Searches over a local feedback representation consisting of an open-loop sequence and an optimal linear feedback law.
result Results in highly efficient training and superior performance compared to global methods.
A method corrects feedback shift in predicting conversion rates with delayed feedback.
problem Delayed feedback leads to mislabeling of positive instances in training data.
method Uses importance weight approach to correct feedback shift.
result Proposed method outperforms existing methods in offline and online experiments.
Study how communication and feedback graphs affect learning outcomes.
problem Understanding the impact of feedback graphs on cooperative online learning.
method Analyzed network regret in terms of the independence number of the strong product of communication and feedback graphs.
result Proved bounds for network regret and demonstrated the non-improvable nature of positive results in pathological cases.
New method uses correlated auxiliary feedback to reduce regret in parameterized bandits.
problem Reducing regret in parameterized bandits with correlated auxiliary feedback.
method Develops a reward estimator using auxiliary feedback with tight confidence bounds.
result Shows significant reduction in regret compared to standard methods.
Test for linearizing 2-input systems with 2D feedback.
problem Linearizability of two-input systems by feedback.
method Algorithmic test for 2D endogenous feedback.
result Systematic derivation of flat outputs.
Reduces user feedback needed for accurate recommender systems.
problem Limited user feedback in recommender systems.
method Partial Bandit and Semi-Bandit approach for efficient user feedback retrieval.
result Similar global accuracy and learning efficiency with reduced feedback.
DeepJSCC-f uses feedback to improve image transmission quality.
problem Improve image transmission quality in feedback channels.
method Autoencoder-based joint source-channel coding (DeepJSCC-f) exploiting feedback.
result Improves reconstruction quality for fixed-length or variable-length transmission.
We present a study on reinforcement learning (RL) from human bandit feedback for sequence-to-sequence learning, exemplified by the task of bandit neural machine translation (NMT). We investigate the reliability of human bandit feedback, and analyze the influence of reliability on the learnability of a reward estimator,…
The problem of feedback equivalence for control systems is considered. An algebra of differential invariants and criteria for the feedback equivalence for regular control systems are found.
New algorithm for recommending best arms with aggregated feedback.
problem Finding the best arm under aggregated feedback when precise rewards are unavailable.
method Gaussian Process Optimistic Optimisation (GPOO) algorithm with adaptive tree construction.
result The proposed algorithm achieves new simple regret bounds with aggregated feedback.
Study of reinforcement learning with additional feedback observations.
problem Episodic reinforcement learning in Markov decision processes with feedback observations.
method Formalization of feedback graph, model-based algorithms leveraging feedback, regret bound analysis.
result Regret bound depends only on the size of the maximum acyclic subgraph of the feedback graph.
CausalRM models rewards from user feedback, overcoming noise and bias.
problem Aligning language models with user preferences from noisy, biased feedback.
method Causal-theoretic reward modeling framework addressing noise and bias in observational feedback.
result CausalRM learns accurate reward signals from noisy and biased observational feedback.
Paper tackles noisy bandit feedback for multiclass classification.
problem Learning multiclass classifier with corrupted feedback.
method Proposes an unbiased estimator technique to estimate noise rates and an end-to-end framework.
result Algorithm achieves mistake bounds of O ( T ) O(\sqrt{T}) O ( T ) in high noise and O ( T i c e f r a c 23 ) O(T^{
icefrac{2}{3}}) O ( T i ce f r a c 2 3 ) in worst case. We propose a generalization of the best arm identification problem in stochastic multi-armed bandits (MAB) to the setting where every pull of an arm is associated with delayed feedback. The delay in feedback increases the effective sample complexity of standard algorithms, but can be offset if we have access to partial…
Active learning framework for optimizing human preferences in reinforcement learning.
problem Selecting most informative feedback for training models of human preferences.
method Proposes an active learning framework to collect preferential feedback online or offline.
result Errors in DPO logit estimates diminish with more feedback.
Easy conditions found for simplifying complex systems.
problem Linearizing complex two-input systems.
method Endogenous dynamic feedback with a dimension of at most two.
result Necessary and sufficient conditions for linearizability.
Paper studies CLO with partial feedback, improving decision-making in uncertain contexts.
problem Improving decision-making in contexts with uncertain cost coefficients using partial feedback.
method Unified class of offline learning algorithms for CLO with different types of feedback, using IERM framework.
result Fast-rate regret bound for IERM with partial feedback and misspecified model classes.
We use online convex optimization (OCO) for setpoint tracking with uncertain, flexible loads. We consider full feedback from the loads, bandit feedback, and two intermediate types of feedback: partial bandit where a subset of the loads are individually observed and the rest are observed in aggregate, and Bernoulli feed…
CAFL breaks feedback loops in recommender systems using causal inference.
problem Feedback loops in recommender systems compromise recommendation quality and homogenize user behavior.
method Causal Adjustment for Feedback Loops (CAFL) algorithm that breaks feedback loops using causal inference.
result CAFL improves recommendation quality compared to prior correction methods.
Unified framework for expert selection with bandit and lower-bound feedback.
problem Selecting the best expert in scenarios with bandit feedback and lower-bound information.
method Introduces a new feedback model combining bandit and lower-bound information, proving optimal regret bounds for modified Exp3 algorithms.
result Optimal regret bounds for modified Exp3 algorithms, generalizing both bandit and full-information settings.
New algorithms handle online prediction with bandit and delayed feedback, improving regret bounds.
problem Achieving finite bounds on surrogate regret with limited feedback.
method Proposed algorithms for bandit and delayed feedback, including inverse-weighted gradient and pseudo-inverse matrix estimators.
result Achieved improved surrogate regret bounds of O ( K T ) O(\sqrt{KT}) O ( K T ) and O ( T 2 / 3 ) O(T^{2/3}) O ( T 2/3 ) . GAIF enhances online multiple testing with feedback, improving statistical power.
problem Sequential online multiple testing with delayed feedback.
method GAIF framework using dynamic threshold adjustment and feedback-driven model selection.
result Improves statistical power through feedback-driven model selection.
New method provides fine-grained feedback on interactive student programs.
problem Time-consuming manual grading of interactive student programs.
method Meta-exploration approach using reinforcement learning.
result 94.3% accuracy in providing fine-grained feedback.
Improves labeling quality in machine learning with pairwise feedback.
problem Scalability and quality of labeled datasets in machine learning.
method Incorporates pairwise feedback into the programmatic creation of labeled datasets.
result Even a small number of pairwise feedback sources can substantially improve label quality.
The paper proposes a new method for learning reward models from ordinal feedback, improving upon binary feedback.
problem Learning reward models from human preferences using binary feedback discards useful samples and loses fine-grained information.
method The paper introduces a framework for learning reward models under ordinal feedback, generalizing the Bradley-Terry model.
result Ordinal feedback reduces the Rademacher complexity compared to binary feedback, leading to better reward learning.
We study an online learning framework introduced by Mannor and Shamir (2011) in which the feedback is specified by a graph, in a setting where the graph may vary from round to round and is \emph{never fully revealed} to the learner. We show a large gap between the adversarial and the stochastic cases. In the adversaria…
New algorithms ensure fair selection in combinatorial semi-bandit with unrestricted delays.
problem Fair selection in stochastic combinatorial semi-bandit with delayed feedback.
method Introduced merit-based fairness constraints and new bandit algorithms for reward and fairness.
result Achieved sublinear expected reward and fairness regrets with dependence on delay distribution quantiles.
New method uses feedback to improve deep reinforcement learning efficiency.
problem Vast amounts of data needed for reasonable performance in deep reinforcement learning.
method Binary corrective feedback combined with probabilistic conditional exploration.
result Achieves drastic improvements in sample efficiency and robustness.
The paper tackles combinatorial pure exploration with various feedback structures and proposes efficient algorithms.
problem Identifying the optimal action in a combinatorial space with limited feedback and nonlinear rewards.
method Designs polynomial-time adaptive algorithms for CPE-BL and CPE-PL, providing sample complexity analyses.
result The proposed algorithms achieve sample complexity close to lower bounds and outperform existing methods.
This study compares traditional ML and deep learning for classifying multilingual user feedback.
problem Classifying multilingual user feedback for software development teams.
method Comparison of traditional machine learning and deep learning approaches.
result Traditional machine learning can achieve comparable results to deep learning for multilingual user feedback classification.
New method for online learning IC models with node-level feedback.
problem Learning IC models with node-level feedback in social networks.
method Detailed analysis and online algorithm with O ( T ) \mathcal{O}( \sqrt{T}) O ( T ) cumulative regret. result First confidence-region result and online algorithm for IC models with node-level feedback.
Graph-based feedback improves bandit algorithms' performance.
problem Stochastic multi-armed bandit problem with graph feedback.
method Analysis of Thompson Sampling and UCB algorithms in graph-based feedback setting.
result Regret bounds that combine graph structure and arm means gaps.
New Q-learning algorithms reduce regret in inventory control problems.
problem Efficiently learning optimal policies in inventory control problems with limited feedback.
method Proposed Elimination-Based Half-Q-Learning (HQL) and Full-Q-Learning (FQL) algorithms with theoretical regret bounds.
result HQL incurs i l d e O ( H 3 T ) ilde{\mathcal{O}}(H^3\sqrt{ T}) i l d e O ( H 3 T ) regret, FQL incurs i l d e O ( H 2 T ) ilde{\mathcal{O}}(H^2\sqrt{ T}) i l d e O ( H 2 T ) regret, independent of state and action space sizes.