We consider combinatorial online learning with subset choices when only relative feedback information from subsets is available, instead of bandit or semi-bandit feedback which is absolute. Specifically, we study two regret minimisation problems over subsets of a finite ground set , with subset-wise relative prefe…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “relative feedback”
A new algorithm for conversational recommendation systems using dueling bandits in GLMs.
problem Limited user feedback in existing conversational bandit methods.
method Integrates dueling bandits with relative feedback in generalized linear models.
result Theoretical and empirical validation of ConDuel's efficacy.
New algorithms improve on bandit feedback in matrix games with unknown payoff matrices.
problem Improving performance in matrix games with unknown payoff matrices and bandit feedback.
method Regret analyses of variants of UCB and K-learning.
result New algorithms achieve lower regret compared to adversarial bandit algorithms.
Proposes a method to select fair performance metrics through metric elicitation.
problem Choosing fair performance metrics in multiclass classification with multiple sensitive groups.
method Metric elicitation strategy that requires only relative preference feedback and is robust to noise.
result Elicits group-fair performance metrics for multiclass classification problems.
GPT-4 assesses its confidence in answering USMLE questions with and without feedback.
problem Understanding AI's performance in healthcare applications, especially in sensitive areas like medical education.
method Used a prompting technique to evaluate GPT-4's confidence scores before and after answering USMLE questions, categorized into with and without feedback.
result Feedback influences relative confidence but doesn't consistently increase or decrease it.
Multi-armed bandit(MAB) problem is a reinforcement learning framework where an agent tries to maximise her profit by proper selection of actions through absolute feedback for each action. The dueling bandits problem is a variation of MAB problem in which an agent chooses a pair of actions and receives relative feedback…
In display advertising, predicting the conversion rate, that is, the probability that a user takes a predefined action on an advertiser's website, such as purchasing goods is fundamental in estimating the value of displaying the advertisement. However, there is a relatively long time delay between a click and its resul…
Study analyzes feedback complexity for sparse feature retrieval in deep networks.
problem Learning sparse superposed features with feedback.
method Analysis of feedback complexity in sparse settings, including triplet comparisons.
result Establishes tight bounds and strong upper bounds for feature recovery.
Paper tackles robustness in reward learning with partial identifiability.
problem Partial identifiability in reward learning leads to unreliable target reward recovery.
method Introduces a robust approach to maximize performance with respect to the worst-case reward in the feasible set.
result Develops Rob-ReL, an algorithm that maximizes performance under worst-case identifiability conditions.
New algorithms handle online prediction with bandit and delayed feedback, improving regret bounds.
problem Achieving finite bounds on surrogate regret with limited feedback.
method Proposed algorithms for bandit and delayed feedback, including inverse-weighted gradient and pseudo-inverse matrix estimators.
result Achieved improved surrogate regret bounds of and .
The paper tackles efficient change point detection with limited samples.
problem Identifying multiple change points with minimal queries in noisy environments.
method Adaptive algorithm that first detects likely change points and refines their locations.
result The sample complexity is jointly governed by jump magnitudes and change point positions.
ADPO optimizes relative advantage in reinforcement learning from human feedback.
problem Optimizing policy alignment in reinforcement learning from human preferences.
method ADPO explicitly parameterizes the optimal structure through anchored logits, decoupling response quality from prior popularity.
result Empirically, ADPO achieves state-of-the-art performance on reasoning tasks, outperforming GRPO by 30.9 percent.
Graph-based approach repairs programs from diagnostic feedback.
problem Learning to repair programs from limited labeled data and compiler error messages.
method Introduces program-feedback graph and graph neural network for reasoning, and self-supervised learning with unlabeled programs.
result DrRepair significantly outperforms prior work, achieving high repair rates.
Study optimizes portfolio to minimize relative drawdown duration, penalizing unfavorable performance states.
problem Minimizing relative drawdown duration in portfolio optimization relative to a benchmark.
method Introduces a benchmark-relative drawdown-duration criterion penalizing unfavorable performance states. Uses a one-dimensional Markovian representation and Hamilton-Jacobi-Bellman equation.
result Derives explicit projection-based characterization of the optimal feedback control and identifies geometric settings for unique strong solutions.
What is the most statistically efficient way to do off-policy evaluation and optimization with batch data from bandit feedback? For log data generated by contextual bandit algorithms, we consider offline estimators for the expected reward from a counterfactual policy. Our estimators are shown to have lowest variance in…
QS-BO optimizes functions using only rank-based feedback.
problem Optimizing expensive functions with unreliable or unavailable metric values.
method Quantile-scaling pipeline to convert ranks into Gaussian targets.
result QS-BO consistently achieves lower objective values and is statistically significant.
Paper improves parameter estimation of continuous distributions using preference feedback.
problem Improving parameter estimation of continuous distributions.
method Preference-based M-estimators and deterministic preferences.
result Preference-based estimators achieve an estimation error scaling of O(1/n), significantly faster than sample-only methods.
Given a binary prediction problem, which performance metric should the classifier optimize? We address this question by formalizing the problem of Metric Elicitation. The goal of metric elicitation is to discover the performance metric of a practitioner, which reflects her innate rewards (costs) for correct (incorrect)…
Algorithm identifies best item from subsets with random utility model feedback.
problem PAC learning the best item from subsets with random utility model feedback.
method Pairwise relative counts and hierarchical elimination for learning algorithm.
result Near-optimal PAC sample complexity guarantee for identifying ε-optimal item.
DRO-REBEL improves LLM alignment by robustly updating models online.
problem Overfitting and drifting of LLMs during RLHF.
method DRO-REBEL uses type- Wasserstein, KL, and ambiguity sets for robust online updates.
result DRO-REBEL achieves faster convergence and better performance than prior methods.
Random Feedback Alignment helps solve low-rank matrix factorization problems.
problem Solving low-rank matrix factorization problems efficiently.
method Random Feedback Alignment (FA) as a bio-plausible alternative to gradient descent.
result FA converges to the optimal solution when the rank of the matrix is at least the rank of the input matrix.
In recent years, Variational Autoencoders (VAEs) have been shown to be highly effective in both standard collaborative filtering applications and extensions such as incorporation of implicit feedback. We extend VAEs to collaborative filtering with side information, for instance when ratings are combined with explicit t…
New algorithms reduce forecasting errors by leveraging optimistic learning and hinting.
problem Real-time climate forecasting with delayed feedback.
method Developed optimistic online learning algorithms that reduce regret under delayed feedback.
result Optimistic learning algorithms achieve optimal regret guarantees under delayed feedback.
Speculation and Power Lawq-fin.ST
It is now well established empirically that financial price changes are distributed according to a power law, with cubic exponent. This is a fascinating regularity, as it holds for various classes of securities, on various markets, and on various time scales. The universality of this law suggests that there must be som…
We study the K-armed dueling bandit problem, a variation of the standard stochastic bandit problem where the feedback is limited to relative comparisons of a pair of arms. The hardness of recommending Copeland winners, the arms that beat the greatest number of other arms, is characterized by deriving an asymptotic regr…
Think Before You Duel: Understanding Complexities of Preference Learning under Constrained Resourcescs.LG
A dueling bandit problem with resource constraints is solved using EXP3.
problem Maximizing rewards in dueling bandits with resource constraints.
method EXP3 algorithm considering resource consumptions.
result Achieves $ ilde{\mathcal{O}}\left({\frac{OPT^{(b)}}{B}}K^{1/3}T^{2/3}
ight)$ regret.
Last-iterate guarantees for learning in co-coercive games under noisy feedback.
problem Learning in co-coercive games with noisy feedback.
method Vanilla stochastic gradient descent with a new noise model.
result Last-iterate bound of order for co-coercive games.
In machine learning, the notion of multi-armed bandits refers to a class of online learning problems, in which an agent is supposed to simultaneously explore and exploit a given set of choice alternatives in the course of a sequential decision process. In the standard setting, the agent learns from stochastic feedback …
We study the problem of combining multiple bandit algorithms (that is, online learning algorithms with partial feedback) with the goal of creating a master algorithm that performs almost as well as the best base algorithm if it were to be run on its own. The main challenge is that when run with a master, base algorithm…
The paper tackles adversarial attacks on recurrent neural networks.
problem Adversarial attacks on recurrent neural networks are easy and lack theoretical guarantees.
method Inspired by dynamical systems theory, the paper dynamically computes adversarial perturbations for each timestep of the input sequence.
result The paper provides theoretical guarantees on the existence of adversarial examples and robustness margins.
Develops a framework for analyzing multi-agent and many-body systems with feedback loops.
problem Optimal order of multi-agent and general many-body systems
method Derive macroscopic properties and optimal degree of order
result Optimal degree of order balances productivity, stability, and adaptability
New algorithm for minimizing regret in adversarial dueling bandits.
problem Minimizing regret in dueling bandits with adversarial feedback.
method Introduced an algorithm with -round regret of .
result Algorithm achieves regret lower bound.
Proposes DRRO to mitigate over-optimization in RLHF from human feedback.
problem Over-optimization due to reward misspecification in RLHF.
method Wasserstein distributionally robust regret optimization (DRRO).
result DRRO mitigates over-optimization more effectively than existing baselines.
The procure to pay process (P2P) in large enterprises is a back-end business process which deals with the procurement of products and services for enterprise operations. Procurement is done by issuing purchase orders to impaneled vendors and invoices submitted by vendors are paid after they go through a rigorous valida…
This research examines relationship between staging of Venture Capital (VC) investments and social feedback visible in publicly available data on the Web. We address the question of Venture Capital investment sensitivity to performance and prospects of new venture, given as likelihood of obtaining future financing, ava…
Paper models and compresses wideband CSI feedback in FDD MIMO systems.
problem Fundamental limits of channel state information (CSI) feedback in FDD massive MIMO systems.
method Modeling CSI as a Gaussian-mixture source with latent geometry states, proposing Gaussian-mixture transform coding (GMTC).
result Near-optimal CSI compression achieved through state-adaptive transform coding without large neural encoders.
New insights into cascade feedback linearization of control systems.
problem Obtaining a cascade feedback linearization for invariant control systems.
method Introducing truncated versions of operators from the calculus of variations to prove new theorems.
result Established new geometry and foundational theorems for future work.
New method learns from either positive or negative feedback alone.
problem Limited applicability of existing preference optimization methods in scenarios with only unpaired feedback.
method Decouples learning from positive and negative feedback, using expectation-maximization (EM) to optimize probability of positive outcomes and explicitly incorporate negative examples.
result Stable learning from negative feedback alone demonstrated.
User preferences for items can be inferred from either explicit feedback, such as item ratings, or implicit feedback, such as rental histories. Research in collaborative filtering has concentrated on explicit feedback, resulting in the development of accurate and scalable models. However, since explicit feedback is oft…
This paper improves image retrieval accuracy through novel relevance feedback methods.
problem Improving image retrieval accuracy in Content-Based Image Retrieval (CBIR).
method Novel addition to feature re-weighting and classification techniques, focusing on 0-th iteration improvement.
result Significantly improved retrieval accuracy from relevance feedback.
New model improves recommendation systems by analyzing user-item interactions.
problem Improving recommendation systems for better user-item interactions.
method Sliced Anti-symmetric Decomposition (SAD) model using tensor decomposition.
result SAD produces the most consistent personalized preferences compared to SOTA models.
Paper tackles LDP bandits learning with improved results and sub-linear regret.
problem Contextual bandits learning with LDP privacy constraints.
method Simple black-box reduction frameworks for context-free bandits, extended to GLB.
result First result for BCO with multi-point feedback under LDP, sub-linear regret for GLB.
We propose a sample-efficient alternative for importance weighting for situations where one only has sample access to the probability distribution that generates the observations. Our new method, called Geometric Resampling (GR), is described and analyzed in the context of online combinatorial optimization under semi-b…
We study the -armed dueling bandit problem, a variation of the standard stochastic bandit problem where the feedback is limited to relative comparisons of a pair of arms. We introduce a tight asymptotic regret lower bound that is based on the information divergence. An algorithm that is inspired by the Deterministic…
Study identifies new stable climate states in climate model.
problem Understanding multistability and transitions in climate models.
method Combination of quasipotential theory and manifold learning.
result Discovery of a third stable climate state not previously known.
Explicit Feedbacks Meet with Implicit Feedbacks : A Combined Approach for Recommendation Systemcs.IR
Recommender systems recommend items more accurately by analyzing users' potential interest on different brands' items. In conjunction with users' rating similarity, the presence of users' implicit feedbacks like clicking items, viewing items specifications, watching videos etc. have been proved to be helpful for learni…
Study evaluates new models using human feedback from another model.
problem Evaluate a new model using human feedback collected for another model.
method Formalize problem, propose model-based and model-free estimators, analyze unbiasedness, and empirically evaluate.
result Proposed estimators can predict absolute values, rank, and optimize evaluated policies.
Develops a new model for RLHF accounting for partially observed states and intermediate feedback.
problem Lack of models for partially observed states and intermediate feedback in RLHF.
method PORRL model with cardinal and dueling feedback methods.
result Demonstrates improved learning and alignment with new model-based and model-free methods.