Robot learns user preferences from brain signals.
problem Decoding user preferences for robot motions from brain signals.
method Proposes a novel approach using electroencephalography to decode user preferences from brain signals.
result Brain signals can reliably infer user preferences for robot trajectories.
Neural ODEs control graph dynamics with low energy feedback.
problem Controlling complex dynamical systems on graphs.
method Neural Ordinary Differential Equation Control (NODEC) framework.
result NODEC learns low-energy control signals for graph dynamical systems.
FinRLlama wins FinRL Challenge 2024 by fine-tuning LLMs with market data.
problem Lack of contextual alignment for financial market applications in traditional LLMs.
method Fine-tuning LLaMA-3.2-3B-Instruct model with custom RLMF prompt design integrating historical data and reward feedback.
result RLMF-tuned FinRLlama framework outperforms baseline methods in signal consistency and trading outcomes.
Improves semantic parsing with human feedback in a counterfactual setup.
problem Expensive gold parses or answers for training semantic parsers.
method Convert model outputs to human-understandable statements for feedback.
result Significant improvement in semantic parsing through counterfactual learning.
New framework uses user feedback in CB problems for better decision-making.
problem Improving decision-making in contextual bandit problems with user-triggered feedback.
method Developed a new framework to leverage user-triggered feedback in CB problems, robust to feedback bias.
result Improved regret guarantees for CB algorithms using user feedback.
Paper proposes verifier engineering for improving foundation models.
problem Challenges in providing effective supervision signals for foundation models.
method Leverages automated verifiers to perform verification tasks and deliver feedback.
result Verifier engineering can enhance foundation models' capabilities.
DeepJSCC-f uses feedback to improve image transmission quality.
problem Improve image transmission quality in feedback channels.
method Autoencoder-based joint source-channel coding (DeepJSCC-f) exploiting feedback.
result Improves reconstruction quality for fixed-length or variable-length transmission.
Enhances on-policy RL with past reward statistics and hot-wiring.
problem Challenges of sparse reward signals in on-policy methods.
method Multi-critic supervision and hot-wiring mechanism.
result Improves on-policy learning for sparse reward tasks.
A new method shapes reinforcement learning environments by abstracting large state spaces.
problem Learning in large, noisy environments with sparse feedback.
method Environment shaping using state abstraction.
result Agent's policy in shaped environment preserves near-optimal behavior in original environment.
DOPL learns from preference feedback to solve RMAB problems.
problem Learning optimal decisions in RMAB with limited reward information.
method Direct online preference learning (DOPL) for Pref-RMAB.
result DOPL achieves sublinear regret for RMAB with preference feedback.
Retraining stabilizes model influence on data.
problem Performativity in predictive models leads to feedback loops.
method Developed the stable signal principle to address retraining dynamics.
result Repeated risk minimization converges geometrically to stable signal direction.
Improved flow-based inference speeds up and boosts accuracy for complex simulations.
problem Challenging inverse problems in astronomy, such as modeling strong gravitational lens systems.
method Refines flow-based generative models with simulator feedback for posterior inference.
result Improves accuracy by 53% and speeds up inference by up to 67x.
Paper introduces a novel reward function for noisy financial markets using imitation learning.
problem Noisy reward function in financial markets hinders RL agent performance.
method Integrates imitation learning feedback with reinforcement learning to improve reward function design.
result Improves financial performance metrics compared to traditional benchmarks and RL agents.
A new learning method for prosthetic arms without explicit rewards.
problem Learning a prosthetic arm to interact with users without explicit reward signals.
method Interaction-Grounded Learning, observing multidimensional context and feedback vectors, discovering latent reward signal.
result The algorithm can discover a latent reward signal and ground its policies for successful interaction.
Self-regulation improves sequence-to-sequence learning by choosing feedback types.
problem Different types of feedback have varying costs and effects on learning.
method Self-regulation strategies decide when to ask for different types of feedback.
result Self-regulator discovers optimal cost-quality trade-off by mixing feedback types.
New algorithm tackles non-stationary delayed feedback in recommender systems.
problem Challenges in learning from delayed feedback in non-stationary environments.
method Developed a UCRL-based algorithm for non-stationary, delayed bandits with intermediate observations.
result Sublinear regret guarantees for the proposed algorithm in non-stationary delayed environments.
Neural networks are commonly trained to make predictions through learning algorithms. Contrastive Hebbian learning, which is a powerful rule inspired by gradient backpropagation, is based on Hebb's rule and the contrastive divergence algorithm. It operates in two phases, the forward (or free) phase, where the data are …
Robustly combines supervised and bandit feedback for contextual bandits.
problem Learning from mixed supervised and bandit data with potentially misaligned costs.
method Developed no-regret algorithms robust to misaligned cost distributions.
result Our approach is feasible and helpful in practice, as shown by empirical evaluations.
New method clusters items with bandit feedback without parametric assumptions.
problem Clustering with noisy observations from unknown distributions.
method Kernel-based approach to nonparametric clustering with bandit feedback.
result Adaptive algorithm with theoretical guarantees for unknown signal-to-noise ratio.
Study shows reinforcement learning is possible with once-per-episode feedback.
problem Challenges of reinforcement learning with sparse feedback.
method Introduced a model where trajectory labels are generated by an unknown parametric model and developed an algorithm for sublinear regret.
result Achieved sublinear regret with a statistically and computationally efficient algorithm.
Study tests feasibility of linear programs with bandit feedback.
problem Testing feasibility of unknown linear programs with bandit feedback.
method Developed a novel test based on low-regret algorithms and a nonasymptotic law of iterated logarithms.
result Proved that the test is reliable and adapts to the signal level, with mean sample costs scaling as \( \widetilde{O}(d^2/Γ^2) \).
Proposes a new theoretical framework for PbRL that requires less human feedback.
problem Lack of theoretical work capturing practical PbRL frameworks.
method Introduces a reward-agnostic PbRL framework that acquires exploratory trajectories before human feedback.
result Demonstrates improved sample complexity for learning optimal policies in linear and low-rank MDPs.
CausalRM models rewards from user feedback, overcoming noise and bias.
problem Aligning language models with user preferences from noisy, biased feedback.
method Causal-theoretic reward modeling framework addressing noise and bias in observational feedback.
result CausalRM learns accurate reward signals from noisy and biased observational feedback.
What is the role of social interactions in the creation of price bubbles? Answering this question requires obtaining collective behavioural traces generated by the activity of a large number of actors. Digital currencies offer a unique possibility to measure socio-economic signals from such digital traces. Here, we foc…
Deep COACH learns complex tasks from human feedback in Minecraft.
problem Learning complex behaviors from human feedback efficiently.
method Deep reinforcement learning with policy updates based on human critiques.
result Demonstrated effectiveness in Minecraft with reduced sample complexity.
We present an approach to interactive-predictive neural machine translation that attempts to reduce human effort from three directions: Firstly, instead of requiring humans to select, correct, or delete segments, we employ the idea of learning from human reinforcements in form of judgments on the quality of partial tra…
Self-feeding chatbots learn from user feedback to improve performance.
problem Lack of training data after deployment of dialogue agents.
method Extracts new training examples from user responses and uses feedback to improve dialogue abilities.
result Self-feeding chatbots significantly improve performance on PersonaChat dataset.
Presentation bias is one of the key challenges when learning from implicit feedback in search engines, as it confounds the relevance signal with uninformative signals due to position in the ranking, saliency, and other presentation factors. While it was recently shown how counterfactual learning-to-rank (LTR) approache…
Artificial neural networks are most commonly trained with the back-propagation algorithm, where the gradient for learning is provided by back-propagating the error, layer by layer, from the output layer to the hidden layers. A recently discovered method called feedback-alignment shows that the weights used for propagat…
ERNN improves RNN accuracy and stability with time-delayed self-feedback.
problem Inaccuracy and instability in RNNs.
method Augmenting RNN with a time-delayed self-feedback loop to stabilize hidden state transitions.
result ERNN achieves state-of-the-art results on benchmark datasets.
Binary feedback outperforms ordinal comparisons in ranking recovery.
problem Challenges the conventional wisdom that ordinal comparisons offer richer information.
method Proposes a parametric framework for modeling ordinal paired comparisons, binarizing ordinal data, and proving faster convergence rates for binary comparisons.
result Binarizing ordinal data significantly improves ranking recovery accuracy and exhibits a substantial performance gap.
Feedback improves ESN performance by 30-60% across various tasks.
problem Reducing computational complexity in ESNs for complex sequential data processing.
method State feedback to modify the internal reservoir state of ESNs.
result Feedback significantly improves ESN performance, reducing error by 30-60%.
In this work, we propose a novel recurrent neural network (RNN) architecture. The proposed RNN, gated-feedback RNN (GF-RNN), extends the existing approach of stacking multiple recurrent layers by allowing and controlling signals flowing from upper recurrent layers to lower layers using a global gating unit for each pai…
Dynamic model pruning improves performance on deep neural networks without retraining.
problem High memory and latency requirements for deep neural networks on low-end devices.
method Dynamic allocation of sparsity pattern and feedback signal to reactivate pruned weights.
result Sparse models achieve state-of-the-art performance with no additional retraining.
This paper extends a Kyle model to include price-responsive traders, revealing new dynamics and equilibria.
problem Real-world market dynamics involve price-responsive traders, affecting market equilibrium and insider profits.
method Developed a continuous-time Kyle model with two types of price-responsive traders (momentum and contrarian), leading to a forward-backward Riccati system for equilibrium.
result The model shows that feedback effects can lead to multiple equilibria and amplify price informativeness.
Feedback alignment methods need to be evaluated for accuracy and gradient cosine similarity.
problem Evaluating feedback alignment methods
method Proposed diagnostic evaluation protocol
result Identified silent failures in standard reporting pair
DEFINED uses decision feedback ICL to detect symbols with minimal pilot data.
problem Limited pilot data in wireless receivers.
method In-context learning with decision feedback mechanism.
result Small Transformer trained with DEFINED achieves significant performance improvements.
This paper explores LETOR for E-Com search, addressing practical challenges and reporting key findings.
problem Applying LETOR to E-Com search presents unique challenges.
method Investigates practical challenges in LETOR for E-Com search, including feature representation, relevance judgments, and feedback signal exploitation.
result LETOR methods can effectively optimize combinations of popularity-based and relevance-based features, and order rate is the most robust training objective.
The paper optimizes portfolios using MACD signals derived from price history.
problem Optimizing risky asset portfolios with latent mean-reverting and momentum factors.
method Derives optimal strategies based on MACD signals from EMA processes.
result Establishes admissibility and verification of optimal strategies.
LLM trading agents show risk feedback can improve alignment without fine-tuning.
problem Aligning LLM trading agents with financial risk.
method TradeArena testbed, risk reports, execution simulation, memory replay.
result Risk feedback can improve alignment without fine-tuning, but not universally.
New RLHF framework handles general preference oracles without reward functions.
problem Handling general preference oracles without assuming a reward function.
method Developed a minimax game between two LLMs for RLHF under a general preference oracle, focusing on KL-regularized preference.
result Proposed algorithms for efficient offline and online RLHF learning.
Paper proposes E/PD-Control for better neural network training.
problem Training efficiency and robustness of CNNs in online data flows.
method E/PD-Control combines feedback PD controller with exponential signal.
result Better learning efficiency and robustness demonstrated experimentally.
AutoStan improves Bayesian models via predictive feedback.
problem Improving Bayesian models written in Stan.
method Iterative improvement of Stan models using NLPD and sampler diagnostics feedback.
result AutoStan can autonomously improve diverse Bayesian models across various structures.
We propose a method for learning cyclic causal models from a combination of observational and interventional equilibrium data. Novel aspects of the proposed method are its ability to work with continuous data (without assuming linearity) and to deal with feedback loops. Within the context of biochemical reactions, we a…
We introduce a new class of reinforcement learning methods referred to as {\em episodic multi-armed bandits} (eMAB). In eMAB the learner proceeds in {\em episodes}, each composed of several {\em steps}, in which it chooses an action and observes a feedback signal. Moreover, in each step, it can take a special action, c…
Paper shows RLHF can be solved similarly to standard RL.
problem Difficulty of RLHF compared to standard RL.
method Reduction to reward-based RL techniques.
result RLHF can be solved using existing algorithms for reward-based RL.
DEFINED improves wireless symbol detection with limited pilot data.
problem Efficient symbol detection over block-fading channels with scarce pilot data.
method In-context learning with decision feedback mechanism.
result Significant performance improvements, often needing only a single pilot pair.
A new method optimizes recommender systems by addressing missing-not-at-random implicit feedback.
problem Missing-not-at-random implicit feedback in recommender systems.
method Defining an ideal loss function and proposing an unbiased estimator to optimize relevance.
result The proposed method improves performance, especially for rare items.