arXiv research

Reinforcement learning with trajectory feedback instead of state-action rewards.

Robot motions in the presence of humans should not only be feasible and safe, but also conform to human preferences. This, however, requires user feedback on the robot's behavior. In this work, we propose a novel approach to leverage the user's brain signals as a feedback modality in order to decode the judgment of rob…

Learning optimal feedback control laws capable of executing optimal trajectories is essential for many robotic applications. Such policies can be learned using reinforcement learning or planned using optimal control. While reinforcement learning is sample inefficient, optimal control only plans an optimal trajectory fr…

Study shows reinforcement learning is possible with once-per-episode feedback.

LLM trading agents show risk feedback can improve alignment without fine-tuning.

Unified LP framework for offline reward learning from human demonstrations and feedback.

Proposes a new theoretical framework for PbRL that requires less human feedback.

Algorithm for online decision making with unknown dynamics and aggregate feedback.

Meta-algorithm for efficient reinforcement learning from human preferences.

Paper tackles offline preference-based RL with human feedback.

This work presents a methodology to design trajectory tracking feedback control laws, which embed non-parametric statistical models, such as Gaussian Processes (GPs). The aim is to minimize unmodeled dynamics such as undesired slippages. The proposed approach has the benefit of avoiding complex terramechanics analysis …

Learning weights in a spiking neural network with hidden neurons, using local, stable and online rules, to control non-linear body dynamics is an open problem. Here, we employ a supervised scheme, Feedback-based Online Local Learning Of Weights (FOLLOW), to train a network of heterogeneous spiking neurons with hidden l…

Active learning method estimates nonlinear systems efficiently.

Neural ODEs control graph dynamics with low energy feedback.

Extended flatness approach for discrete-time systems considers forward and backward shifts.

Deep neural nets approximate random dynamical system trajectories uniformly in time.

We consider the problem of learning from sparse and underspecified rewards, where an agent receives a complex input, such as a natural language instruction, and needs to generate a complex response, such as an action sequence, while only receiving binary success-failure feedback. Such success-failure rewards are often …

RLHF fails when humans only partially observe, leading to inflated or overjustified feedback.

Study preference-based reinforcement learning in episodic kernel MDPs.

Unsupervised exploration and representation learning become increasingly important when learning in diverse and sparse environments. The information-theoretic principle of empowerment formalizes an unsupervised exploration objective through an agent trying to maximize its influence on the future states of its environme…

Enhanced feedback model improves sample-efficiency in POMDPs.

Algorithm learns NE in imperfect information games with imperfect feedback.

Autoregressive state transitions, where predictions are conditioned on past predictions, are the predominant choice for both deterministic and stochastic sequential models. However, autoregressive feedback exposes the evolution of the hidden state trajectory to potential biases from well-known train-test discrepancies.…

A new method infers neural trajectories in real-time, improving experimental design.

Probabilistic vehicle trajectory prediction is essential for robust safety of autonomous driving. Current methods for long-term trajectory prediction cannot guarantee the physical feasibility of predicted distribution. Moreover, their models cannot adapt to the driving policy of the predicted target human driver. In th…

We study control systems invariant under a Lie group with application to the problem of nonlinear trajectory planning. A theory of symmetry reduction of exterior differential systems is employed to demonstrate how symmetry reduction and reconstruction is effective in the explicit, exact construction of planned system t…

Study learns optimal strategies in imperfect information games with self-play.

Geometric framework for dynamic feedback linearization of control systems with symmetry.

New budget quantifies drift in closed-loop learning, improving reproducibility.

Paper analyzes online reinforcement learning with outcome-based feedback, providing efficient algorithms and fundamental limits.

Trajectory-level supervision allows efficient offline reinforcement learning.

Model predicts multi-agent trajectories using a differentiable simulator.

Adaptive OMD reduces variance in learning optimal strategies for imperfect information games.

In preference-based reinforcement learning (RL), an agent interacts with the environment while receiving preferences instead of absolute feedback. While there is increasing research activity in preference-based RL, the design of formal frameworks that admit tractable theoretical analysis remains an open challenge. Buil…

Paper classifies conic submanifolds in control systems.

This thesis tackles learning reward functions from human comparative feedback.

For sophisticated reinforcement learning (RL) systems to interact usefully with real-world environments, we need to communicate complex goals to these systems. In this work, we explore goals defined in terms of (non-expert) human preferences between pairs of trajectory segments. We show that this approach can effective…

Algorithm learns actions from past states in complex tasks.

This work proposes a new method for simultaneous probabilistic identification and control of an observable, fully-actuated mechanical system. Identification is achieved by conditioning stochastic process priors on observations of configurations and noisy estimates of configuration derivatives. In contrast to previous w…

Interactive learning with hindsight instruction feedback achieves better performance than traditional methods.

We seek a discussion about the most suitable feedback control structure for stock trading under the consideration of proportional transaction costs. Suitability refers to robustness and performance capability. Both are tested by considering different one-step ahead prediction qualities, including the ideal case, correc…

Paper develops methods to optimize policies directly from human feedback without reward inference.

Agents learn to outperform in trading by using past and current prices.

A new framework uses stochastic optimal control to estimate rare events more accurately.

Imitation learning is a control design paradigm that seeks to learn a control policy reproducing demonstrations from expert agents. By substituting expert demonstrations for optimal behaviours, the same paradigm leads to the design of control policies closely approximating the optimal state-feedback. This approach requ…

This paper introduces a hierarchical associative memory model with multiple layers.

OSGM uses online learning to adapt stepsize for faster convergence.

New RLHF algorithm identifies optimal policies from human feedback without explicit reward inference.