Aims to learn from multiple unpredictable teachers with minimal interaction.
problem Learning from multiple non-deterministic teachers with low interaction cost.
method Develops a framework and an active learning algorithm to estimate a distribution over policy space.
result Significantly reduces interaction with teachers without compromising performance.
A new RL framework helps align RL solutions with existing practices.
problem Discrepancies between RL solutions and existing procedures.
method Student-teacher RL mechanism with a constraint on policy difference.
result Proves asymptotic optimality and effectiveness in multiple scenarios.
ReOPD uses pre-collected teacher trajectories to distill knowledge from multi-turn interactions.
problem The cost of fully online on-policy distillation for multi-turn interactions.
method ReOPD, an off-environment alternative that reuses pre-collected teacher trajectories as replayed prefixes, addressing the prefix trap and distribution shift.
result ReOPD preserves or improves OPD-level accuracy, uses zero tool calls, and is at least 4imes faster per training step. Researchers study how teachers' advising relationships influence their perceptions of satisfaction and students, not policy influence.
problem Understanding the relationship between teachers' advising relationships and their perceptions of satisfaction and students.
method Proposed a novel joint model of network and item responses (JNIRM) with correlated latent variables.
result Teachers' advising relationships contribute more to satisfaction and students than to influence over educational policies.
While bigger and deeper neural network architectures continue to advance the state-of-the-art for many computer vision tasks, real-world adoption of these networks is impeded by hardware and speed constraints. Conventional model compression methods attempt to address this problem by modifying the architecture manually …
SDM Policy accelerates inference for robotic tasks while maintaining high action quality.
problem Prolonged inference times in diffusion-based policies for high-frequency control tasks.
method Two-stage optimization: score matching and distribution matching; dual-teacher mechanism.
result 6x inference speedup with state-of-the-art action quality.
Proposes a method to mitigate compounding errors in sequence prediction models.
problem Compounding errors in generated sequences at test time due to exposure bias.
method Curriculum learning based method that gradually replaces deterministic teacher policy with stochastic policy.
result Improves generalization in neural language models and performs well when used with scheduled sampling.
TREK uses distillation to help students solve hard problems.
problem Stalled progress on hard prompts when current policy lacks useful reasoning trajectories.
method TREK combines distillation and reinforcement learning to expand student support.
result TREK significantly improves student performance on mathematical reasoning and agentic tasks.
Adaptive teacher improves sample efficiency and mode coverage in sampling tasks.
problem Efficient exploration and mode coverage in sampling tasks with amortized inference.
method An adaptive training distribution (teacher) guides the training of the primary sampler (student).
result Improves sample efficiency and mode coverage across various tasks.
New framework for RL transfer learning with state-action mismatch.
problem High sample complexity in RL from scratch.
method Embeddings to transfer knowledge between MDPs with different state- and action-spaces.
result Successful transfer learning in scenarios with state- and action-space mismatches.
Paper improves teaching by considering learner's preferences and constraints.
problem Teaching without considering learner's preferences and constraints.
method Design of learner-aware teaching algorithms that account for learner's preferences and constraints.
result Significant performance improvements over learner-agnostic teaching.
Real-time policy distillation speeds up and improves reinforcement learning.
problem Slow and inefficient policy distillation in reinforcement learning.
method Simultaneous training and distillation of a teacher and student model.
result Significantly reduced distillation time and improved small model performance.
Interactive teaching speeds up IRL learning with adaptive demonstrations.
problem Tackles the challenge of accelerating IRL learning with teacher assistance.
method Interactive teaching framework where a teacher adapts demonstrations based on the learner's policy.
result Teaching algorithms converge in the omniscient setting, speeding up learning.
AC-Teach uses an ensemble of suboptimal teachers to improve exploration in RL.
problem Improving exploration efficiency in long-horizon tasks with sparse rewards.
method Bayesian Actor-Critic with an ensemble of suboptimal teachers.
result AC-Teach improves sample efficiency over baselines on various tasks.
We describe theoretical bounds and a practical algorithm for teaching a model by demonstration in a sequential decision making environment. Unlike previous efforts that have optimized learners that watch a teacher demonstrate a static policy, we focus on the teacher as a decision maker who can dynamically choose differ…
New algorithm learns Gaussian policies from corrective human feedback, outperforming current methods.
problem Learning from corrective human feedback for complex systems.
method Gaussian Process Coach (GPC) that uses Gaussian Processes and policy uncertainty for optimal feedback selection and learning rate adaptation.
result Demonstrated superior performance in OpenAI Gym benchmarks compared to COACH.
Paper introduces a simulator-free approach to reinforcement learning policy distillation.
problem Learning multiplicity of cases corresponding to a given action in reinforcement learning.
method Generative adversarial approach to find multiple exemplars for each output class.
result Improves over state-of-the-art on data-free learning of student networks.
Transformers can implement reinforcement learning algorithms from data without updates.
problem Training reinforcement learning algorithms from data without parameter updates.
method Design a teacher-mimicking training procedure for transformers to implement policy-improvement methods.
result Gradient flow converges to an optimal parameter manifold corresponding to the desired RL update.
Dual active learning improves RLHF by selecting optimal conversations and teachers.
problem Efficiently aligning LLMs with human preferences using RLHF from feedback.
method Offline RL for conversation and teacher selection, dual active reward learning, pessimistic RL.
result The proposed algorithm achieves minimal generalized variance and outperforms state-of-the-arts.
Methods for learning to search for structured prediction typically imitate a reference policy, with existing theoretical guarantees demonstrating low regret compared to that reference. This is unsatisfactory in many applications where the reference policy is suboptimal and the goal of learning is to improve upon it. Ca…
Variational inference improves hierarchical imitation learning of control programs.
problem Learning structured control policies from demonstrations.
method Variational inference for discovering hierarchical structure in observation-action traces.
result Variational inference leads to more efficient and generalized control policies.
Teaches AI models to learn effectively through dynamic loss functions.
problem Optimizing machine learning models' performance through dynamic loss functions.
method Develops a method for a teacher model to dynamically output loss functions for a student model, enabling gradient-based optimization.
result Significantly improves the performance of various student models in real-world tasks.
Many currently deployed Reinforcement Learning agents work in an environment shared with humans, be them co-workers, users or clients. It is desirable that these agents adjust to people's preferences, learn faster thanks to their help, and act safely around them. We argue that most current approaches that learn from hu…
Algorithm safely learns from sub-optimal baseline policies while satisfying constraints.
problem Safe reinforcement learning with constraints when baseline policy is sub-optimal.
method Iterative policy optimization alternating between return maximization, baseline distance minimization, and constraint projection.
result Consistently outperforms baselines, achieving 10x fewer constraint violations and 40% higher reward.
AutoDIME automates design of multi-agent environments for RL.
problem Designing multi-agent environments for reinforcement learning is challenging.
method Developed intrinsic teacher rewards for multi-agent settings and evaluated them in various tasks.
result Value disagreement was found to be most consistent and effective across tasks.
Proposes RaT to mitigate bias in student-teacher estimation.
problem Systematic bias in teacher's predictions propagates to student model.
method Uses teacher to estimate residuals in student's predictions.
result RaT method reduces teacher bias effect and achieves optimal rate.
Enhances model compression with multi-teacher knowledge distillation.
problem Uncertainty evaluation and diverse teacher expertise in model deployment.
method Bayesian inference and teacher-informed prior with entropy-based weighting.
result Improved predictive accuracy and robust uncertainty quantification.
Improved knowledge distillation using a teacher assistant to bridge the gap between student and teacher networks.
problem Large neural networks are hard to deploy on edge devices due to size constraints.
method Introduce multi-step knowledge distillation with an intermediate-sized teacher assistant.
result The proposed multi-step distillation method improves student network performance.
Student-teacher learning improves generalization with noisy inputs.
problem Transfer knowledge from clean inputs to noisy inputs.
method Analyzes student-teacher learning using deep linear networks and experiments with nonlinear networks.
result Three factors are vital for success: zero training loss, teacher knowledge, and feature decomposition.
Paper proposes a novel policy distillation method for better order execution in noisy markets.
problem Effective order execution in noisy and imperfect market conditions.
method Policy distillation method to guide reinforcement learning towards optimal trading strategies.
result Significant improvements over various baselines in order execution.
Conditional T/S learning improves student model performance by selectively learning from teacher or ground truth.
problem Teacher's occasional wrong guidance leads to suboptimal student model performance.
method Proposes a conditional T/S learning scheme where the student selectively chooses between teacher and ground truth based on teacher correctness.
result The conditional learning achieves significant performance improvements over traditional T/S learning.
Paper proposes a new RL approach combining IL and RL methods to improve decision-making.
problem Challenges in RL with large state and action spaces, and difficulty in reward determination.
method Combines Imitation Learning and RL methods (SARSA and A3C) to learn sequential decision-making policies.
result Significantly decreases human effort and exploration time in learning decision-making policies.
In this paper, we make an important step towards the black-box machine teaching by considering the cross-space machine teaching, where the teacher and the learner use different feature representations and the teacher can not fully observe the learner's model. In such scenario, we study how the teacher is still able to …
We generalise the problem of inverse reinforcement learning to multiple tasks, from multiple demonstrations. Each one may represent one expert trying to solve a different task, or as different experts trying to solve the same task. Our main contribution is to formalise the problem as statistical preference elicitation,…
Subclass distillation improves small models by matching teacher's subclass probabilities.
problem Improving small models trained on limited data.
method Train a small model to match probabilities of subclasses invented by a large teacher model.
result Better small models trained on limited data.
A new teacher-class network method compresses DNNs by distributing knowledge to multiple student networks.
problem Overwhelming size of Deep Neural Networks (DNNs).
method Single teacher with multiple student networks, transferring knowledge to each student.
result The combined knowledge of the class of students achieves better performance and reduces parameters.
A new distillation method transfers channel information from teacher to student.
problem Transfer knowledge from teacher to student with fewer parameters and calculations.
method Channel Distillation (CD) and Guided Knowledge Distillation (GKD) with loss decay.
result Achieved 27.68% top-1 error on ImageNet with ResNet18, outperforming state-of-the-art methods.
Master-slave architecture tackles combinatorial multi-armed bandits with diversity constraints.
problem Solving top-K combinatorial multi-armed bandits with non-linear feedback and diversity constraints. method Master-slave architecture with six slave models, teacher learning, and policy co-training.
result Significantly outperforms existing algorithms in synthetic and real datasets.
Teaches manifolds from teacher's structured data.
problem Learning a manifold from teacher's structured data.
method Extends existing approaches to learning from randomly sampled data points, considering structured data provided by a teacher.
result Demonstrations can significantly reduce data points needed for manifold learning.
Distilled models often fail to match teacher models, despite improving generalization.
problem The discrepancy between teacher and student predictive distributions remains large.
method Investigated the optimization difficulties and dataset details affecting student performance.
result Optimizing for matching the teacher does not always lead to better generalization.
A new method transfers knowledge without data, matching teacher's predictions closely.
problem Lack of access to training data for knowledge transfer.
method Adversarial training to match teacher's predictions without data.
result Zero-shot student performs well on CIFAR10, improving state-of-the-art.
Random feature models can outperform a weak teacher with early stopping.
problem Generalization from a weak to a strong model in random feature networks.
method Random feature models, early stopping, proving weak-to-strong generalization.
result Random feature models can outperform a weak teacher with early stopping.
This paper investigates teacher hacking during language model distillation and proposes methods to mitigate it.
problem Teacher hacking during language model distillation, leading to suboptimal performance.
method A controlled experimental setup involving an oracle LM, teacher LM, and student LM, using fixed offline or online data generation techniques.
result Data diversity is the key factor in preventing teacher hacking during distillation.
This paper proposes a method to embed teacher knowledge into a student network without increasing parameters.
problem The need for portable neural networks on mobile devices with limited resources.
method Feature embedding approach to distill knowledge from a teacher network to a student network without introducing new parameters.
result The proposed method maintains the performance of the teacher network while significantly reducing computational and storage complexity.
Under-parameterized networks can either copy or average teacher weights, leading to universal optimal solutions.
problem Approximating a teacher network with an under-parameterized student network.
method Analyzing shallow neural networks with erf activation function and unitary teacher weights, proving copy-average configurations are critical points and finding the optimal solution.
result The optimal solution for under-parameterized networks has a universal structure, whether copying or averaging teacher neurons.
Study reveals how initial weights influence convergence in deep ReLU networks.
problem Understanding the dynamics and generalization of deep ReLU networks.
method Teacher-student setting, gradient analysis, and activation assumptions.
result Initial weights close to teacher nodes lead to faster convergence, and fan-out weights of other nodes converge to zero in over-parameterized cases.
Study shows how task similarity affects forgetting in teacher-student setup.
problem Catastrophic forgetting in continual learning.
method Extended teacher-student setup to multiple teachers, analyzing similarity between tasks.
result Task similarity, whether at readouts or features, influences forgetting and transfer.
AMIGo uses adversarial intrinsic goals to teach RL agents new skills.
problem Learning in sparse reward environments.
method Adversarial intrinsic goals to generate a curriculum for a student policy.
result AMIGo enables agents to learn new skills without extrinsic rewards.