Benchmarking Imitation Learning Algorithms Under Computational Constraints

Imitation learning enables neural networks (students) to imitate humans, optimization algorithms and other neural networks (teachers) in solving complex tasks. However, difficulties arise when the students are qualitatively different than teachers and therefore cannot directly imitate them using standard imitation learning methods. The differences may arise in their sensory capabilities, dynamics, or computational capabilities.

We at AILS have developed a method to train strong students from teachers even under severe differences in their sensory capabilities. Now we want to test how our method and others perform under large deviations in the agents’ computational capabilities.

Project goal

The main goal of the project is to evaluate our method and other existing imitation learning algorithms under large differences in computational capabilities between teacher and student, such as when the student uses a significantly smaller neural network or a fundamentally different architecture.

Deliverables:

1. Evaluation of imitation learning algorithms when transferring from a large teacher network to a substantially smaller student network.
2. Evaluation of transfer between different neural network architectures, such as recurrent or attention-based teachers and simpler feedforward students.
3. Evaluation of memory-rich teachers and memory-constrained students, where the teacher can use long observation histories while the student has limited or no memory.
4. Optional: Investigation of transfer from computationally expensive trajectory-level planners to lightweight reactive neural policies suitable for real-time deployment.
5. Optional: Exploration of other computational asymmetries and promising research directions based on the candidate’s interests.

Required knowledge and qualifications

• Basic knowledge of machine learning.
• Programming experience in Python.
• Interest in reinforcement learning, imitation learning, and neural network architectures.

Experience with PyTorch, RL/IL, and robotics or simulation environments is advantageous.

The project provides an opportunity to gain practical experience in imitation and reinforcement learning, neural network design and evaluation, and research on learning under computational constraints.