Development of Robotics Simulation Environments for Benchmarking Imitation Learning Algorithms

Imitation learning enables neural networks (students) to imitate humans, optimization algorithms and other neural networks (teachers) in solving complex tasks. However, difficulties arise when the students are qualitatively different than teachers and therefore cannot directly imitate them using standard imitation learning methods.

We at AILS have developed a method to train strong students from teachers even under severe differences in the information available to each agent. The method performs well under controlled simulation settings, but implementation and evaluation in high-dimensional robotics environments remains an open question.

Project goal

The main goal of the project is to develop challenging robotics simulation environments and tasks that exhibit teacher-student mismatches and use them to benchmark several imitation learning algorithms. The main emphasis is on software development and evaluation of existing algorithms, with opportunities to explore new research directions depending on the candidate’s interests.

Deliverables:

1. Development of drone- and mobile-robot-based simulation environments and tasks for studying teacher–student imitation.
2. Implementation and systematic evaluation of existing imitation learning algorithms in the developed environments.
3. Optional: Evaluation of our method in cross-sensor transfer settings, where the teacher and student use different sensors, such as a LiDAR-equipped teacher and an RGB-camera-based student.
4. Optional: Exploration of new research directions, such as diffusion-based imitation learning, with flexibility to investigate other promising ideas based on the candidate’s interests.

Required knowledge and qualifications

• Basic knowledge of machine learning and/or robotics.
• Programming experience in Python.
• Interest in working with simulation environments and learning-based robotics.

Experience with PyTorch, RL/IL, and robotic simulation environments is advantageous.

The project provides an opportunity to gain practical experience in robotics simulation environments, imitation and reinforcement learning algorithms, and development of learning-based robotics systems, while contributing to ongoing research in robot learning.