Introduction

This project has been taken.

The spinal cord in many vertebrates contains a central pattern generator (CPG)[1] that can control the physical body to interact with the environment and produce diverse rhythmic motor patterns, such as walking and swimming. And amphibious robots are good biological counterparts of animals for understanding their locomotion. In this project, we would like to use the Deep reinforcement learning framework BRAX[2] in MuJoCo[3] for exploring and designing CPG controllers for a bio-inspired robot Polymander[4] and then verify the controller in real experiments.

Main Contents (PdM contents in bold)

• Migrate the existing CPG-controller into MuJoCo Jax framework to prepare robot simulation experiments
• Use Deep reinforcement learning to train a CPG network for Polymander to walk [, swim, and transition] in different terrains. If needed, access to EPFL scitas computing cluster would be provided.
• Deploy the network on real robots for validation [and reduce the reality gap].

The student is expected to be rigorous and patient, have good programming skills and prior knowledge in reinforcement learning. Having taken the course CS-433 Computational Motor Control is a plus, but not mandatory. Students interested in this project could send CV, transcripts, and materials that can demonstrate project experience, if possible, to supervisors listed above.

Reference

• Ijspeert, Auke Jan, et al. "From swimming to walking with a salamander robot driven by a spinal cord model." science 315.5817 (2007): 1416-1420.

• Freeman, C. Daniel, et al. "Brax--a differentiable physics engine for large scale rigid body simulation." arXiv preprint arXiv:2106.13281 (2021).
• Todorov, Emanuel, Tom Erez, and Yuval Tassa. "Mujoco: A physics engine for model-based control." 2012 IEEE/RSJ international conference on intelligent robots and systems. IEEE, 2012.
• Gevers, Louis, et al. "Investigating the effect of morphology on the terrestrial gaits of amphibious fish using a reconfigurable robot." Bioinspiration & Biomimetics (2025).

In nature, animals have many compliant structures that benefit their locomotion. For example, compliant foot/leg structures help adapt to uneven terrain or negotiate obstacles, flexible tails allow efficient undulatory swimming, and muscle-tendon structures help absorb shock and reduce energy loss. Similar compliant structures may benefit salamander-inspired robots as well.

In this study, the student will try simulating compliant structures (the feet of the robot) in Mujoco and optimizing the design. To bridge the sim-to-real gap, the student will first work with other lab members to perform experiments to measure the mechanical properties of a few simple compliant structures. Then, the student needs to simulate these experiments using the flexcomp plugin of Mujoco or theoretical solid mechanics models and tune the simulation models to match the dynamical response in simulation with the experiments. Afterward, the student needs to optimize the design parameters of the compliant structures in simulation to improve the locomotion performance of the robot while maintaining a small sim-to-real gap. Finally, prototypes of the optimal design will be tested on the physical robot to verify the results.

The student is thus required to be familiar with Python programming, physics engines (preferably Mujoco), and optimization/learning algorithms. The student should also have basic mechanical design abilities to design models and perform experiments. Students who have taken the Computational Motor Control course or have experience with data-driven design and solid mechanics would also be preferred.

The student who is interested in this project shall send the following materials to the assistants: (1) resume, (2) transcript showing relevant courses and grades, and (3) other materials that can demonstrate your skills and project experience (such as videos, slides, Git repositories, etc.).