Project Database

In nature, animals have many compliant structures that benefit their locomotion. For example, compliant foot/leg structures help adapt to uneven terrain or negotiate obstacles, flexible tails allow efficient undulatory swimming, and muscle-tendon structures help absorb shock and reduce energy loss. Similar compliant structures may benefit salamander-inspired robots as well.

In this study, the student will try simulating compliant structures (the feet of the robot) in Mujoco and optimizing the design. To bridge the sim-to-real gap, the student will first work with other lab members to perform experiments to measure the mechanical properties of a few simple compliant structures. Then, the student needs to simulate these experiments using the flexcomp plugin of Mujoco or theoretical solid mechanics models, and tune the simulation models to match the dynamical response in simulation with the experiments. Afterward, the student needs to optimize the design parameters of the compliant structures in simulation to improve the locomotion performance of the robot while maintaining a small sim-to-real gap. Finally, prototypes of the optimal design will be tested on the physical robot to verify the results.

The student is thus required to be familiar with Python programming, physics engines (preferably Mujoco), and optimization/learning algorithms. The student should also have basic mechanical design abilities to design mechanical structures and perform experiments. Students who have taken the Computational Motor Control course or have experience with data-driven design and solid mechanics would also be preferred.

The student who is interested in this project shall send the following materials to the assistants: (1) resume, (2) transcript showing relevant courses and grades, and (3) other materials that can demonstrate your skills and project experience (such as videos, slides, Git repositories, etc.).

NOTE: this is a collaboration project, to be conducted at Cornell University, USA. To apply, e-mail Steve Heim stating why you are interested in this project (brief, 1-2 sentences each), and attach your CV and transcript.

Last edited: 11/03/2026

Egocentric wearable sensing is becoming a key enabler for embodied AI and robotics. Meta’s Project Aria research glasses provide rich, multimodal, first-person observations, including RGB video, scene cameras, IMUs, microphones, eye-tracking, and more, in a socially acceptable, all-day wearable form factor. They are specifically designed to advance egocentric AI, robotics, and contextual perception research.

In collaboration with Meta, we aim to tightly couple Aria research glasses with our existing manipulation platforms at EPFL. This project will:

1. Integrate the Aria Research Kit, including Aria Gen 2, with our ROS2-based robot platforms, including ViperX 300S and WidowX-250 arms on a mobile base. This includes calibration and time-synchronisation with RGB-D cameras and robot state.
2. Design and execute egocentric data collection in household-like environments, combining Aria, RealSense cameras, robot joint trajectories, and language annotations.
3. Develop egocentric data-driven policies for robotic manipulation through learning from demonstration. The robot can be a robot arm or a humanoid robot such as Reaman.
4. Explore one or more robotics applications powered by Aria signals, such as intention-aware teleoperation, egocentric demonstrations for policy learning, or vision-language-action fine-tuning for assistance tasks.
5. Perform systematic platform testing, validation, and documentation to deliver a reusable research pipeline for future projects. Excellent programming skills in Python are a plus.

We have well-documented tutorials on using the robots, teleoperation interfaces for data collection, using the HPC cluster, and a complete pipeline for training robot policies. The Aria Research Kit and tools, including recording, calibration, dataset tooling, and SDK, will be integrated into this ecosystem. This allows the student to focus on the research questions rather than low-level setup. We already have a good base and results from the ongoing project.

Project Aria glasses are multi-sensor research smart glasses. They include multiple cameras with wide field of view, IMUs, microphones, eye gaze, and a Machine Perception Service that provides SLAM poses, hand poses, and related perception outputs. They are explicitly positioned by Meta as a research kit for contextual AI and robotics, especially for using egocentric sensing to build embodied agents that understand and act in the world.

Compared to a normal RGB-D camera, Aria provides:

• Egocentric view: what the human or robot sees while acting.
• Calibrated head pose and trajectory through SLAM in MPS.
• Hand and gaze information, depending on which parts of the system are used.
• A portable, wearable, and socially acceptable form factor.

1. Ready-and-easy-to-use robot platforms: ViperX 300S and WidowX-250 arms, configured with 4 RealSense D405 cameras, various grippers, and a mobile robot platform.
2. Egocentric sensing hardware: Meta Project Aria research glasses, including Aria Gen 2, with access to the Aria Research Kit and tooling for data recording and processing.
3. Computing resources: two desktop PCs with NVIDIA GPUs 5090 and 4090.

Interested students can apply by emailing sichao.liu@epfl.ch or lixuan.tang@epfl.ch.

Please attach your transcript and a short description of your past/current experience on related topics, such as robotics, computer vision, machine learning, AR, or egocentric perception.

The position is open until we have final candidates. Otherwise, the position will be closed.

1. Kareer, Simar, et al. “EgoMimic: Scaling Imitation Learning via Egocentric Video.” 2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025.
2. Liu, Vincent, et al. “EgoZero: Robot Learning from Smart Glasses.” arXiv preprint arXiv:2505.20290, 2025.
3. Punamiya, Ryan, et al. “EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World.” arXiv preprint arXiv:2604.07607, 2026.
4. Saroha, Abhishek, et al. “EgoFlow: Gradient-Guided Flow Matching for Egocentric 6DoF Object Motion Generation.” arXiv preprint arXiv:2604.01421, 2026.
5. Zheng, Ruijie, et al. “EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data.” arXiv preprint arXiv:2602.16710, 2026.
6. Aria Project: https://www.projectaria.com/resources/
7. Aria GitHub: https://github.com/facebookresearch/projectaria_tools

Generalist Vision-Language-Action (VLA) policies can now perform a wide range of robot manipulation skills, but evaluating and improving them remains slow and expensive: rigorous evaluation requires hundreds of real-world rollouts, and systematic improvement demands additional corrective data with expert labels. Generative world models offer a scalable alternative by letting policies roll out inside imagination space, and recent work has shown that controllable, multi-view, action-conditioned world models can both rank policy performance without real-world execution and boost success rates by synthesising successful trajectories for supervised fine-tuning. In parallel, goal-conditioned formulations specify tasks through a target visual or language goal rather than a scalar reward, enabling reward-free online improvement through hindsight relabelling.

This project will tightly couple these three directions on our existing manipulation platforms at EPFL. Concretely, we will:

1. Integrate a controllable multi-view world model with our ROS2-based robot platforms (ViperX 300S and WidowX-250 arms, optionally a humanoid such as Reaman), including time-synchronisation with RGB-D cameras and robot state.
2. Adapt the world model into a goal-conditioned predictor that, given a current observation and a visual or language goal, imagines temporally consistent multi-view futures.
3. Implement an iterative co-improvement loop in which a small batch of real-world policy rollouts is used to ground the world model on contact-rich, deformable-object tasks, and the grounded model in turn generates large-scale synthetic data.
4. Replace the explicit success-classifier reward model with hindsight goal relabelling and LoRA-based fine-tuning of a π0.5-class VLA policy, enabling reward-free autonomous improvement.
5. Perform systematic evaluation, ablation, and documentation to deliver a reusable goal-conditioned RL pipeline for future projects.

We have well-documented tutorials on using the robots, teleoperation interfaces for data collection, the HPC cluster, and a complete pipeline for training robot policies. Open-source codebases for π0.5 (openpi), Ctrl-World, Act2Goal, and video diffusion will be integrated into this ecosystem, so the student can focus on the research questions rather than low-level setup.

We already have a strong base and results from an ongoing project: world model and RL for robot manipulation. Compared to standard language-conditioned VLAs trained only on demonstrations, this paradigm gives us:

• Imagination-based policy evaluation that closely tracks real-world performance rankings, with no physical rollouts.
• Targeted policy improvement on unseen objects, novel spatial layouts, and out-of-distribution instructions via synthetic successful trajectories.
• Physically grounded predictions of contact-rich and deformable-object interactions once the world model is fine-tuned on online rollout data.
• Reward-free online adaptation through hindsight relabelling.

1. Ready-and-easy-to-use robot platforms: ViperX 300S and WidowX-250 arms configured with 4 RealSense D405 cameras, various grippers, and a mobile robot platform, fully compatible with the LeRobot V3.0 data format.
2. Pretrained models: access to π0.5 / π0-FAST policies, the Ctrl-World world-model checkpoint, and Stable Video Diffusion backbones.
3. Computing resources: two desktop PCs with NVIDIA GPUs 5090 and 4090, plus HPC cluster access for world-model fine-tuning.

Interested students can apply by emailing sichao.liu@epfl.ch or lixuan.tang@epfl.ch. Please attach your transcript and a short description of your past/current experience on related topics such as robotics, computer vision, reinforcement learning, generative models, and VLA.

The position is open until we have final candidates. Otherwise, the position will be closed.

1. “Ctrl-World: A Controllable Generative World Model for Robot Manipulation.” ICLR 2026. arXiv:2510.10125.
2. “VLAW: Iterative Co-Improvement of Vision-Language-Action Policy and World Model.” arXiv preprint arXiv:2602.12063, 2026.
3. “Act2Goal: From World Model To General Goal-conditioned Policy.” arXiv preprint arXiv:2512.23541, 2025.
4. “LeRobot: An Open-Source Library for End-to-End Robot Learning.” arXiv preprint arXiv:2602.22818, 2026.
5. “π0.5: A Vision-Language-Action Model with Open-World Generalization.” arXiv preprint arXiv:2504.16054, 2025.
6. “Evaluating Robot Policies in a World Model.” arXiv preprint arXiv:2506.00613, 2025.