Telerobotics Test on Earth Led by Orbiting Astronaut

A two hour telerobotics test took place in a simulated planetary environment in the German Aerospace Center, DLR, at Oberpfaffenhofen near Munich. Working from Europe’s Columbus module, NASA astronaut Frank Rubio directed a trio of robotic assets to remove a seismometer from a lunar lander and place it on the ground to detect moonquakes.

“Our METERON project undertaken with DLR, proved the feasibility of this force-feedback control technique from local orbit, expanding to work with astronauts on the ISS and culminating in an outdoor campaign in the Moon-like environment of Mount Etna in Sicily.”

Thomas Krueger, ESA

Frank served as overseer for the robot team, working on a 'point and click' basis. He took direct control as needed using a force-feedback device. A DLR robotic lander arm was combined with DLR’s Rollin’ Justin humanoid robot and ESA’s four-wheeled, two-armed Interact rover.

The goal of the telerobotics test is to shed light on what astronauts need when operating robots with varying levels of independence, known as 'scalable autonomy'. This approach enables astronauts to have more control over the robots.

In the test scenario, the first in the ‘Surface Avatar’ series, a stuck pin had to be removed from the lander before the seismometer could be positioned by Justin. Meanwhile the Interact rover had been damaged by a quake during a surface sampling operation, so that the sample still stuck in its gripper needed to be retrieved by Justin and be placed on the lander.

In the event, Frank accomplished all the tasks in time, while also giving the team valuable feedback on features of the user interface and the robots. Those will be implemented and validated in the next session of Surface Avatar in January 2024.

Surface Avatar Test to Follow First Telerobotics Test

This first telerobotics test opens the way to a follow-up Surface Avatar test with a larger team of robots, planned to be undertaken by Danish ESA astronaut Andreas Mogensen during his Huginn mission later this year.

The distances involved in spaceflight make it impractical to perform direct remote control of robots on the Moon or other planetary surfaces from Earth. Instead, the idea is to have astronauts in orbit control these robots – to explore and take action on the Moon, then later other alien environments, without the cost and risk of landing.

“Our METERON project undertaken with DLR, proved the feasibility of this force-feedback control technique from local orbit, expanding to work with astronauts on the ISS and culminating in an outdoor campaign in the Moon-like environment of Mount Etna in Sicily,” explains Thomas Krueger, head of ESA’s Human Robot Interaction Lab, partnering with DLR's Institute of Robotics and Mechatronics.

“With Surface Avatar we combining direct teleoperation approach with supervised autonomy to control multiple robotic assets to accomplish complex tasks: picture a building site on Earth, where a crane might work alongside a bulldozer or excavator to get a big task done," Krueger continued. “What we want to find out with this testing is how the astronauts perform in practice. What control mode do they choose? Do they have situational awareness by seeing the world through the eyes of multiple robots? And what judgements they make in their head about how to act?”

The Surface Avatar telerobotics test scenario assumes the relatively structured environment of a planetary surface base, where the robots already know how to interact with some of the objects around them and the other robots in place. So, an augmented reality display produces a list of options for the controller to select. If the desired option is not available, then the astronaut switches to direct control, feeling precisely what the robot arm experiences through a force-feedback control device, allowing a high level of precision.

(Source: ESA news release. Images provided)