Aquatic robot's self-learning optimization enhances underwater object manipulation skills

In recent years, roboticists have introduced robotic systems that can complete missions in various environments, ranging from the ground to underground, aboveground and underwater settings. While several of these robots can grasp and move objects on the ground, the handling of objects by robotic systems underwater has so far proved more challenging.

Researchers at Columbia University recently developed AquaBot, a new underwater robot that can autonomously complete basic object manipulation tasks underwater. The robot, introduced in a paper published on the arXiv preprint server, combines accessible hardware with a computational model trained on data extracted from human demonstrations.

"Underwater robotic manipulation faces significant challenges due to complex fluid dynamics and unstructured environments, causing most manipulation systems to rely heavily on human teleoperation," Ruoshi Liu, Huy Ha and their colleagues wrote in their paper. "We introduce AquaBot, a fully autonomous manipulation system that combines behavior cloning from human demonstrations with self-learning optimization to improve beyond human teleoperation performance."

Aquabot, the robot designed by Liu, Ha and their colleagues, is based on the QYSEA V-EVO underwater drone. The researchers added a QYSEA parallel jaw gripper and two cameras to this aquatic drone, as this would allow it to collect relevant images underwater and complete object manipulation tasks.

They also created code that would enable the robot to autonomously learn end-to-end visuomotor policies guiding its manipulation of objects underwater. The computational models that plan and control the robot's actions were trained in two different stages.

"In the first stage, we distill human adaptability into a closed-loop visuomotor policy," the researchers wrote. "To do so, we record human demonstrations that teleoperate the robot to perform various manipulation tasks and then use the data to train a visuomotor policy. By shortening the policy's action horizon, we effectively increase its reactiveness, which is critical in dealing with unexpected underwater dynamics."

As part of the second training stage, Liu, Ha and their colleagues tried to speed up the robot's learning of new behaviors. To do this, they used self-guided optimization, an approach that allows systems to autonomously adjust its learning process using feedback derived from its past predictions or decisions.

"In this step, we repeatedly execute the learned policy and use the execution time as a reward to accelerate the policy with a surrogate-based optimization algorithm," wrote the researchers. "This step allows the system to further optimize suboptimal parameters (e.g., execution speed) in the human demonstration data."

The researchers evaluated Aquabot in a series of real-world experiments, testing its ability to complete different object manipulation tasks. These tasks included grasping previously unseen rocks underwater, sorting trash into different containers and retrieving large deformable objects designed to emulate human bodies.

The robot performed well in all these tasks, completing them faster than a human operator would. Notably, its underlying self-guided optimization approach also allows the root to continuously improve its base policy as it gains more experience in real-world environments.

"With extensive real-world experiments, we demonstrate AquaBot's versatility across diverse manipulation tasks, including object grasping, trash sorting, and rescue retrieval," wrote Liu, Ha and their colleagues. "Our real-world experiments show that AquaBot's self-optimized policy outperforms a human operator by 41% in speed. AquaBot represents a promising step towards autonomous and self-improving underwater manipulation systems."

The hardware design and the software used by the research team are open source and can be accessed by other computer scientists worldwide. In the future, the robotic system they developed could be further improved, tested in other experiments and eventually deployed in natural environments to complete various missions. For instance, it could be used to assist human agents during search and rescue missions and to collect rubbish, minerals or other objects from the seabed.