AI-powered robots can now autonomously repair other bots

Back in January, researchers showed off the kitchen skills of an open-source Aloha housekeeping robot. Now the Unleashed project gets to grips with knotting shoelaces, hanging shirts and even repairing other robots.

The Aloha Unleashed project includes Stanford Ph.D student Tony Z. Zhao and assistant professor Chelsea Finn from the Mobile Aloha team, along with a bunch of roboticists at the Google DeepMind lab.

The research group makes use of facing pairs of Aloha 2 manipulator arms, which were developed to improve on the performance and durability of the original Aloha model while enabling “fleet-scale data collection on more complex tasks.”

Upgrades include replacing the scissor mechanism on the grippers of old with a low-friction rail to allow for grasping smaller objects, smaller RealSense cameras with a broader field of view, a passive gravity compensation setup made from off-the-shelf components to support leader arms for teleoperation, and a desktop frame fashioned from aluminum with a vision camera overhead. The team also developed a new simulation model with improved accuracy and better visuals to help with learning tasks.

The researchers have been “pushing the scale and dexterity of tasks on our Aloha 2 fleet” for the last year, and have now released a series of videos that show the dual-arm setup autonomously performing tasks, while adapting to slips and placement issues in real-time.

“We picked three of the most dexterous tasks we could think of: tying shoelaces; replacing a robot finger; hanging a shirt,” said Finn in a post on X. “And tried to see if we could train a policy to do each. All of them worked!”

Zhao reports that the robots at each station are fully autonomous, and that the footage was captured in one continuous shot. In the first demo, the manipulator team can be seen seeding Skynet nightmares by helping out a robot buddy – in the shape of DeepMind’s SARA-RT model – by slotting in a replacement gripper/finger, suggesting that a future where robots can repair each other (or themselves) is not too far off. Next, the bots set about coordinating their efforts to knot a shoelace.

Though we’ve seen humanoids folding laundry before, the Aloha manipulators were tasked with getting a shirt on a hanger and popping it on a nearby rack. The training policy did account for different-colored items but didn’t include examples of adult shirts, yet Zhao removed his sweater and tossed it in the work zone and found that the model was able to generalize and complete the task.

The Aloha Unleashed project is ongoing, but if you want to get in some manipulator action of your own, an Aloha research kit is available from Trossen Robotics for just shy of US$30k.

Source: Tony Zhao (on X)