Teaching Robots To Move by Sketching Trajectories – News
Getting robots to perform even a simple task requires tons of behind-the-scenes work. Part of the challenge is planning and executing movements, everything from turning wheels to lifting a robotic arm. To make this happen, roboticists collaborate with programmers to develop a set of trajectories — or pathways — that are clear of obstacles and doable for the robot.
Researchers at Carnegie Mellon University’s Robotics Institute(opens in new window) (RI) are creating new ways to chart these trajectories.
William Zhi, a postdoctoral fellow in RI, worked with Ph.D. student Tianyi Zhang and RI Director Matthew Johnson-Roberson(opens in new window) to come up with a way to use sketches to show robots how to move. The team will present their work(opens in new window) at the upcoming IEEE International Conference on Robotics and Automation(opens in new window) in Yokohama, Japan.
“Traditional approaches to generate robot motion trajectories require specific programming of the robot,” said Zhi. “Humans can infer complex instructions through sketches. We seek to empower robots to do the same.”
Recently, there’s been preliminary work that studies using natural language to control robots, but researchers have primarily been testing different ways to teach robots to learn through demonstration. There are two main ways to go about this. One method relies on kinesthetic teaching, where a human records where the robot goes and then physically adjusts the robot to place its joints into the desired positions. The other approach is teleoperation, where the user manipulates the robot with a specialized remote controller or a joystick and then records the demonstration for the robot to copy.
Both methods have their drawbacks. Kinesthetic teaching, specifically, requires the user to be in the same space as the robot. More notably, it’s difficult to manually adjust some robots — and that difficulty only increases with mobile robots, like a quadruped robot with an arm attached to it. Teleoperation requires the user to have precise control and takes time to put a robot through its paces.
The RI team’s approach of sketching trajectories teaches the robot how to move without the drawbacks of kinesthetic teaching or teleoperation. In this novel method, the robot learns from movements drawn on an image of the environment where it will work.
To capture the image of the environment, the team placed cameras at two locations to take pictures from different perspectives. They then sketched the trajectory of the robot’s desired movement on the image and converted the 2D images into 3D models the robot could understand. They did this conversion through a technique called ray tracing(opens in new window), which uses light and shadows on objects to estimate their distance from the camera.
