MIT Engineers Aim to Advance Household Robots With AI

Thanks to advancements in artificial intelligence, household robots are getting smarter and could soon evolve far beyond their current roles as mostly self-guided vacuum cleaners.

MIT engineers are working on a method that enhances the intelligence and adaptability of household robots, integrating them with large language models (LLMs) to navigate and respond to unexpected challenges intelligently. Industry experts are heralding this advancement as a leap forward, anticipating a near future where robots can autonomously adapt to new situations, boosting efficiency in both homes and businesses.

Interest in robots is growing among both consumers and businesses. Reports suggest that Apple is contemplating a venture into the personal robotics market. Following the conclusion of its electric vehicle initiative, the company is investigating potential projects in the robotics arena as it searches for new avenues of expansion. Among the projects under consideration is a mobile robot for household use, and another is a robotic tabletop device equipped with a movable display. And the global industrial robotics market is expected to grow to reach $60 billion by 2030. 

LLM Bots Get Smart

To take over more household and industrial tasks, robots have to be taught to get better at their work. Just like children, robots need to be taught how to complete tasks. 

The MIT research team may have uncovered a breakthrough by making the process of robot education simpler. The scientists explained in their paper published on the open-access platform Open Review that jobs can be broken down into smaller steps that need to be done one after the other for the task to be completed successfully.

For example, a robot must execute a series of actions like reaching, scooping and pouring when moving marbles from one bowl to another. Traditionally, if a robot deviated from its task due to an error, it had to restart from the beginning, requiring laborious programming for each potential mistake. However, the team used LLMs to automate the identification and sequencing of these subtasks, simplifying the process.

LLMs, which understand and generate natural language, were used to logically outline the steps involved in a task, such as “reach,” “scoop,” and “pour.” This innovation allowed the researchers to connect a robot’s physical actions with the corresponding stage in a task, enabling the robot to adjust and recover autonomously through an algorithm that maps its physical state or an image to a natural language label, a process known as “grounding.”

In practice, the team applied this approach to a marble-scooping robot. Despite interruptions like being nudged or dropping marbles, the robot, guided by the grounding classifiers learned from the LLM, could self-correct and proceed with the task without restarting, demonstrating the potential for robots to adapt and recover independently in real time.

Robots that use AI to learn how to move and interact with things are usually taught based on past data or by watching others do a task. They can also learn by trying to get the highest score or reward, Figueroa explained. When using robots in places where humans are the main focus, the skills these robots have learned might not work well.

“This can be caused by changes in the environment that were not considered during training or due to perturbations or changes imposed by the human or simply because the robot was not trained sufficiently or properly,” she added.

“In these cases, the robots have no idea how to recover or understand that they are failing to do a task as they don’t really understand what they are doing and are simply performing paths where they have been trained. In other words, classically trained AI robots have no common sense like humans do. The work proposed by the MIT engineers is solving this problem by exploiting LLMs to provide a common sense of what actions should be performed in the case that the robot is failing to perform a task.”

Bots Are Coming for Commerce

The MIT invention could have practical implications for robots that work in commerce. For example, in a warehouse environment demand can be highly unpredictable, Andy Williams, EVP North America at Exotec, a warehouse robotics company, told PYMNTS. Having robotics that can anticipate unforeseen events while moving throughout the warehouse (like a box falling from a rack in what would normally be their pathway) can severely decrease downtime and the need for human intervention or maintenance.

“As robots increasingly work alongside people in the warehouse, understanding their role in a task and understanding their place as a part of a whole, becomes crucial,” he added. “It enables them to operate effectively and safely in various environments to avoid inefficiencies and safety risks for warehouse workers.”

Warehouse and logistics robots are automating tasks for many of the world’s top retailers, improving throughput and increasing safety for human workers, Christina Gomez-Terry, director of operations at Plus One Robotics, told PYMNTS. AI enables these robots to pick and pack more efficiently and adjust gripper strength based on a package’s unique qualities, among many other applications.

“Other popular versions of AI robots can be found in many people’s homes, including Roomba and Braava,” she added. “AgTech and vertical farming organizations are testing AI-powered robots for weeding, harvesting and crop monitoring.”

In the future, new kinds of machine learning could boost robot productivity, but most will stay under the control of people. The robotics company XTEND is building an operating system that lets robots handle specific tasks autonomously — entering buildings, scanning floors, or even pursuing suspects. However, the “common sense” decisions — like judging situations or adapting to unforeseen circumstances — remain in the hands of human supervisors. 

“This human-machine teaming, or ‘Practical Human Supervised Autonomy’ allows our robots to work alongside supervisors, who can manage dozens of robots simultaneously,” Matteo Shapira, co-founder and CXO of XTEND, told PYMNTS. “It’s a powerful fusion that unlocks real-world solutions.”