Teaching Robots What Really Matters

Giving a robot an instruction is the easy part. Getting it to understand what you actually mean is a lot harder. Tell a household robot to bring you a cup of coffee, for example, and it might happily carry it right over your expensive laptop. You never told it not to, after all. Researchers have now developed a new system called FLEX that could make correcting these kinds of mistakes as simple as telling the robot what it did wrong.

FLEX, or Flexible Feature Editing Pipeline, was developed by a group of researchers at MIT and NYU. Rather than using language only to tell a robot how to perform a task, the system uses natural language corrections to help it figure out which parts of the environment actually matter.

That distinction is important because demonstrations can show a robot what behavior you prefer, but they don't necessarily explain why. If you demonstrate a path that keeps a cup of coffee away from a laptop, the robot might learn that the laptop should be avoided. What it may not understand is that you really want it to keep liquids away from electronics in general.

An overview of FLEX (📷: A. Lee et al.)

FLEX tackles the problem with three operations: ADD, GENERALIZE, and MASK. ADD introduces a feature that the robot wasn't previously considering. GENERALIZE expands a specific feature into something broader, and MASK removes features that are no longer relevant.

Suppose the robot is carrying coffee and you tell it, "Don't spill coffee on my laptop!" FLEX can ADD the laptop as something to avoid, then GENERALIZE that feature to include other electronics, such as phones and calculators. If the robot was previously paying attention to an unrelated bowl on the table, MASK can remove that from consideration. A language model interprets the correction and updates the robot's internal representation of the task.

The team tested FLEX in a 2D navigation environment and with simulated tabletop manipulation tasks involving a seven-degree-of-freedom Franka robot arm. The latter included carrying coffee around electronics, moving a lit candle past flammable objects, and transporting a bowling ball around fragile items.

The simulated environments (📷: A. Lee et al.)

Across seven 2D navigation scenarios, FLEX achieved the highest normalized reward in every case. It also performed well when corrections were given sequentially, generally learning the intended reward after one or two corrections and maintaining that performance as additional corrections accumulated. Competing methods tended to collect unnecessary features, fail to generalize earlier ones, or overfit to individual stages.

To see how the system handled less predictable human feedback, the team also recruited 12 participants to provide their own natural language corrections. Participants rated FLEX highest across all five categories measured, including task performance, understanding intent, ease of use, and ease of collaboration. The closed-loop study used simulated robot behavior for safety, but the researchers separately tested the collected corrections with demonstrations recorded using a physical Franka robot.

FLEX isn't ready to handle everything just yet. The researchers note that the current approach focuses on relatively static tasks involving rigid objects, and unusual personal preferences can still trip up the language model. But the basic idea could make teaching robots much more intuitive. Instead of trying to anticipate everything that might go wrong, you could simply point out a mistake and give the robot a chance to learn from it.

Nick Bild

R&D, creativity, and building the next big thing you never knew you wanted are my specialties.