FEIJOA: Task-Driven Grounding and Execution for Mobile Manipulation
FEIJOA: Task-Driven Grounding and Execution for Mobile Manipulation
The paper introduces FEIJOA, a language-model agent that grounds instructions at test time using RGB-D scans and verifies skill execution. On 103 open-ended grounding queries, it resolves 94.2% exactly versus 68.0% for the best prebuilt representation. Over 50 real-world trials on Spot, it completes 35 tasks; 17 successes follow recovery from a failed tool call or verification check. Of 15 failed trials, 13 were caused by physical skills and only 2 by grounding or reasoning.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
real-world trials on Spot
>50
Reported trials
View original evidence
nded grounding queries, FEIJOA resolves 94.2% exactly, compared with 68.0% for the best prebuilt representation. Over 50 real-world trials on Spot, it completes 35 tasks; 17 of these successes follow recovery from a failed tool call or verification check. Of the 15 failed trials, 13 were caused by the robot's physical skills, and only 2 by grounding or the agent's reasoning. Project website: https://williamshen-nz.gi
Open source S4tasks completed
35
Reported trials
View original evidence
nded grounding queries, FEIJOA resolves 94.2% exactly, compared with 68.0% for the best prebuilt representation. Over 50 real-world trials on Spot, it completes 35 tasks; 17 of these successes follow recovery from a failed tool call or verification check. Of the 15 failed trials, 13 were caused by the robot's physical skills, and only 2 by grounding or the agent's reasoning. Project website: https://williamshen-nz.gi
Open source S4successes following recovery from failure
17
Reported trials
View original evidence
nded grounding queries, FEIJOA resolves 94.2% exactly, compared with 68.0% for the best prebuilt representation. Over 50 real-world trials on Spot, it completes 35 tasks; 17 of these successes follow recovery from a failed tool call or verification check. Of the 15 failed trials, 13 were caused by the robot's physical skills, and only 2 by grounding or the agent's reasoning. Project website: https://williamshen-nz.gi
Open source S4failed trials caused by robot physical skills
13
Reported trials
View original evidence
nded grounding queries, FEIJOA resolves 94.2% exactly, compared with 68.0% for the best prebuilt representation. Over 50 real-world trials on Spot, it completes 35 tasks; 17 of these successes follow recovery from a failed tool call or verification check. Of the 15 failed trials, 13 were caused by the robot's physical skills, and only 2 by grounding or the agent's reasoning. Project website: https://williamshen-nz.gi
Open source S4failed trials caused by grounding or agent reasoning
2
Reported trials
View original evidence
nded grounding queries, FEIJOA resolves 94.2% exactly, compared with 68.0% for the best prebuilt representation. Over 50 real-world trials on Spot, it completes 35 tasks; 17 of these successes follow recovery from a failed tool call or verification check. Of the 15 failed trials, 13 were caused by the robot's physical skills, and only 2 by grounding or the agent's reasoning. Project website: https://williamshen-nz.gi
Open source S4
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
nded grounding queries, FEIJOA resolves 94.2% exactly, compared with 68.0% for the best prebuilt representation. Over 50 real-world trials on Spot, it completes 35 tasks; 17 of these successes follow recovery from a failed tool call or verification check. Of the 15 failed trials, 13 were caused by the robot's physical skills, and only 2 by grounding or the agent's reasoning. Project website: https://williamshen-nz.gi
Open source S4
thub.io/feijoa/.
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org