Localizing and Editing the Goal Representation of a Vision-Language-Action Model
Localizing and Editing the Goal Representation of a Vision-Language-Action Model
This research investigates how OpenVLA handles safety prohibitions by analyzing its internal goal representations. Using activation patching, linear probes, and runtime monitors on the LIBERO-Goal benchmark, the authors find that negation instructions have no detectable effect on task execution in the base model. The study identifies that steering vectors at layer 8 can redirect behavior, while effective goal editing requires intervention before layer 12 or patching every decoded action token starting from layer 16. External probes combining image and instruction features perform comparably to internal failure detectors for detecting non-compliance.
The study evaluates OpenVLA's adherence to safety prohibitions using activation patching and linear probes, finding that goal representations require intervention before layer 12 or at every action token to enforce constraints effectively.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
bes, activation steering and runtime monitors. On LIBERO-Goal, negation had no detectable effect on which task OpenVLA executed. The model executed the forbidden task in 79% of negated rollouts and completed 78% of plain tasks. A goal probe gives similar read-outs under "not X" and "X". Activation patching shows that the first action depends mostly on the instruction tokens before layer 12 and on the last prompt toke
Open source S3
n from layer 16 onward. A fixed steering vector at the instruction tokens at layer 8 redirected 12 of 18 rollouts. Fixed vectors and prefill patches at layers 12 to 20 redirected none of 158. From layer 16, only patches at every decoded action token redirected the robot (15 of 18 rollouts at layer 24). When the instruction changes during a rollout, the layer-32 goal read-out detects non-compliance better than failure
Open source S4
Source：arXiv Robotics — research abstracts · arxiv.org