Auditing Grasp Feasibility Predictors with Planner-Verified Counterfactuals
Auditing Grasp Feasibility Predictors with Planner-Verified Counterfactuals
This research paper introduces a method to audit grasp-feasibility predictors by inserting solid obstacles into the approach corridor while holding the grasp pose fixed, revealing that many high-performing models fail to correctly use collision geometry. The authors demonstrate that across eight different predictor types and four planners, these models' accuracy drops to chance levels when subjected to counterfactual tests involving real-object obstacles. A clearance-aware read of corridor points achieves high AUROC but remains bound to trained obstacle placements, indicating that a placement-invariant representation is still an open challenge. The paper releases counterfactual labels, metrics, a datasheet, and a relabeling pipeline for further research.
The study identifies a gap between high benchmark accuracy and actual collision geometry usage in grasp-feasibility predictors, proposing a planner-verified counterfactual test that exposes reliance on spurious correlations or missing obstacle registration.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Dataset details reported by the source
- Modalities
- point clouds, voxels, depth, and object tokens Open source S3
- code: Available Artifact link Open source S5
- paper: not_reported
What remains unknown
Dataset licensing and commercial-use rights are not established here.
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
d its points to the model input, and let a planner re-derive the label, which flips from feasible to infeasible. Across eight predictors spanning point clouds, voxels, depth, and object tokens, CF-pair AUROC falls from high natural AUROC (0.920-0.996) to chance (0.470-0.504), and the flip holds across four planners, two arms, and a real-object obstacle. Predictors fail at different resolutions, from a bounding box to
Open source S3
presentation is still open. We release the counterfactual labels, metrics, datasheet, and relabeling pipeline at https://github.com/codewithsajid/graspfc-cf.
Open source S5
Implications for data suppliers
RoboSignal interpretation and collection questions, not statements of buyer demand.
- Confirm the required data type and collection setting with the buyer; this source does not establish a complete collection specification.
- Check the exact dataset version, license, modalities and task coverage against the intended use.
- Validate demand and acceptance criteria with a buyer before scaling. Publication, popularity and a research result do not establish a purchase commitment.
Source：arXiv Robotics — research abstracts · arxiv.org