Shades of Success: Evaluating Manipulation Quality from Object-Centric Constraints
Overview
This research presents SPECTRA, a framework for evaluating the quality of robotic manipulation tasks by analyzing object-centric constraints. Using a small set of oracle demonstrations in simulation, SPECTRA identifies admissible variations in relative poses and calculates a Task Quality Index (Q) to score unseen executions. The method achieves 94% accuracy in recovering constraints across seven simulated tasks and demonstrates strong correlation with human judgments. Additionally, the study shows that selecting high-scoring demonstrations for training a diffusion policy on hardware leads to higher success rates compared to random selection.
Generated from attributed reports · Updated 1 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Reported quantity · correlation: 0.81 other · Basis not reported · Differing source assertions
“To score an unseen execution, deviations beyond the admissible variation are summarized by a Task Quality Index Q in [0,1]. Across seven simulated tasks, SPECTRA recovers constraints with 94% accuracy using 30 demonstrations per task. On rollouts from three learned policies, its scores achieve pooled Spearman correlations with human judgments of p=0.81 over all rollouts and p=0.61 among successful rollouts, exceeding”
Exact source · revision 1Source owner not reported
Reported quantity · correlation: 0.61 other · Basis not reported · Differing source assertions
“To score an unseen execution, deviations beyond the admissible variation are summarized by a Task Quality Index Q in [0,1]. Across seven simulated tasks, SPECTRA recovers constraints with 94% accuracy using 30 demonstrations per task. On rollouts from three learned policies, its scores achieve pooled Spearman correlations with human judgments of p=0.81 over all rollouts and p=0.61 among successful rollouts, exceeding”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalShades of Success: Evaluating Manipulation Quality from Object-Centric Constraints
This research presents SPECTRA, a framework for evaluating the quality of robotic manipulation tasks by analyzing object-centric constraints. Using a small set of oracle demonstrations in simulation, SPECTRA identifies admissible variations in relative poses and calculates a Task Quality Index (Q) to score unseen executions. The method achieves 94% accuracy in recovering constraints across seven simulated tasks and demonstrates strong correlation with human judgments. Additionally, the study shows that selecting high-scoring demonstrations for training a diffusion policy on hardware leads to higher success rates compared to random selection.
Event coverage history
There is not enough continuous observation data to show a trend.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).