Closed-Loop Physics-Based Manipulation Evaluation at a Commodity Price: What 4.4 L4 GPU hours at $3.50 Buys You
Closed-Loop Physics-Based Manipulation Evaluation at a Commodity Price: What 4.4 L4 GPU hours at $3.50 Buys You
This paper introduces an affordable and reproducible evaluation harness for physics-based robotic manipulation, demonstrating that high-scale statistical evaluations can be performed on commodity hardware for approximately $3.50 USD and 38 minutes of wall-clock time per replication. The authors report a negative result regarding action chunking with sparse data, showing zero genuine successes across 780 rollouts for three independently trained chunked checkpoints compared to success rates between 2.6% and 11.8% for single-step checkpoints. The work isolates failures to the training formulation and multi-step prediction mechanisms rather than open-loop execution, aiming to democratize policy development and evaluation.
The authors present a reproducible evaluation harness for contact-rich robotic manipulation that runs on commodity serverless GPUs, reporting specific failure rates for action chunking strategies under partially controlled conditions.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
GPU hours
4.4
View original evidence
$3.50 USD and 38 minutes of wall-clock per replication (4.4 L4-GPU-hours across eight parallel shards). Using the harness, we report a partially controlled negative result on action chunking with sparse data: three independently trained chunked checkpoints produce zero genuine successes across 780 rollouts, while three single-step checkpoints range from 2.6% to 11.8% success rate. A per-step-replanning control partia
Open source S3wall-clock time
38 minutes
View original evidence
$3.50 USD and 38 minutes of wall-clock per replication (4.4 L4-GPU-hours across eight parallel shards). Using the harness, we report a partially controlled negative result on action chunking with sparse data: three independently trained chunked checkpoints produce zero genuine successes across 780 rollouts, while three single-step checkpoints range from 2.6% to 11.8% success rate. A per-step-replanning control partia
Open source S3rollouts
780
Reported trials
View original evidence
$3.50 USD and 38 minutes of wall-clock per replication (4.4 L4-GPU-hours across eight parallel shards). Using the harness, we report a partially controlled negative result on action chunking with sparse data: three independently trained chunked checkpoints produce zero genuine successes across 780 rollouts, while three single-step checkpoints range from 2.6% to 11.8% success rate. A per-step-replanning control partia
Open source S3Reported success rate
11.8%
View original evidence
$3.50 USD and 38 minutes of wall-clock per replication (4.4 L4-GPU-hours across eight parallel shards). Using the harness, we report a partially controlled negative result on action chunking with sparse data: three independently trained chunked checkpoints produce zero genuine successes across 780 rollouts, while three single-step checkpoints range from 2.6% to 11.8% success rate. A per-step-replanning control partia
Open source S3GPU hours
4.4
View original evidence
Closed-Loop Physics-Based Manipulation Evaluation at a Commodity Price: What 4.4 L4 GPU hours at $3.50 Buys You
Open source S1rollouts
200
Reported trials
View original evidence
arXiv:2611.07441v1 Announce Type: new Abstract: Running evaluations of contact-rich robotic manipulation at statistical scale has largely been possible only at well-resourced labs, highlighting the need for affordable and efficient options for broader audiences. We present a reproducible evaluation harness wrapper based on a previous study at 200-rollout scale on serverless commodity GPUs, at a cost of approximately
Open source S2
- paper: not_reported
What remains unknown
Not established in the collected evidence: Environment, Control, Data origin.
Reported performance applies to the described task. It does not establish general autonomy or deployment readiness.
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
$3.50 USD and 38 minutes of wall-clock per replication (4.4 L4-GPU-hours across eight parallel shards). Using the harness, we report a partially controlled negative result on action chunking with sparse data: three independently trained chunked checkpoints produce zero genuine successes across 780 rollouts, while three single-step checkpoints range from 2.6% to 11.8% success rate. A per-step-replanning control partia
Open source S3
arXiv:2611.07441v1 Announce Type: new Abstract: Running evaluations of contact-rich robotic manipulation at statistical scale has largely been possible only at well-resourced labs, highlighting the need for affordable and efficient options for broader audiences. We present a reproducible evaluation harness wrapper based on a previous study at 200-rollout scale on serverless commodity GPUs, at a cost of approximately
Open source S2
Closed-Loop Physics-Based Manipulation Evaluation at a Commodity Price: What 4.4 L4 GPU hours at $3.50 Buys You
Open source S1
Source：arXiv Robotics — research abstracts · arxiv.org