Beyond Infraction Scores: Failure-Boundary Profiles for Closed-Loop Evaluation of Driving and Robot Navigation Agents
Overview
This paper proposes failure-boundary profiles as a new evaluation method for driving and robot navigation agents. Unlike existing benchmarks that rely on infraction scores, this approach estimates commitment and competence curves across varying scenario factors to determine where an agent should proceed or stop. The authors evaluate seven driving agents in CARLA simulations and apply the same method to a warehouse robot in Isaac Sim, reporting decision accuracy metrics and highlighting differences in failure modes among agents with similar scores.
Generated from attributed reports · Updated 3 days ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Reported quantity · decision accuracy: 0.74 other · Basis not reported · Differing source assertions
“aches a decision accuracy of 0.74 and TFv6, the highest-scoring learned agent, 0.60. Agents with similar driving scores fail in different ways, and agents that reliably stop at a blocked corridor still rarely wait for a gap in oncoming traffic. We also apply the method, unchanged, to a warehouse robot in Isaac Sim. The navigation policy we test stops at blocked gaps in only one of three families, and two fine-tuned v”
Exact source · revision 1Source owner not reported
Reported quantity · decision accuracy: 0.6 other · Basis not reported · Differing source assertions
“aches a decision accuracy of 0.74 and TFv6, the highest-scoring learned agent, 0.60. Agents with similar driving scores fail in different ways, and agents that reliably stop at a blocked corridor still rarely wait for a gap in oncoming traffic. We also apply the method, unchanged, to a warehouse robot in Isaac Sim. The navigation policy we test stops at blocked gaps in only one of three families, and two fine-tuned v”
Exact source · revision 1Source owner not reported
Reported quantity · blocked widths average: 52.2 other · Basis not reported · Differing source assertions
“arXiv:2611.06453v1 Announce Type: new Abstract: Closed-loop driving benchmarks in CARLA rank agents by a driving score that multiplies route completion by an infraction penalty. This score does not distinguish an agent that stops in front of an obstacle from one that pushes past it. In a corridor closed by two parked cars, for example, one agent that drives through with contact averages 52.2 on the blocked widths, wh”
Exact source · revision 1Source owner not reported
Reported quantity · blocked widths average: 37.8 other · Basis not reported · Differing source assertions
“ile the agents that stop average between 37.8 and 40.6. We propose failure-boundary profiles, an evaluation in which each scenario family varies one measured factor, such as corridor width or the time gap in oncoming traffic, and each variant is certified from measurement as one where the agent should proceed or one where it should stop. Along this axis we estimate two curves per agent, commitment (the probability of”
Exact source · revision 1Source owner not reported
Reported quantity · blocked widths average: 40.6 other · Basis not reported · Differing source assertions
“ile the agents that stop average between 37.8 and 40.6. We propose failure-boundary profiles, an evaluation in which each scenario family varies one measured factor, such as corridor width or the time gap in oncoming traffic, and each variant is certified from measurement as one where the agent should proceed or one where it should stop. Along this axis we estimate two curves per agent, commitment (the probability of”
Exact source · revision 1Source owner not reported
Reported quantity · warehouse robot families tested: 3 other · Basis not reported
“aches a decision accuracy of 0.74 and TFv6, the highest-scoring learned agent, 0.60. Agents with similar driving scores fail in different ways, and agents that reliably stop at a blocked corridor still rarely wait for a gap in oncoming traffic. We also apply the method, unchanged, to a warehouse robot in Isaac Sim. The navigation policy we test stops at blocked gaps in only one of three families, and two fine-tuned v”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalBeyond Infraction Scores: Failure-Boundary Profiles for Closed-Loop Evaluation of Driving and Robot Navigation Agents
This paper proposes failure-boundary profiles as a new evaluation method for driving and robot navigation agents. Unlike existing benchmarks that rely on infraction scores, this approach estimates commitment and competence curves across varying scenario factors to determine where an agent should proceed or stop. The authors evaluate seven driving agents in CARLA simulations and apply the same method to a warehouse robot in Isaac Sim, reporting decision accuracy metrics and highlighting differences in failure modes among agents with similar scores.
Event coverage history
Current attention 3·Peak within the comparable range 3（2026-11-06 23:00 UTC）·Change within the comparable range over 24 hours –
The trend compares participants observed continuously; its scope may be narrower than current attention totals. Hover or click to explore hourly observations, or use the left and right arrow keys.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).