A Benchmark and Layered Evaluation Protocol for Evidence-Grounded Shared Belief Updating in Connected Robotics
Overview
This research introduces the Connected Robotics Evidence Graph (CREG) Benchmark to evaluate evidence-grounded shared belief updating in connected robotics systems. The benchmark comprises 350 deterministically generated scenarios across seven regimes, validated by an independent log-odds checker and screened by a blinded expert audit. Evaluating four open-weight instruction models and six transparent fusion baselines reveals that entity grounding accounts for a substantial part of the weak-model deficit, deleting semantic relations costs 15.3 points of strict belief-state accuracy on relation-dependent scenarios, and belief accuracy is strongly non-monotonic along the evidence sequence. All scenarios, references, prompts, metrics, and analysis code are released for offline, hardware-free reproduction.
Generated from attributed reports · Updated 8 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Reported quantity · scenarios: 350 other · Basis not reported · Differing source assertions
“uld be evaluated as an evidence-to-belief trajectory, and introduce the Connected Robotics Evidence Graph (CREG) Benchmark to make that trajectory measurable. CREG comprises 350 deterministically generated scenarios across seven regimes (justified revision, belief preservation, relation association, contradiction onset and resolution, stale/delayed evidence, and source reliability), each exposing a normative posterio”
Exact source · revision 1Source owner not reported
Reported quantity · accuracy points: 15.3 other · Basis not reported
“, deleting semantic relations costs 15.3 points of strict belief-state accuracy on relation-dependent scenarios (Holm-corrected p<0.001), but an information-matched structured list outperforms the full graph and shuffling the graph does not hurt: relation content matters where required, while graph serialization confers no consistent advantage. Third, belief accuracy is strongly non-monotonic along the evidence seque”
Exact source · revision 1Source owner not reported
Reported quantity · p-value: 0.001 other · Basis not reported
“, deleting semantic relations costs 15.3 points of strict belief-state accuracy on relation-dependent scenarios (Holm-corrected p<0.001), but an information-matched structured list outperforms the full graph and shuffling the graph does not hurt: relation content matters where required, while graph serialization confers no consistent advantage. Third, belief accuracy is strongly non-monotonic along the evidence seque”
Exact source · revision 1Source owner not reported
Reported quantity · scenarios: 7 other · Basis not reported · Differing source assertions
“uld be evaluated as an evidence-to-belief trajectory, and introduce the Connected Robotics Evidence Graph (CREG) Benchmark to make that trajectory measurable. CREG comprises 350 deterministically generated scenarios across seven regimes (justified revision, belief preservation, relation association, contradiction onset and resolution, stale/delayed evidence, and source reliability), each exposing a normative posterio”
Exact source · revision 1Source owner not reported
Reported quantity · regimes: 7 other · Basis not reported
“uld be evaluated as an evidence-to-belief trajectory, and introduce the Connected Robotics Evidence Graph (CREG) Benchmark to make that trajectory measurable. CREG comprises 350 deterministically generated scenarios across seven regimes (justified revision, belief preservation, relation association, contradiction onset and resolution, stale/delayed evidence, and source reliability), each exposing a normative posterio”
Exact source · revision 1Source owner not reported
Reported quantity · conditions: 9 other · Basis not reported
“r reference at every evidence prefix, validated by an independent log-odds checker and screened by a blinded expert audit. Nine representation conditions separate relation content from serialization, ordering, and metadata. Evaluating four open-weight instruction models and six transparent fusion baselines yields three findings. First, entity grounding accounts for a substantial part of the weak-model deficit. Second”
Exact source · revision 1Source owner not reported
Artifact availability · code: reported pending
“nce, with mid-trajectory failures that endpoint-only evaluation cannot detect. All scenarios, references, prompts, metrics, and analysis code are released for offline, hardware-free reproduction.”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalA Benchmark and Layered Evaluation Protocol for Evidence-Grounded Shared Belief Updating in Connected Robotics
This research introduces the Connected Robotics Evidence Graph (CREG) Benchmark to evaluate evidence-grounded shared belief updating in connected robotics systems. The benchmark comprises 350 deterministically generated scenarios across seven regimes, validated by an independent log-odds checker and screened by a blinded expert audit. Evaluating four open-weight instruction models and six transparent fusion baselines reveals that entity grounding accounts for a substantial part of the weak-model deficit, deleting semantic relations costs 15.3 points of strict belief-state accuracy on relation-dependent scenarios, and belief accuracy is strongly non-monotonic along the evidence sequence. All scenarios, references, prompts, metrics, and analysis code are released for offline, hardware-free reproduction.
Event coverage history
There is not enough continuous observation data to show a trend.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).