SAM-VTP: Video Memory and Measured Geometry for Continuous Traversability Segmentation
Overview
This paper presents SAM-VTP, a method that fine-tunes SAM2's video memory pathway to produce dense, prompt-free traversability segmentation. By measuring terrain geometry (slope, roughness, height) from depth and fusing it with image features, the approach achieves temporally consistent predictions. An offline continuous supervision pipeline generates per-pixel targets without robot deployment or pixel-level annotation. On TartanGround, SAM-VTP improves fuzzy Jaccard from 0.908 to 0.924 and binary IoU from 0.854 to 0.886 over a frame-wise baseline, reduces frame-to-frame flicker by 31%, and lowers planner dropouts by 3.3x on paved courses and 6.9x on rough ground in closed-loop field trials, running at 21.4 FPS on an Orin Nano.
Generated from attributed reports · Updated 7 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Control: reported autonomous
“arXiv:2611.12099v1 Announce Type: new Abstract: Traversability prediction is a core component of autonomous navigation, yet most vision-based methods operate frame-wise and produce temporally inconsistent predictions. We present SAM-VTP (Video Traversability Prediction), which fine-tunes SAM2's video memory pathway for dense, prompt-free traversability segmentation, conditioning each frame on a persistent memory of p”
Exact source · revision 1Source owner not reported
Reported quantity · frame rate: 21.4 other · Basis not reported
“baseline's dropouts leave the planner without a feasible path 3.3x more often on a paved course and 6.9x on rough ground, while SAM-VTP runs at 21.4 FPS on the Orin Nano the robot carries.”
Exact source · revision 1Source owner not reported
Reported quantity · dropout reduction factor (paved): 3.3 other · Basis not reported
“baseline's dropouts leave the planner without a feasible path 3.3x more often on a paved course and 6.9x on rough ground, while SAM-VTP runs at 21.4 FPS on the Orin Nano the robot carries.”
Exact source · revision 1Source owner not reported
Reported quantity · dropout reduction factor (rough ground): 6.9 other · Basis not reported
“baseline's dropouts leave the planner without a feasible path 3.3x more often on a paved course and 6.9x on rough ground, while SAM-VTP runs at 21.4 FPS on the Orin Nano the robot carries.”
Exact source · revision 1Source owner not reported
Reported quantity · fuzzy Jaccard (SAM-TP): 0.908 other · Basis not reported
“onditional feasibility that binary labels cannot represent. On TartanGround, a large-scale synthetic ground-robot dataset, SAM-VTP improves fuzzy Jaccard from 0.908 to 0.924 and binary IoU from 0.854 to 0.886 over SAM-TP, a frame-wise SAM2 baseline, against a seed-to-seed spread below 0.001, and cuts frame-to-frame flicker by 31%. In closed-loop trials at two field sites, driving the same planner and controller, the”
Exact source · revision 1Source owner not reported
Reported quantity · fuzzy Jaccard (SAM-VTP): 0.908 other · Basis not reported
“onditional feasibility that binary labels cannot represent. On TartanGround, a large-scale synthetic ground-robot dataset, SAM-VTP improves fuzzy Jaccard from 0.908 to 0.924 and binary IoU from 0.854 to 0.886 over SAM-TP, a frame-wise SAM2 baseline, against a seed-to-seed spread below 0.001, and cuts frame-to-frame flicker by 31%. In closed-loop trials at two field sites, driving the same planner and controller, the”
Exact source · revision 1Source owner not reported
Reported quantity · binary IoU (SAM-TP): 0.854 other · Basis not reported
“onditional feasibility that binary labels cannot represent. On TartanGround, a large-scale synthetic ground-robot dataset, SAM-VTP improves fuzzy Jaccard from 0.908 to 0.924 and binary IoU from 0.854 to 0.886 over SAM-TP, a frame-wise SAM2 baseline, against a seed-to-seed spread below 0.001, and cuts frame-to-frame flicker by 31%. In closed-loop trials at two field sites, driving the same planner and controller, the”
Exact source · revision 1Source owner not reported
Reported quantity · binary IoU (SAM-VTP): 0.854 other · Basis not reported
“onditional feasibility that binary labels cannot represent. On TartanGround, a large-scale synthetic ground-robot dataset, SAM-VTP improves fuzzy Jaccard from 0.908 to 0.924 and binary IoU from 0.854 to 0.886 over SAM-TP, a frame-wise SAM2 baseline, against a seed-to-seed spread below 0.001, and cuts frame-to-frame flicker by 31%. In closed-loop trials at two field sites, driving the same planner and controller, the”
Exact source · revision 1Source owner not reported
Dataset · modalities: depth · Differing source assertions
“rior observations. Rather than inferring terrain geometry from appearance, we measure it: slope, roughness and height are computed from depth and fused into the image features. We also introduce an offline continuous supervision pipeline that generates dense per-pixel traversability targets in [0,1] from depth-derived geometry and semantic priors, requiring no robot deployment and no per-pixel annotation, capturing c”
Exact source · revision 1Source owner not reported
Dataset · modalities: semantic priors · Differing source assertions
“rior observations. Rather than inferring terrain geometry from appearance, we measure it: slope, roughness and height are computed from depth and fused into the image features. We also introduce an offline continuous supervision pipeline that generates dense per-pixel traversability targets in [0,1] from depth-derived geometry and semantic priors, requiring no robot deployment and no per-pixel annotation, capturing c”
Exact source · revision 1Source owner not reported
Dataset · tasks: traversability segmentation
“arXiv:2611.12099v1 Announce Type: new Abstract: Traversability prediction is a core component of autonomous navigation, yet most vision-based methods operate frame-wise and produce temporally inconsistent predictions. We present SAM-VTP (Video Traversability Prediction), which fine-tunes SAM2's video memory pathway for dense, prompt-free traversability segmentation, conditioning each frame on a persistent memory of p”
Exact source · revision 1Source owner not reported
Dataset · embodiments: ground-robot
“onditional feasibility that binary labels cannot represent. On TartanGround, a large-scale synthetic ground-robot dataset, SAM-VTP improves fuzzy Jaccard from 0.908 to 0.924 and binary IoU from 0.854 to 0.886 over SAM-TP, a frame-wise SAM2 baseline, against a seed-to-seed spread below 0.001, and cuts frame-to-frame flicker by 31%. In closed-loop trials at two field sites, driving the same planner and controller, the”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalSAM-VTP: Video Memory and Measured Geometry for Continuous Traversability Segmentation
This paper presents SAM-VTP, a method that fine-tunes SAM2's video memory pathway to produce dense, prompt-free traversability segmentation. By measuring terrain geometry (slope, roughness, height) from depth and fusing it with image features, the approach achieves temporally consistent predictions. An offline continuous supervision pipeline generates per-pixel targets without robot deployment or pixel-level annotation. On TartanGround, SAM-VTP improves fuzzy Jaccard from 0.908 to 0.924 and binary IoU from 0.854 to 0.886 over a frame-wise baseline, reduces frame-to-frame flicker by 31%, and lowers planner dropouts by 3.3x on paved courses and 6.9x on rough ground in closed-loop field trials, running at 21.4 FPS on an Orin Nano.
Event coverage history
There is not enough continuous observation data to show a trend.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).