Effect of Temporal Resolution on Zero-Shot Semantic Understanding of Rapid Human Actions
Effect of Temporal Resolution on Zero-Shot Semantic Understanding of Rapid Human Actions
This research investigates how temporal resolution affects zero-shot semantic understanding of rapid human actions. Using kendo as a case study, the authors propose a training-free pipeline combining a pre-trained video-language model with large language model-based reasoning. Experiments show that Nearest Class Prototype (NCP) accuracy is highest at 120 Hz, and mean prototype margin improves significantly in the 70-120 Hz range compared to lower frequencies. Secondary experiments indicate that tracking overlays reduce full-observation performance but improve partial-observation performance.
The study provides preliminary evidence that high temporal resolution (120 Hz) improves zero-shot semantic understanding of rapid actions like kendo using a training-free pipeline, though tracking overlays degrade full-observation performance while aiding partial-observation tasks.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
video frames per second
10–120
View original evidence
and subtle motion patterns, we propose a training-free pipeline that combines a pre-trained video-language model for semantic representation with large language model-based reasoning for pairwise action comparison. Our primary experiment evaluates full-observation raw videos from 10 to 120 Hz in 10-Hz increments. Nearest class prototype (NCP) accuracy is highest at 120 Hz, and the mean over 70-120 Hz is higher than t
Open source S4similarity-score points
2.52
View original evidence
hat over 10-60 Hz. The corresponding mean prototype margin changes from -5.74 to +2.52 similarity-score points between these two halves and is also highest at 120 Hz. Secondary experiments show that tracking overlays reduce mean full-observation performance but improve mean partial-observation performance. These findings provide preliminary evidence that retaining high-temporal-resolution visual information can benef
Open source S5
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
and subtle motion patterns, we propose a training-free pipeline that combines a pre-trained video-language model for semantic representation with large language model-based reasoning for pairwise action comparison. Our primary experiment evaluates full-observation raw videos from 10 to 120 Hz in 10-Hz increments. Nearest class prototype (NCP) accuracy is highest at 120 Hz, and the mean over 70-120 Hz is higher than t
Open source S4
hat over 10-60 Hz. The corresponding mean prototype margin changes from -5.74 to +2.52 similarity-score points between these two halves and is also highest at 120 Hz. Secondary experiments show that tracking overlays reduce mean full-observation performance but improve mean partial-observation performance. These findings provide preliminary evidence that retaining high-temporal-resolution visual information can benef
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org