ToolDex: An Ego-Exo Human Demonstration Dataset for Creative Physical Tool Use
ToolDex: An Ego-Exo Human Demonstration Dataset for Creative Physical Tool Use
This paper introduces ToolDex, a large-scale dataset featuring synchronized egocentric and exocentric camera recordings of humans selecting, adapting, and repurposing objects as tools. The dataset includes high-fidelity multi-view observations and rich annotations for hands, tools, target objects, and deformable interactions. It also establishes a benchmark suite evaluating physical tool-use reasoning, sparse-view 3D and 4D reconstruction, and robot policy learning capabilities.
The authors present ToolDex, a new dataset capturing human demonstrations of creative physical tool use with synchronized egocentric and exocentric camera views to support reasoning about object affordances and robot policy learning.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Dataset details reported by the source
- Modalities
- egocentric-exocentric Open source S3hands Open source S4tools Open source S4target objects Open source S4
- Tasks
- physical tool-use reasoning Open source S4sparse-view 3D and 4D reconstruction Open source S4robot policy learning Open source S4
- Embodiments
- human demonstration Open source S1
- paper: not_reported
What remains unknown
Dataset licensing and commercial-use rights are not established here.
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
ol-use behavior with sufficient scale, fidelity, and diversity. Existing benchmarks are often simulation-based, focus on simplified interactions, or lack detailed observations and annotations. We introduce ToolDex, a large-scale egocentric-exocentric (ego-exo) human demonstration dataset for creative physical tool use. ToolDex captures scenarios in which humans select, adapt, and repurpose available objects as tools.
Open source S3
Using synchronized egocentric cameras and eight exocentric cameras, the dataset provides high-fidelity multi-view observations and rich annotations for hands, tools, target objects, and their deformable interactions. Beyond data collection, we establish a benchmark suite characterizing three core capabilities: physical tool-use reasoning, sparse-view 3D and 4D reconstruction, and robot policy learning. ToolDex provi
Open source S4
ToolDex: An Ego-Exo Human Demonstration Dataset for Creative Physical Tool Use
Open source S1
arXiv:2611.09036v1 Announce Type: new Abstract: Creative tool use is a fundamental human capability and an important step toward general-purpose embodied intelligence. It requires agents to reason about object affordances, physical constraints, and manipulation strategies, especially when standard tools are unavailable. However, progress in this area is limited by the lack of real-world datasets that capture human to
Open source S2
des a foundation for studying human tool-use behavior and developing embodied agents that can adaptively use tools in complex real-world environments. Dataset visualizations and benchmarking results are available at https://tooluse-dataset.github.io/.
Open source S5
Implications for data suppliers
RoboSignal interpretation and collection questions, not statements of buyer demand.
- Confirm the required data type and collection setting with the buyer; this source does not establish a complete collection specification.
- Check the exact dataset version, license, modalities and task coverage against the intended use.
- Validate demand and acceptance criteria with a buyer before scaling. Publication, popularity and a research result do not establish a purchase commitment.
Source：arXiv Robotics — research abstracts · arxiv.org