Instead of robots, humans move: The expanding field of training data collection in China
Instead of robots, humans move: The expanding field of training data collection in China
This piece explores how businesses and researchers in China are developing methods to collect robot learning data using human movements instead of physical robots. It covers projects like Mifeng Technology's human demonstration data collection, research on converting hand movements to robot actions (e.g., HandUMI), and companies like Noematrix and DexForce that integrate virtual world models for training. The article emphasizes the importance of extracting meaningful spatial and motion information from recordings rather than just increasing data volume, while cautioning that results in simulations or limited tasks do not directly translate to real-world factory performance.
The article discusses emerging methods in China for collecting robot training data using human demonstrations and virtual environments, noting that while these approaches reduce reliance on physical robots during collection, they do not eliminate the need for actual machine verification or guarantee improved operational performance without rigorous evaluation.
Source: Robotics — Robot learning news discovery · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
human demonstration data hours
>1,000,000 hours
View original evidence
erged that are trying to expand this type of collection that does not use actual machines to more people and locations. Mifeng Technology states on its official website that it has produced over 1 million hours of human demonstration data in total. It lists actual machines, simulations, and human demonstrations as methods for supplying data, and does not claim that human recordings replace everything. [3] What this m
Open source S6robot task success count
7
Reported trials
View original evidence
, the task was completed 12 out of 20 times without human intervention. Another model, π0.5, also succeeded 7 out of 20 times after additional learning with the same records. [4] This is a result where movements learned from human demonstrations could be executed on an actual machine without adding samples recorded on the actual machine. However, the target is limited to one task and an experimental environment. Also
Open source S9human video input trials
40 trials
Reported trials
View original evidence
ion is represented rather than the amount of collection. In a task of watering flowers, the success rate of a configuration that inputs human video was 7.5%, whereas it became 85% when information representing the spatial relationship between the hand and the object was added. The evaluation is based on 40 trials for each configuration, according to the research team's report. [5] This result shows the importance of
Open source S12spatial relationship input trials
40 trials
Reported trials
View original evidence
ion is represented rather than the amount of collection. In a task of watering flowers, the success rate of a configuration that inputs human video was 7.5%, whereas it became 85% when information representing the spatial relationship between the hand and the object was added. The evaluation is based on 40 trials for each configuration, according to the research team's report. [5] This result shows the importance of
Open source S12
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
ed by ScienceNet). The information regarding the collection participants at the beginning is based on this report and is not the author's own field reporting. Read original source [2] Chi et al., "Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots" (RSS 2024), research project official website. Read original source [3] Mifeng Technology official website. The figure of over 1 milli
Open source S20
erged that are trying to expand this type of collection that does not use actual machines to more people and locations. Mifeng Technology states on its official website that it has produced over 1 million hours of human demonstration data in total. It lists actual machines, simulations, and human demonstrations as methods for supplying data, and does not claim that human recordings replace everything. [3] What this m
Open source S6
es a gripping tool worn on the human hand to record the position, orientation, opening width of the fingertips, and video. That information is converted into the joints and fingertip movements of the target robot. [4] The research team used 562 records demonstrated by humans as samples for this task and conducted an experiment to fold a towel in half using the dual-arm robot "PiPER." With a learning method called ACT
Open source S8
, the task was completed 12 out of 20 times without human intervention. Another model, π0.5, also succeeded 7 out of 20 times after additional learning with the same records. [4] This is a result where movements learned from human demonstrations could be executed on an actual machine without adding samples recorded on the actual machine. However, the target is limited to one task and an experimental environment. Also
Open source S9
ion is represented rather than the amount of collection. In a task of watering flowers, the success rate of a configuration that inputs human video was 7.5%, whereas it became 85% when information representing the spatial relationship between the hand and the object was added. The evaluation is based on 40 trials for each configuration, according to the research team's report. [5] This result shows the importance of
Open source S12
Source：Robotics — Robot learning news discovery · note.com