Accelerated Reinforcement Learning for Soft Robotics: Hours Instead of Years
Accelerated Reinforcement Learning for Soft Robotics: Hours Instead of Years
This research demonstrates that using adiabatic spectral submanifolds (aSSMs) to reduce the dynamics of soft robots allows reinforcement learning policies to be trained in hours instead of years. On a pneumatically actuated soft arm, a trajectory-tracking policy was trained in 3 hours on a six-dimensional aSSM-reduced model and deployed without fine-tuning. This approach achieved 40% lower tracking error compared to policies trained on larger recurrent neural networks (RNN) or transformers, which required significantly more time and data to identify.
The authors present a method using adiabatic spectral submanifolds to reduce the complexity of soft robot dynamics, enabling reinforcement learning training in hours rather than years on a pneumatically actuated soft arm.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
training_hours
3 hours
View original evidence
SSM-reduced model, a simple polynomial ordinary differential equation learned purely from pre-collected data, can serve as an RL training environment. On a pneumatically actuated soft arm, we train a trajectory-tracking policy in 3 hours on a six-dimensional aSSM-reduced model and deploy it without fine-tuning, whereas reproducing this policy training extrapolates to 3 years of hardware interaction. Our policy tracks
Open source S3training_hours
100 hours
View original evidence
dynamic targets with 40% lower error than otherwise identical policies trained on a 4.8-million-parameter recurrent neural network (RNN) or a 1.7-million-parameter transformer, which take 13 and 100 hours to identify, against 5 minutes for our mathematically justified 6,924-parameter model.
Open source S4training_minutes
5 minutes
View original evidence
dynamic targets with 40% lower error than otherwise identical policies trained on a 4.8-million-parameter recurrent neural network (RNN) or a 1.7-million-parameter transformer, which take 13 and 100 hours to identify, against 5 minutes for our mathematically justified 6,924-parameter model.
Open source S4parameter_count
6,924
View original evidence
dynamic targets with 40% lower error than otherwise identical policies trained on a 4.8-million-parameter recurrent neural network (RNN) or a 1.7-million-parameter transformer, which take 13 and 100 hours to identify, against 5 minutes for our mathematically justified 6,924-parameter model.
Open source S4
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
SSM-reduced model, a simple polynomial ordinary differential equation learned purely from pre-collected data, can serve as an RL training environment. On a pneumatically actuated soft arm, we train a trajectory-tracking policy in 3 hours on a six-dimensional aSSM-reduced model and deploy it without fine-tuning, whereas reproducing this policy training extrapolates to 3 years of hardware interaction. Our policy tracks
Open source S3
dynamic targets with 40% lower error than otherwise identical policies trained on a 4.8-million-parameter recurrent neural network (RNN) or a 1.7-million-parameter transformer, which take 13 and 100 hours to identify, against 5 minutes for our mathematically justified 6,924-parameter model.
Open source S4
Source：arXiv Robotics — research abstracts · arxiv.org