When Do Discrete Actions Make Good Continuous Controllers?

This research challenges the common consensus that continuous action parametrizations are superior to discrete ones in robotic control. The authors compare single-step regression and multi-step flow variants of both continuous and discrete policies in simulation and real-world settings. They find that discrete action prediction, even with naive quantization, can match or outperform continuous-action policies in task success rates. The study highlights that the main challenge for discrete policies is factorization error, which can be mitigated through interventions like iterative computation or distribution sharpening.