Learning When to Think: Adaptive Internal Computation for Reinforcement Learning
Overview
This paper presents Adaptive Internal Computation (AIC), an iterative neural policy that learns when to stop refining its latent representation and act. By jointly training action selection and stopping with auxiliary supervision from expected-depth penalties and intermediate value predictions, AIC outperforms state-independent schedules on tasks where variable computation is beneficial. On twelve benchmark tasks, it executes an average of 3.3 refinement steps (with a maximum of eight), achieving a 31--44% reduction in per-decision latency relative to a fixed eight-step policy on four control and robotics tasks. The method exceeds the best fixed-depth policy in mean reward on seven tasks but trails on four. Ablation studies indicate that adaptive stopping contributes most of the reward gain on PickCube and AntMaze, while value supervision is more critical for RareEvent.
Generated from attributed reports · Updated 4 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
No current evidence-backed claims. Missing information remains not reported.
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalLearning When to Think: Adaptive Internal Computation for Reinforcement Learning
This paper presents Adaptive Internal Computation (AIC), an iterative neural policy that learns when to stop refining its latent representation and act. By jointly training action selection and stopping with auxiliary supervision from expected-depth penalties and intermediate value predictions, AIC outperforms state-independent schedules on tasks where variable computation is beneficial. On twelve benchmark tasks, it executes an average of 3.3 refinement steps (with a maximum of eight), achieving a 31--44% reduction in per-decision latency relative to a fixed eight-step policy on four control and robotics tasks. The method exceeds the best fixed-depth policy in mean reward on seven tasks but trails on four. Ablation studies indicate that adaptive stopping contributes most of the reward gain on PickCube and AntMaze, while value supervision is more critical for RareEvent.
Event coverage history
There is not enough continuous observation data to show a trend.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).