What Happens to the Goal? Distinct Failures of Policy Alignment in Goal Misgeneralization
This research explores how agents fail to maintain goal alignment during distribution shift, identifying three distinct failure modes based on where the goal is lost within the agent's internal processing. The study shows that recovery methods vary depending on the depth of goal loss and demonstrates how steering and downstream wiring can improve goal reach in different scenarios.