Mechanistic Interpretability for End-to-End Self-Driving
Mechanistic Interpretability for End-to-End Self-Driving
This research applies mechanistic interpretability to end-to-end autonomous driving models by training Sparse Autoencoders (SAEs) on internal activations. The study reveals interpretable structures aligned with route intent, vehicle speed, and braking behavior, recovering route intent with 98.3% accuracy without supervision. Through counterfactual image edits, specific SAE features are shown to be visually grounded in scene changes like traffic light states and stop sign presence. Steering individual SAE latents produces predictable changes in planner outputs. A lightweight adapter trained on SAE latents reduces trajectory selection error by 3.44%, outperforming an equivalent adapter that omits SAE latents. Code and visualizations are available at https://mgagvani.github.io/mechanistic-interpretability-for-e2e-driving/.
The study applies mechanistic interpretability techniques to an end-to-end autonomous driving model using Sparse Autoencoders (SAEs) to decode internal activations into interpretable features like route intent and braking behavior, achieving 98.3% accuracy in recovering route intent without supervision.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
Reported accuracy
98.3%
View original evidence
retable structure aligned with route intent, vehicle speed, and braking behavior, recovering route intent with 98.3% accuracy without supervision. Through paired counterfactual image edits, we further show that specific SAE features are visually grounded and systematically respond to scene changes such as traffic light state and stop sign presence. Steering individual SAE latents produces predictable, monotonic chang
Open source S3
What remains unknown
Not established in the collected evidence: Environment, Control, Data origin.
Reported performance applies to the described task. It does not establish general autonomy or deployment readiness.
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
retable structure aligned with route intent, vehicle speed, and braking behavior, recovering route intent with 98.3% accuracy without supervision. Through paired counterfactual image edits, we further show that specific SAE features are visually grounded and systematically respond to scene changes such as traffic light state and stop sign presence. Steering individual SAE latents produces predictable, monotonic chang
Open source S3
es in planner outputs including lateral displacement, curvature, and braking. Finally, we demonstrate a practical application of these representations: we build a lightweight adapter trained on SAE latents that reduces trajectory selection error by 3.44%, outperforming an equivalent adapter that omits SAE latents, confirming that the sparse representation captures useful information for interpretability. Code and vis
Open source S4
ualizations are available at https://mgagvani.github.io/mechanistic-interpretability-for-e2e-driving/
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org