DREAMSTEER: Latent World Models Can Steer VLA Policies During Deployment

Pretrained generalist policies, such as vision-language-action (VLA) models, promise impressive zero-shot generalization in robot manipulation. However, their real-world performance tapers quickly under distribution shift, leading to decreased robustness and inconsistent instruction following abilities.

To address these challenges, we propose DreamSteer, a deploy-time steering framework to enhance pretrained VLAs without the need for finetuning on demonstration data collected in the target distribution. The key insight in DreamSteer is to leverage a latent world model and a general-purpose value function to steer pretrained VLA policies. During deployment, DreamSteer generates diverse action candidates, sourced from the VLA policy and a set of predefined motion primitives, and imagines the outcome of each action sequence by rolling it out within the latent world model.

By evaluating these predicted trajectories with the value model, DreamSteer identifies and executes the highest-scoring action, resulting in better instruction following and filtering out task-irrelevant behaviors. Across four real-world manipulation benchmarks of unseen objects, DreamSteer improves task success rates by 42.5 percentage points, from 23.75% to 66.25%, and increases instruction following accuracy by 17.5 percentage points, from 38.75% to 56.25%, compared to the base VLA. These results suggest that latent world models can steer VLA policies during deployment and provide an effective pathway for improving the reliability of generalist robot policies when finetuning may not be desired or feasible.