ROCKET-2: Steering Visuomotor Policy via Cross-View Goal Alignment

Shaofei Cai | Zhancun Mu | Anji Liu | Yitao Liang

All authors are affiliated with Team CraftJarvis.

ROCKET-2 successfully inflicted damage on the Ender Dragon for the first time and spontaneously developed the ability to build bridges. (end-to-end training, directly control mouse & keyboard)

ROCKET-2 was pre-trained only on Minecraft, yet generalizes zero-shot to other 3D games.

Latest updates

• 03/21/2025 -- We have released the codebase for ROCKET-2!

Docker

docker build -t rocket2 .
docker run -it -p 7860:7860 --platform=linux/amd64 --gpus all rocket2:latest

Gradio

Citing ROCKET-2

If you use ROCKET-2 in your research, please use the following BibTeX entry.

@article{cai2025rocket,
  title={ROCKET-2: Steering Visuomotor Policy via Cross-View Goal Alignment},
  author={Cai, Shaofei and Mu, Zhancun and Liu, Anji and Liang, Yitao},
  journal={arXiv preprint arXiv:2503.02505},
  year={2025}
}