A Foundational Physical World Model for Efficient Real-World Interactions

📚 Contents

• 🗂️ Repository Layout
• ⚙️ Environment
• 📦 Model Assets & Checkpoints
• 🧩 Data Preparation
• 🚀 Training
• 🧪 RoboTwin Evaluation
• 🧪 LIBERO Evaluation
• 📌 Citation
• 🙏 Acknowledgements
• 📄 License

InternW0 is a physical world-action model with a slow Wan2.2 VideoDiT planner and a fast chunked ActionDiT executor. The video expert produces reusable future context; observation-conditioned routing adapts its layerwise K/V to each new chunk observation, so the model can predict slowly and act quickly.

This repository contains:

• multi-domain soft-prompt/domain-bank pretraining;
• DINOv3 observation encoding and IDM video/action training;
• RoboTwin and LIBERO posttraining readers;
• target-domain checkpoint conversion;
• RoboTwin and LIBERO closed-loop evaluation adapters;
• focused implementation-contract and distributed tests.

Pretraining and target-domain posttraining share one checkpoint contract: shared world/action modules are inherited, while action-side prompts and interfaces are initialized for the target embodiment. Force/tactile, real-robot, and laboratory assets described in the paper are not included here.

🗂️ Repository Layout

InternW0/
├── configs/
│   ├── data/                 Dataset and domain-registry configurations
│   ├── model/                W0 and future/IDM model configurations
│   ├── task/                 Pretraining and posttraining task graphs
│   ├── posttrain_*.yaml      RoboTwin/LIBERO posttraining entry points
│   ├── sim_*.yaml            Evaluation entry points
│   └── train.yaml            Base Hydra training graph
├── scripts/
│   ├── train.py              Hydra training entry point
│   ├── train_zero1.sh        ZeRO-1 launcher
│   ├── train_zero2.sh        ZeRO-2 launcher
│   ├── prepare_*             Statistics, data, and checkpoint preparation
│   ├── restore_pretraining_weights.py
│   │                         DCP-to-canonical exporter for the source format
│   └── run_libero_eval.sh    Two-process LIBERO evaluation launcher
├── experiments/
│   ├── robotwin/             RoboTwin policy adapter and evaluation entries
│   └── libero/               LIBERO policy server, client, and adapter
├── src/w0/
│   ├── datasets/             Native readers, normalization, and temporal masks
│   ├── models/wan22/         Wan, W0, ActionDiT, DINO, OVCR, and banks
│   ├── runtime.py            Model/data construction and training entry
│   └── trainer.py            Distributed trainer and checkpoint lifecycle
├── tests/                    Contract and distributed regression tests
├── docs/                     Reproduction and architecture documentation
└── assets/                   README figures

⚙️ Environment

Python 3.10 is required.

conda create -n w0 python=3.10 -y
conda activate w0
pip install -U pip setuptools wheel
pip install torch==2.7.1 torchvision==0.22.1 \
  --extra-index-url https://download.pytorch.org/whl/cu128
pip install -e .
conda install -c conda-forge ffmpeg -y

The Python package is imported as w0.

RoboTwin and LIBERO are external simulation dependencies and are not installed by this package.

📦 Model Assets & Checkpoints

InternW0 does not bundle model weights, datasets, or trained checkpoints. Use explicit local paths or the environment variables below.

Asset	Purpose	Environment variable
Wan2.2 TI2V-5B model directory	VideoDiT, VAE, tokenizer, and text encoder	W0_WAN_MODEL_DIR
DINOv3 ViT-B/16	Future/IDM observation encoder	W0_DINO_WEIGHTS
Canonical or DCP pretraining checkpoint	Source for target-domain conversion	command argument
RoboTwin dataset	Joint-space posttraining and evaluation	W0_ROBOTWIN_DATA
LIBERO dataset	Native 20 Hz EEF10 posttraining	W0_LIBERO_DATA
LIBERO checkout	Closed-loop LIBERO environment	LIBERO_PATH
RoboTwin checkout	Closed-loop RoboTwin environment	EVALUATION.robotwin_root

A convenient local layout is:

assets/
  wan22/
  dinov3-vitb16-pretrain-lvd1689m/
  robotwin/
  libero/
data/
  robotwin/
  libero/
external/
  RoboTwin/
  LIBERO/
outputs/

Hugging Face checkpoints

Checkpoint URLs are not included in this initial release.

Asset	Hugging Face URL
InternW0 foundation / pretraining checkpoint
RoboTwin target initialization
LIBERO target initialization
RoboTwin posttrained checkpoint
LIBERO posttrained checkpoint

When a checkpoint repository is published, download it outside the source tree and pass the resulting path through the corresponding environment variable or Hydra override. Do not commit model files to this repository.

🧩 Data Preparation

RoboTwin

First precompute the per-episode UMT5 text embeddings, then build train-only q01/q99 qpos statistics:

python scripts/prepare_robotwin_text_embeds.py \
  --dataset-root "$W0_ROBOTWIN_DATA"

python scripts/prepare_robotwin_stats.py \
  --root "$W0_ROBOTWIN_DATA" \
  --output assets/robotwin/normalization.json

Use the clean-only export for Clean2Random, or a combined clean+randomized export for the Full setting. Actions and proprio are scattered into 37D slots:

[0, 1, 2, 3, 4, 5, 16, 17, 18, 19, 20, 21, 22, 33]

LIBERO

First precompute UMT5 text embeddings, then build train-only statistics:

python scripts/prepare_libero_text_embeds.py \
  --dataset-root "$W0_LIBERO_DATA" \
  --cache-dir assets/libero/text_embeds_cache

python scripts/prepare_libero_posttrain.py \
  --dataset-root "$W0_LIBERO_DATA" \
  --output assets/libero/normalization.json

LIBERO remains at native 20 Hz. Its EEF10 representation occupies:

[7, 8, 9, 10, 11, 12, 13, 14, 15, 16]

Multi-domain pretraining registry

The reference registry is:

configs/data/domain_banks/interndata_a1_robomind2.json

It preserves domain identity, source ordering, native fields, and camera slots. For a local copy of the source datasets, regenerate it:

python scripts/build_action_domain_registry.py \
  --roots external/InternData-A1 external/RoboMIND2 \
  --output configs/data/domain_banks/interndata_a1_robomind2.json

Regenerate domain statistics after changing the registry because its fingerprint is part of the data contract:

python scripts/prepare_domain_statistics.py \
  --registry configs/data/domain_banks/interndata_a1_robomind2.json \
  --output assets/domain_data/interndata_robomind_stats.json

🚀 Training

InternW0 uses Hydra. init_checkpoint loads model weights only and starts a new optimizer; resume restores a complete compatible training state. The two modes are mutually exclusive.

Multi-domain soft-prompt pretraining

export W0_DOMAIN_STATS=assets/domain_data/interndata_robomind_stats.json
export W0_DOMAIN_TEXT_CACHE=assets/domain_data/text_embeds

bash scripts/train_zero2.sh 8 \
  task=interndata_robomind_idm_sp8 \
  output_dir=outputs/interndata_robomind_idm_sp8

The task uses K=8 prompts, 11 reference domains, DINOv3 B/16 observations, IDM training, H=64, C=16, and no action history.

Convert a pretraining checkpoint

From a canonical model-only export:

python scripts/prepare_posttrain_checkpoint.py \
  --source /path/to/canonical_export \
  --output assets/robotwin/posttrain_init.pt \
  --config-name posttrain_robotwin_clean_64gpu \
  --proprio-init new_domain \
  --seed 42

From a distributed checkpoint:

python scripts/prepare_posttrain_from_dcp.py \
  --checkpoint /path/to/dcp/30000 \
  --canonical-output /path/to/canonical_export \
  --output assets/robotwin/posttrain_init.pt \
  --config-name posttrain_robotwin_clean_64gpu \
  --proprio-init new_domain \
  --seed 42

For LIBERO conversion, use:

--config-name posttrain_libero_step30000_zero1_64gpu

The converter records every copied, split, initialized, and discarded tensor in a .transfer.json sidecar.

RoboTwin posttraining

Offset-aware recipe:

export W0_ROBOTWIN_DATA=data/robotwin
export W0_ROBOTWIN_STATS=assets/robotwin/normalization.json
export W0_ROBOTWIN_INIT=assets/robotwin/posttrain_init.pt
export W0_WAN_MODEL_DIR=assets/wan22
export W0_DINO_WEIGHTS=assets/dinov3-vitb16-pretrain-lvd1689m

bash scripts/train_zero1.sh 8 \
  --config-name posttrain_robotwin_clean_64gpu

Offset-zero ablation:

bash scripts/train_zero1.sh 8 \
  --config-name posttrain_robotwin_clean_no_offset_64gpu

LIBERO posttraining

export W0_LIBERO_DATA=data/libero
export W0_LIBERO_STATS=assets/libero/normalization.json
export W0_LIBERO_INIT=assets/libero/posttrain_init.pt
export W0_LIBERO_TEXT_CACHE=assets/libero/text_embeds_cache
export W0_WAN_MODEL_DIR=assets/wan22
export W0_DINO_WEIGHTS=assets/dinov3-vitb16-pretrain-lvd1689m

bash scripts/train_zero1.sh 8 \
  --config-name posttrain_libero_step30000_zero1_64gpu

🧪 RoboTwin Evaluation

Use an external RoboTwin checkout:

python experiments/robotwin/eval_robotwin_single.py \
  EVALUATION.robotwin_root=external/RoboTwin \
  EVALUATION.task_name=<task> \
  EVALUATION.ckpt=<checkpoint.pt> \
  EVALUATION.dataset_stats_path=<normalization.json>

The adapter applies the same camera canvas, action slots, normalization, and fixed control clock used by the posttraining reader. See Evaluation for the full protocol.

Paper result

Setting	Clean	Randomized	Average
Full	93.20	93.04	93.12
Clean2Random	83.20	68.00	75.60

Full jointly uses clean and randomized demonstrations. Clean2Random trains only on clean demonstrations and evaluates transfer to unseen randomized scenes. These paper-reported numbers are not rerun as part of code-release validation.

🧪 LIBERO Evaluation

The launcher starts a W0 policy server and a separate LIBERO environment client:

export CKPT=assets/libero/checkpoint.pt
export W0_WAN_MODEL_DIR=assets/wan22
export W0_LIBERO_STATS=assets/libero/normalization.json
export W0_LIBERO_TEXT_CACHE=assets/libero/text_embeds_cache
export LIBERO_PATH=external/LIBERO

SUITE=libero_spatial TASK_ID=0 NUM_TRIALS=50 \
bash scripts/run_libero_eval.sh

The default protocol executes a complete 16-action chunk unless EXECUTE_N_ACTIONS is explicitly overridden. Changing that value changes the closed-loop protocol and is not equivalent to training a chunk-8 model.

Paper result

InternW0 covers all 40 tasks in the four official LIBERO suites.

Spatial	Object	Goal	Long	Average
99.4	99.4	98.6	97.0	98.6

These paper-reported numbers are not rerun as part of code-release validation.

📌 Citation

If you find InternW0 useful, please cite:

@article{internw0,
  title = {InternW0: A Foundational Physical World Model for Efficient Real-World Interactions},
  author = {Jisong Cai and Yao Mu and Ganlin Yang and Zhe Cao and Zhangzheng Tu and Xing Gao and Kailin Li and Xinyu Zhan and Lixin Yang and Yangkun Zhu and Haoxiang Ma and Ming Zhou and Qiaojun Yu and Yufei Xue and Liqun He and Yifei Yao and Yifan Zhu and Long Ling and Bingqi Jiang and Haoyu Guo and Xueyue Zhu and Bowen Zhou and Bin Zhao and Tianfan Xue and Chunhua Shen and Weinan Zhang},
  journal = {arXiv preprint arXiv:2609.27656},
  year = {2026}
}

🙏 Acknowledgements

InternW0 builds on ideas and open-source infrastructure from Wan2.2, DINOv3, RoboTwin, LIBERO, LeRobot. We thank their authors and contributors for releasing useful models, data tools, and evaluation environments.

📄 License

This repository is released under the MIT License, except for third-party code and assets that retain their original licenses.