Our sandbox installed Video Prediction Policy 2 in 63 seconds and completed the build step in 1 second. Run the released project if you want to study manipulation video prediction or adapt policies to LIBERO and RoboDojo. Walk away if your immediate requirement is reproducing its headline real-world robot policy: the authors have withheld the distilled planner and Stage-3 policy. The useful question is which experiment you can reproduce with the assets actually released.
What people use video-prediction-policy-2 for
The authors use VPP2 for instruction-conditioned video prediction and robot manipulation, while a community experiment exposes the difference between the paper's planner and released RoboDojo checkpoints.
| Who used it | What they did | What happened |
|---|---|---|
| The authors | Deployed on real ALOHA manipulation tasks without task-specific fine-tuning | Reported 58.5% mean success across 10 task categories on an embodiment seen during training. ALOHA evaluation |
| The authors | Judged whether predicted manipulation videos followed instructions | Reported 0.80 success for human-hand prompts and 0.90 for robot prompts using a GPT-based evaluator. Video evaluation |
| The authors | Post-trained policies for simulation benchmarks | Reported 45.0% on LIBERO-Pro, 63.9% on LIBERO-OOD and 32.26% on RoboDojo. Updated paper |
| The issue #9 reporter | Added a reconstructed VLM subtask controller to the released RoboDojo policy | Original instructions scored 49 versus 40.5 with subtasks across 10 tasks, with one episode per method per task. Community comparison |
The reporter used one episode per method per task, rather than repeated trials. Evaluation design
For a robotics researcher, the video model is an interesting starting point because VPP2 trains predictions around a complete manipulation event before adapting them to shorter future chunks. The instruction describes a specific intended action, rather than merely asking for plausible motion. The paper then conditions an action expert on video features. That gives you separate questions to investigate: whether the predicted event follows the instruction, and whether the resulting actions complete the task. The authors' architecture
The issue #9 experiment makes that distinction concrete. In hang_mugs, the reporter's subtask caption stayed active for all 19 action chunks, yet the benchmark recorded full-task success after 445 actions. Their progress checker never advanced the caption. A completed task therefore wasn't evidence that their reconstructed planner had successfully tracked subtask transitions. The authors replied that the released RoboDojo checkpoints were trained on full instructions and contain no subtask-planning components. Start your comparison with the original instructions. Reporter and author response
What happened when we ran it
Our run at commit a103a1e on November 14, 2026 installed dependencies successfully in 63 seconds, with 106 dependencies recorded and 6,203 MB on disk.
| Check in our sandbox | Recorded result |
|---|---|
| Install | Succeeded in 63 seconds |
| Build | Succeeded in 1 second |
| Tests | Skipped: no tests script or target |
| Dependency audit | pip-audit reported 13 known vulnerabilities |
| Repository scan | 226 files, approximately 24,815 source lines, 8 MB checked out |
| Automation and packaging signals | 0 CI workflow files, no Dockerfile, no tests directory |
The recorded configuration was 3 CPUs and 8 GB RAM, with image label ghcr.io/astral-sh/uv:python3.12-bookworm. The small checkout and much larger dependency footprint are a practical reason to give this project its own environment. A successful build step makes the source easier to evaluate, but the skipped test step supplies no regression evidence. Keep those two outcomes separate when deciding whether to modify the policy code.
The one-second build result is the recorded harness step, not the time needed to train a model or make a robot move. Likewise, 6,203 MB describes the recorded installed environment. The README separately lists downloadable video and action checkpoints plus shared encoders and normalization assets. Those are additional setup decisions for the workflow you choose. Released asset list
Where people get stuck: checkpoint names hide different training paths
RoboDojo's single-forward video inference comes from maximum-noise post-training, while the reported real-world planner uses a separate consistency-distillation route.
| Symptom or question | What the record says | Practical response |
|---|---|---|
| A registry lookup cannot find the project | A separate package-discovery attempt reports that vpp2 was not found in the package registry | Follow the README's checkout-based loading instructions. This diagnostic record is separate from the successful sandbox measurements above. |
| More video sampling steps produce worse frames | TangGuohh reported blurrier frames with 10 steps than with one. The authors confirmed maximum-noise-only RoboDojo post-training | Use the intended single-forward video path for that checkpoint. Issue #11 |
| Adding a planner fails to improve scores consistently | The issue #9 reporter's reconstructed controller scored below original instructions on the mean native score | Establish the released full-instruction baseline before adding caption switching. Issue #9 |
| A short future window ends before the captioned task finishes | Haodong-Yan says Stage-2 windows reuse the source event's detailed caption | Let the observation indicate progress. Completion within every window is not required. Issue #4 |
| A short prompt behaves differently across workflows | Robert-gyj says detailed-caption training calls for prompt enhancement, with RoboDojo an exception | Use original prompts for RoboDojo training and inference. Follow prompt enhancement for the general video-model route. Issue #2 |
The sampling report is especially useful because "try more steps" sounds like a reasonable quality adjustment. Here the author response supplies a specific reason to stop: intermediate noise levels were outside RoboDojo post-training. Treat video sampling and action denoising as separate settings. A checkpoint designed for a single video forward pass does not become the paper's consistency-distilled planner when you change a sampler option. Sampling clarification
The missing-recipe discussion has a different outcome. In issue #3, a reader asked how the history-conditioned robodojo_his10k initializer had been produced before joint policy training. The authors explained its pre-distillation Stage-2 origin and promised a recipe. The README subsequently announced the history-conditioned training code and Video-10k guide on October 12, 2026. That closes the documentation complaint about the preceding stage; it doesn't turn the withheld real-world policy into a released checkpoint. Provenance discussion, README announcement
The 8 MB checkout is only the start of the environment
Our sandbox's 6,203 MB installed footprint is the useful disk-planning observation, and the README requires separately selected model assets for policy evaluation.
The README starts with a Python 3.10 Conda environment and points to CUDA/PyTorch setup before repository requirements. Its scripts load VPP2 from the checkout. The model download route matters too: Hugging Face is public, while ModelScope requires an account with access. You can choose the public route without treating a ModelScope account as a universal prerequisite. Installation and downloads
For policy evaluation, pair the matching video and action checkpoints, rather than collecting filenames that look approximately right. The README also lists shared Wan encoders, CLIP, UMT5 and tokenizer assets, plus normalization files. Save an asset manifest alongside the configuration you use. That recommendation follows directly from the number of separate components involved: changing a checkpoint while retaining another benchmark's normalization files would make the resulting experiment hard to interpret. Policy assets
Captioning is another resource decision when preparing your own data. Haodong-Yan says the project reuses an event caption across Stage-2 windows and warns that finer window-specific annotation can introduce inconsistent descriptions and higher annotation costs. Before paying to recaption overlapping windows, establish what the existing caption-and-observation pairing gets wrong on your data. An event caption supplies the goal, while the observed frame shows how far the task has progressed. The released design already expects a window to cover only part of an event. Caption reuse explanation
Security: 13 dependency advisories need a separate decision
Our pip-audit result reported 13 known vulnerabilities, so the successful installation is insufficient grounds to approve this environment for a shared research machine.
Keep the dependency environment isolated and review the advisory details before connecting it to valuable datasets or robot-control systems. The supplied summary gives a total, rather than a package-by-package remediation map. It would be irresponsible to name a vulnerable library, prescribe an upgrade or claim a remote exploit from that total alone. The scan also found no CI workflow files and no tests directory, so your own change checks will matter if you carry patches.
The README licenses the code under MIT and explicitly gives model weights, datasets and simulator assets their own terms. A team planning commercial use should inventory those assets alongside dependencies. The code license is useful permission for the code itself, but it doesn't settle permission for every downloaded component. License boundaries
Video Prediction Policy 2 vs π0.5, Fast-WAM and Cosmos3
The authors report a higher ALOHA average for VPP2 than for π0.5 or Fast-WAM, while Cosmos3 is a video-prediction comparison with a different outcome measure.
| Alternative | Supplied comparison | How to use it in a decision |
|---|---|---|
| π0.5 | Author-reported ALOHA mean: VPP2 58.5%, π0.5 40.0%. Fold favors π0.5, 80% versus 70% | Include π0.5 when folding resembles your target task. The average conceals a category where it wins. ALOHA results |
| Fast-WAM | Author-reported ALOHA mean: VPP2 58.5%, Fast-WAM 20.5% | Relevant to comparing action policies under the reported ALOHA setup. ALOHA results |
| Cosmos3-64B | The paper reports an 11.0 percentage-point VPP2 advantage in video instruction following | Compare instruction-faithful predictions, rather than reading this as a robot task-success margin. Paper abstract |
| Temporal Ratio | The supplied analysis reports LIBERO-OOD success of 63.9% for VPP2 versus 59.4% | A useful benchmark reference. The comparison does not isolate which mechanism produced the difference. LIBERO-OOD comparison |
| Original VPP | The predecessor also uses predictive video representations; no direct score comparison is supplied | Study it when the research question concerns what VPP2 changes in prediction and action learning. Predecessor discussion |
The fixed-duration VPP2 ablation provides a more focused research comparison than a larger model's parameter count. The supplied analysis reports instruction-following results of 0.48/0.42 for human/robot prompts with fixed-step prediction, against VPP2's 0.80/0.90. That supports examining event-level prediction for this evaluation. It still calls for a carefully matched experiment before attributing every gain to caption detail alone. Event-level comparison
What the other reviews get wrong
The v1 paper's RoboDojo figure is 29.47%, while the supplied updated paper reports 32.26% success and the README separately labels 39.26 as Score.
Use the updated 32.26% success rate when discussing the supplied updated results. Do not replace it with 39.26%: the README's native Score and SR are different metrics. This matters especially when comparing the issue #9 reporter's partial-progress scores with the paper's success rates. The former can credit incomplete task progress. Original v1 figure, updated paper, README metric labels
The project page's "11%" Cosmos3 advantage also needs a unit. The abstract specifies 11.0 percentage points in video instruction-following success. That is the wording to carry into a comparison, rather than turning the difference into a relative percentage or a robot-control result. Project-page wording, abstract's units
More consequential than either number is the release boundary. Robert-gyj states in issue #10 that video distillation and Stage-3 training serve the real-world zero-shot experiments, that there are no plans to release them, and that the current code uses no Stage-3 checkpoints. A summary of the paper's full architecture needs that sentence alongside any recommendation to download the repository. Release clarification
Questions people ask
The authors' issue replies answer checkpoint provenance and inference settings more directly than the headline benchmark descriptions.
Will the consistency-distilled Stage-2 checkpoint and Stage-3 action policy be released?
The authors say they currently have no plans to release the distilled video model or Stage-3 training components. Their response identifies these with the real-world zero-shot experiments. A closed issue here records an answer about release scope, rather than delivery of the requested artifacts. Issue #10
Which reported evaluations use Stage-3 action-pretrained weights?
The authors identify the real-world zero-shot experiments as using video distillation and Stage-3 training. They also state that the current public code uses no Stage-3 checkpoints, and explain why simulation tasks use a different action-learning path. Issue #10
Are the LIBERO and RoboDojo video models initialized before or after consistency distillation?
RoboDojo starts from the Stage-2 video checkpoint before distillation, according to issue #3. For LIBERO, issue #11 says it is also initialized from the Stage-2 checkpoint, without explicitly naming the distillation status. The replies support a precise RoboDojo answer and a narrower LIBERO provenance statement. Issue #3, issue #11
What training recipe produced RoboDojo Video-10k?
The authors describe standard video-model fine-tuning on RoboDojo from the pre-distillation Stage-2 checkpoint, adding history frames for memory-dependent tasks. The README's October 12 announcement points to the released history-conditioned Video-10k recipe. Provenance reply, recipe announcement
What consistency-distillation settings are needed to reproduce the unreleased model?
The authors disclose lambda of 0.5 and 6,000 training steps in issue #11. That reply gives specific settings, rather than a complete reproducibility package. The same discussion says the distilled checkpoint will be withheld. Distillation response
Why can 10-step sampling produce blurrier RoboDojo videos than one-step inference?
The authors confirm that RoboDojo post-training deliberately uses only the maximum noise level so inference takes one forward pass. TangGuohh's report of worse multi-step output fits that training distinction. Use the checkpoint's intended inference path before treating extra steps as an improvement. Sampling report and reply
How should RoboDojo history slots be filled at episode startup?
The authors say to repeat available frames until there are eight when fewer than eight are available. Use that explicit startup instruction for the history slots. It answers an episode-initialization question, rather than requiring the robot to wait while accumulating a full history. Startup reply
How does action training align a predicted video trajectory with ground-truth actions?
The authors say detailed captions constrain possible futures, and RoboDojo uses history to reduce ambiguity. Their response acknowledges the multimodal-trajectory problem raised by LilyLove0810. Shared observations and a task description can still permit more than one plausible future, which is why the question matters. Alignment discussion
Does the action expert use predicted future dynamics or mainly pretrained visual representations?
Issue #1 raises exactly this distinction, but the supplied author reply addresses ambiguity through captions and history rather than demonstrating which information the expert relies on. Treat the question as an experiment to run with feature interventions, not as a settled interpretation of the benchmark score. Representation question
Have ground-truth future features or shuffled and removed video KV caches been tested as ablations?
The supplied issue #1 response reports no results for those requested ablations. A reader proposes them to distinguish dependence on future dynamics from dependence on learned visual features. They would answer a narrower causal question than comparing VPP2's overall success with a different policy. Requested ablations
Which training stages use detailed instructions?
The authors say detailed captions are used in both event-level training and chunk-level video/action learning. RoboDojo post-training is their stated exception. Carry that exception into any training-data plan instead of applying the general caption recipe indiscriminately. Caption-stage reply
Is VLM prompt enhancement required at inference?
The authors say the general video model requires prompt enhancement to match detailed-caption training. RoboDojo training and inference instead use original prompts without enhancement. This is a workflow-specific requirement, so identify the checkpoint and benchmark before adding a VLM call. Inference-prompt reply
What we have not tested yet
Our measurements cover dependency installation, a build step and a dependency audit, not video generation, simulator task success or physical robot operation.
We did not measure GPU memory, inference latency, training throughput, deployment cost or performance under concurrent workloads. The recorded 8 GB RAM configuration is a sandbox allocation, not observed peak memory or a model-serving requirement. The container image label does not identify the actual runtime or operating-system versions. Those require the raw toolchain logs. No test count can be inferred from the skipped test target.
The ALOHA, LIBERO and RoboDojo performance numbers remain author-reported. The community controller comparison is a single-episode-per-task pilot, not a repeated-trial benchmark. Neither establishes reliability for a new robot. The authors' ALOHA evaluation used an embodiment seen during training, and their issue #6 response says they currently have no plan for RoboDojo's real-world track. That track must not be silently substituted with ALOHA results. Evaluation scope, real-world-track response
Cross-embodiment deployment also needs more work than a checkpoint download. In issue #5, the authors describe dataset-specific heuristic workspace and end-effector alignment. The supplied discussion leaves moving-base handling, camera dependence, calibration files and the complete output-coordinate convention unresolved. We have not assessed those deployment questions, alignment guarantees, the requested feature ablations, or whether window-specific captions improve results. Calibration discussion, feature questions, caption tradeoff
The English README and issue replies are readable sources for these workflows, but they supply no evidence of worldwide production adoption. Our ratings concern the documented evaluation path: on a 1-to-5 scale, 3 means usable with material constraints, and 2 means substantial unresolved barriers. Setup receives 3 for successful recorded installation and build steps, documentation 3 for workflow guides plus issue-dependent clarifications, and maturity 2 for withheld real-world components and absent automated-test evidence. Community support capacity is unassessed.
For a first trial, choose the released RoboDojo or LIBERO workflow, record the paired checkpoints and establish the original-instruction baseline. If your deliverable is the reported real-world zero-shot policy, stop at the release boundary before allocating robot time.


