What IFM released
The Institute of Foundation Models (IFM) at Abu Dhabi's MBZUAI published PAN 2 on 10 November 2026 and calls it "a foundation world model for physical intelligence". A world model is a system that predicts how an environment will change after an action. PAN 2 does that by generating video. You give it a scene, optionally as a first frame, plus a written action, and it renders what it expects to happen next. According to the Hugging Face model card, the weights and code are both under Apache 2.0. The GitHub repository contains the pretraining and supervised fine-tuning code as well as the inference scripts.
Plenty of video models release weights only. IFM has also published the training pipeline under a permissive licence, which makes PAN 2 one of the more complete open releases in its class. In MarkTechPost's comparison table, MiniMax-H3 and LTX-2.5 ship under community licences and their training code isn't disclosed. NVIDIA's Cosmos 3 Super uses the OpenMDW 1.1 licence.
How the model works
PAN 2 follows the Generative Latent Prediction design from IFM's Critiques of World Models paper. That design divides the work into two stages. In the first, a language model acts as the "conceptual reasoner". It reads the observation and the requested action, then writes the next world state as a structured, time-stamped caption covering events, actions tied to actors, and camera motion. The model card says any instruction-following LLM can fill that role, and MarkTechPost reports that IFM's own tests use Kimi K3 by default.
In the second stage, a 29-billion-parameter MMDiT (multimodal diffusion transformer, a diffusion model that processes text and image tokens together) predicts the video latents. IFM trained the MMDiT from scratch on hundreds of millions of annotated videos. A frozen Qwen3.5-9B model encodes the caption, and the frozen Wan 2.2 VAE decodes the latents into frames. Output is 704 by 1248 pixels at 24 frames per second. Clips run up to 20 seconds (481 frames), and the default is 241 frames, about 10 seconds.
The planner stage carries much of the load. In IFM's ablation, as reported by MarkTechPost, removing conceptual reasoning dropped full-plan completion from 34.0% to 0.0% and step completion from 75.1% to 8.7%. The vLLM recipe warns that a plain-text prompt produces unusable video, so every request has to be a structured caption serialised as one-line JSON. To handle the rewriting, IFM ships a Conceptual Reasoner skill for Claude Code and Codex.
Benchmarks, all self-reported
Every benchmark figure so far comes from IFM, and we haven't found an independent evaluation. On IFM's suite of 100 multi-step everyday tasks, PAN 2 completed 34.00% of full plans. MiniMax-H3 completed 32.50% and FLUX 3 completed 26.77%, according to MarkTechPost's summary of the model card. MiniMax-H3 led in all six single-action capability categories and on human preference, with an Elo of 1620 against PAN 2's 1595. On the Physics-IQ Verified leaderboard setting, PAN 2 scored 49.38. That's the highest score among open models and below the closed FLUX 3 at 51.11.
Hardware and cost
PAN 2 won't run on a gaming PC or a laptop. The model card gives no VRAM figure, but it notes that each inference worker holds the full 29B model in BF16, so one worker runs per GPU. The vLLM recipe is more specific. Generating the longest clip on a single 141 GB H200 peaked at 107.4 GiB per GPU. A 96 GB RTX PRO 6000 can only run the model with offloading or tensor parallelism, and those setups peaked at 84.4 GiB on one GPU or 84.5 GiB across two. Offloaded profiles also need a large amount of pinned system RAM. Neither IFM nor the vLLM recipe publishes generation times.
The software stack is current. The repository targets CUDA 13 builds of PyTorch, FlashAttention 4 and quack-kernels, and it links to an H200 setup guide. Serving through vLLM requires vLLM 0.33.0 or later with the vllm-omni package. The model also runs through Hugging Face Diffusers and FastVideo. The model card's short Diffusers snippet uses an image-style output call, so we'd follow the Diffusers pipeline docs rather than copy that snippet.
For New Zealand teams, "local" here means a self-managed GPU server or rented H200 or RTX PRO 6000 capacity, and that capacity is often hosted offshore. The Apache 2.0 licence still matters. You can fine-tune the model, deploy it commercially and keep your prompts and footage on infrastructure you control, without a usage clause or territory restriction. Several recent open video and image models have shipped with licences that carry those restrictions.
Limits IFM documents
IFM's model card is frank about failure modes. About two in three simulated plans contain at least one step that isn't carried out. Output is sensitive to prompt wording. Appearance drifts across clips in long interactions because each clip starts from a single frame plus text. Text describes coarse actions well and fine dexterous motion poorly. Intermediate states can show physically impossible shortcuts. PAN 2 also generates no audio, where MiniMax-H3, Cosmos 3 Super and LTX-2.5 all do.
IFM recommends running PAN 2 behind its pan2-guardrail package, which screens requests with Qwen3Guard-Gen-4B and checks output frames with a classifier. In IFM's tests, the guardrail cut the share of harmful text-to-video requests that produced unsafe video from 33.9% to 1.5%. It also blocked 11.5% of benign text-to-video requests and 3.3% of benign image-to-video requests, according to the model card. A false-positive rate of 11.5% is high enough that teams should test the guardrail on their own prompts before putting it in front of users.
What builders could do with PAN 2
PAN 2 is designed for what IFM calls simulative reasoning. An agent proposes several candidate plans, PAN 2 renders each one as video, and the agent picks the plan whose simulated outcome looks best. Robotics groups could use the model to preview manipulation or navigation sequences before running them on hardware. Teams building agents for warehouses, farms or construction sites could use it to produce synthetic footage of rare events for training perception models. Because the planner can be swapped, a team can pair the 29B renderer with whatever instruction-following LLM it already runs, including a self-hosted one.
Researchers get more out of PAN 2 than product teams do. With the pretraining and fine-tuning code in the repository, a lab can fine-tune on footage from its own domain, such as orchard rows, dairy sheds or port cranes, and avoid generic internet video. IFM notes that reward learning and model soup (averaging the weights of several fine-tuned checkpoints) aren't included, so the release doesn't reproduce PAN 2's full training pipeline. If your group fine-tunes PAN 2 or wires it into a robot planning loop, we'd like to see the results on the ASAI projects page.
Sources
- PAN 2: Technical Blog, MBZUAI IFM
- IFM/PAN2 model card, Hugging Face
- ifm-ai/PAN2 repository, GitHub
- PAN 2 serving recipe, vLLM Recipes
- MBZUAI's IFM Releases PAN 2, an Open 29B World Model for Physical AI, MarkTechPost
- Critiques of World Models, arXiv
- PAN 2 Conceptual Reasoner skill, GitHub
- pan2-guardrail, GitHub
- PAN 2 pipeline, Hugging Face Diffusers docs
- PAN 2 cookbook, FastVideo
Build with people working on the same things
Join the Auckland Society for AI: NZ$599 a year with a ForgeBox and the members' projects area. Our Auckland meetups are open to everyone.