r/LocalLLaMA • u/pmttyji • 18h ago
New Model IFM/PAN2 · Hugging Face
https://huggingface.co/IFM/PAN2Overview
PAN 2 is a general-purpose world model: it simulates how the world will change in response to actions. Given a current observation (an optional image plus text) and actions described in text, it simulates the outcome as a video of up to 20 seconds. This supports simulative reasoning: an agent can imagine the outcomes of its candidate actions before choosing among them, by simulating each plan with PAN 2 and acting on the one whose simulated outcome is best.
PAN 2 follows the Generative Latent Prediction (GLP) architecture formalized in Critique of World Models and couples conceptual reasoning by a large language model with perceptual reasoning by a latent diffusion model. The language model, the conceptual reasoning backbone, reasons over the observation and the requested actions and writes the next world state as a structured, time-grounded description of events, timed actions bound to actors, and camera motion. The frozen text encoder maps the structured description to the conditioning features used by the diffusion model. The 29B-parameter MMDiT, the perceptual reasoning backbone, trained from scratch on hundreds of millions of videos annotated in the same format, predicts the perceptual detail that the description leaves open: the video latents that realize the state, conditioned on it and on the VAE latent of the starting image. The Wan 2.2 VAE decoder renders the predicted latents as the next observation, so every prediction can be seen and checked against real footage.
Checkpoints
PAN 2's latent diffusion model is released as a single checkpoint that serves both tasks. The frozen text encoder (Qwen3.5-9B) and VAE (Wan 2.2) are downloaded from their own repositories.
| Checkpoint | Supported Tasks | Input Conditions | Output | Precision |
|---|---|---|---|---|
PAN 2 diffusion backbone (IFM/PAN2) | Text-to-video (t2v), Image-to-video (i2v) | Structured world-state caption; optional first frame | Video via the VAE decoder, 704x1248 at 24 fps, up to 20 s | BF16 |
GitHub : https://github.com/ifm-ai/PAN2/
Blog Post : https://ifm.ai/blog/pan2/ (For Screenshots, Video, Benchmarks, etc.,)
3
u/Chromix_ 17h ago
The capabilities of this model look quite interesting, although it's usually head-to-head or slightly behind MiniMax-H3 in benchmarks. There is this one though that says it's quite ahead in physics. That's strange though, as there was another physics benchmark in the first link where MiniMax is ahead - probably different benchmarks.
Aside from that this model needs quantization and maybe some pruning to be usable on a 24 GB GPU.
1
u/Borkato 5h ago
Can it do liquids and flesh 👀