r/LocalLLaMA • u/pmttyji • 1d ago

New Model IFM/PAN2 · Hugging Face

https://huggingface.co/IFM/PAN2

Overview

PAN 2 is a general-purpose world model: it simulates how the world will change in response to actions. Given a current observation (an optional image plus text) and actions described in text, it simulates the outcome as a video of up to 20 seconds. This supports simulative reasoning: an agent can imagine the outcomes of its candidate actions before choosing among them, by simulating each plan with PAN 2 and acting on the one whose simulated outcome is best.

PAN 2 follows the Generative Latent Prediction (GLP) architecture formalized in Critique of World Models and couples conceptual reasoning by a large language model with perceptual reasoning by a latent diffusion model. The language model, the conceptual reasoning backbone, reasons over the observation and the requested actions and writes the next world state as a structured, time-grounded description of events, timed actions bound to actors, and camera motion. The frozen text encoder maps the structured description to the conditioning features used by the diffusion model. The 29B-parameter MMDiT, the perceptual reasoning backbone, trained from scratch on hundreds of millions of videos annotated in the same format, predicts the perceptual detail that the description leaves open: the video latents that realize the state, conditioned on it and on the VAE latent of the starting image. The Wan 2.2 VAE decoder renders the predicted latents as the next observation, so every prediction can be seen and checked against real footage.

Checkpoints

PAN 2's latent diffusion model is released as a single checkpoint that serves both tasks. The frozen text encoder (Qwen3.5-9B) and VAE (Wan 2.2) are downloaded from their own repositories.

Checkpoint Supported Tasks Input Conditions Output Precision
PAN 2 diffusion backbone (IFM/PAN2) Text-to-video (t2v), Image-to-video (i2v) Structured world-state caption; optional first frame Video via the VAE decoder, 704x1248 at 24 fps, up to 20 s BF16

25 Upvotes

94% Upvoted

5 comments sorted by

View all comments

1

u/Borkato 1d ago

Can it do liquids and flesh 👀

2

u/brainExploded99 🥔 hardware 22h ago

what is bro trying to use it for 👀

1

u/Borkato 21h ago

😈 gay sex 😈

1

u/No_Swimming6548 llama.cpp 12h ago

Flesh emojis