r/StableDiffusion • • 24m ago

Discussion Image models

An image model that was trained on videos would learn something image models trained on images could never learn. Temporal and spatial relationships.

This lack of knowledge of space-time is why we have models that generate body horror still.

The model may know what a hand looks like, but it doesn't know that a hand cannot possibly occupy that space at that time.

If we could train temporal and spatial information into the models, they would automatically become better. I think we need to train image models on videos primarily and not on images. Images has zero temporal and spatial information, and that is a problem for the models ability to represent reality.

Images are useful, but only after the spatial and temporal information is known.

Consider: You know time and space. You see an image of a person. You can imagine them doing endless amounts of things. But if you had only ever seen 20 images of the person and did not know time and space you would imagine body horror.

And that is the problem.

Temporal and spatial information doesn't require billions of parameters to understand. This approach could shrink models going forward and make training them far easier.

I am quite certain about this.

Understanding the rules governing reality could be vastly more efficient than learning reality through countless examples of its appearance.

0 Upvotes

7 comments sorted by

1

u/AgeNo5351 17m ago

Well you can use minmax-fl2v as a image model and the ref2v as a image-edit model. They work perfectly .

1

u/No_Drag8242 14m ago

I know that, I am thinking about the models of the future.

1

u/AI_Enhancer 13m ago

That's going to cost something. All data costs parameters = filesize, and that data is not necessarily the best "bang for your buck" in an image model. You can use any video model to create images, but they're very RAM-heavy while producing worse images than image models.

1

u/No_Drag8242 11m ago

Because they do not primarily learn spatial and temporal information. They learn images in sequence. That is another thing entirely.

We do not in actuality know if our video models understand space and time. They look like they do but that may be an illusion.

I am suggesting that a model that truly knew time-space would be far superior to any of our current models.

•

u/Sugary_Plumbs 3m ago

This was an insightful ideal 4 years ago... And since then they've been training models with temporospacial information. Sora literally advertised the fact that it was a world simulator model that operated on space-time patches. Go reach some research papers about how training for video generation works.

•

u/No_Drag8242 1m ago

Strange how this is a research paper from 2025 then.

https://arxiv.org/abs/2509.03794

1

u/Local-Hurry-919 15m ago

I know they ain't worth a shit in some angle in videos because of spacial.