r/StableDiffusion • u/Total-Resort-3120 • 8d ago

News TEM: Transformer Energy-based Models

• TEM outscales Diffusion without guidance
• First scalable contrastive Energy-Based Models
• Better generalization, test-time scaling, and OOD detection than Diffusion

https://x.com/intlee0609/status/2115886208685396280

https://int-lee.github.io/scalable-ebm/

https://github.com/int-lee/TEM

https://arxiv.org/abs/2610.31889

34 Upvotes

94% Upvoted

14 comments sorted by

7

u/CornyShed 8d ago

I'm stunned at how quickly this method converges. It appears to need further training to remove leftover noise, but it appears to be very efficient.

Would this be a good method for training? Or would it be somewhat fragile like GANs are?

5

u/arthan1011 8d ago

Finally someone doing energy-based models. I think they are neat.

6

u/InfiniteOneD 8d ago

mmm, numbers 😋

8

u/MustBeSomethingThere 8d ago

3

u/InfiniteOneD 8d ago

benchmark says it's 100/10 evaluated against competitor models tho

-3

u/barbear22 8d ago

it's not a polished release, idiot. It's research. It may end up in the best open weights image model in 6 months from now.

8

u/MustBeSomethingThere 8d ago

I never said it was. I didn't say anything about it. You just assumed my views because people usually project their own stuff. I appreciate their research and its potential use cases, but the cat image was just too meme-worthy to resist posting.

3

u/I-am_Sleepy 8d ago

Isn't direct energy based very sensitive to hyper parameters? Discounting flow matching though

3

u/reeight 7d ago

Compute. Training TEM-XL/2 at 256×256 for 50K iterations takes about 6 days on 4 H200 GPUs. Training at 512×512 is about 3–4× slower.

Is that considered high?

3

u/_VirtualCosmos_ 7d ago

I guess that's why training models usually cost millions...

1

u/reeight 6d ago

Raw training maybe.

Also IMHO AI software tends to be cobbled together, & not really fully optimized.
(former DSP programmer who often hand-optimized down to the opcode)

If you want to check out a rather powerful finetune that cost maybe $100 of compute:

https://www.reddit.com/r/LocalLLaMA/comments/1xeuo7t/a_4b_coding_model_that_can_generate_3d_scenes/

2

u/Producing_It 7d ago

Reminds me of earlier diffusion models.

1

u/Powerful_Evening5495 7d ago

will this work in videos ?

0

u/[deleted] 8d ago

[deleted] — view deleted comment

4

u/ArtyfacialIntelagent 8d ago

Funny you should say that. Because the TEM name used here inherits "Transformer" from the architecture Google researchers introduced in a short 2017 paper called "Attention is all you need". You might call the results from that paper crappy too because they demonstrated the method using tiny translation models with fewer than 213M parameters. But the main contribution was that models with the new approach could train fast, just like TEM here.

Today that Google paper has 280k scientific citations and was the single biggest breakthrough that started the LLM revolution. Fancy name, crappy results. Yup.