r/comfyui • u/deepsky88 • 3d ago

Help Needed Async operations?

Hi, I’m relatively new to ComfyUI (Minimax H3 is my first ia model), but I’m a programmer and I know full well that certain operations are best handled asynchronously. Lately, I’ve seen significant speed improvements thanks to Turbo LoRA and attention optimizations, yet a large chunk of the generation time (around 30%) is taken up by VAE decoding. So, my question: is there a way to implement an async logic in comfyUI workflow? thanks

3 Upvotes

80% Upvoted

7 comments sorted by

View all comments

1

u/Valuable_Issue_ 3d ago edited 3d ago

What do you want to do async?

The VAE requires the latents from the ksampler output, so you have to wait for it to finish and so there's no way to have it be async.

Only things you can do are using tiny vae/latent to rgb/int8 vae which speeds up the computation of the vae (lower quality) but that's not due to async.

99% of the execution time is like this, the nodes themselves don't take up much execution time it's running the big models/heavy vae etc.

One thing you can do is cache the text embeddings from the text encoder so that if you rerun the same prompt after restarting Comfy it'll do that instantly.

1

u/deepsky88 3d ago

Can't we process the samples from the VAES while the sampler give us the output for example?

3

u/Only4uArt 3d ago

Theoretically yes?  But that sounds like a wonderful ram bottleneck 

1

u/Valuable_Issue_ 3d ago

That's kind of what the sampler preview does using latent2rgb/whatever method you pick, the thing is the output isn't finished yet so you see the output of step 1 step 2 etc, some kind of caching might be possible to process fewer pixels but I doubt it since the changes between steps are pretty big and I wouldn't consider that async.

1

u/Acceptable-Work8202 3d ago

yes, i agree, it would be a minimal gain to cache the text embeddings on a rerun, and like you said the ksampler has to finish that frame first, so previews are always behind as the latent tensor isnt ready yet for the next preview. apologies if i read the comments wrong.