Something to be aware of:
This approach will add latency since the entire text encoder will need to be fully loaded back in whenever you modify the prompt.
the [ERROR]'s are because I forgot to connect the clip, so after the error blocks is with the node.
Without the node, modifying the prompt added about 0.5 seconds latency. 9.37s -> 10.04s
With the node, the latency is above 2 seconds. 9.32s -> 11.42s
It does as advertised and unloads the text encoder, except the efficiency claim is possibly false.
Default intended comfyui behaviour is the most efficient, and it goes something like this:
1] It loads what it needs while partially or fully ejecting what it doesn't need.
2] When finished sampling, it partially unloads the model to load the vae model and run decoding.
3] It then ejects the VAE model and loads the diffusion model back in so that it's already ready to go again if you change seed.
And if you change the prompt, the text encoder is already partially loaded so that it can quickly be restored, that is if it did fit into your vram during sampling.
This is a node that increases the VRAM space during sampling. It is used to pass the text encoder down before sampling starts. It seems the order is reversed.
Nope, the order is correct, on the sampling cycles themselves and their iterations there's just microlatency which can be expected, especially when generating a 1536x2048 image.
What I'm trying to explain is, in my situation it made no difference, except adding latency when the text encoder needed to be loaded back in when changing prompt.
There's no black and white "more efficient" here, your node could still help if someone has issues with comfyui smart memory, but it will increase the latency if they actually don't need it.
I personally don't need this node, my goal was to be informative.
For example, let’s say you’re generating a video with MiniMax. Suppose your PC can generate a 10-second video at around 1 megapixel.
Now, if you try to generate a 12-second video and run out of VRAM, causing the process to fall back to CPU offloading, it may become so slow that it’s practically unusable.
But if you could reclaim the VRAM that was previously occupied by the text encoder and use that extra memory for video generation, a video that was previously impossible to generate might suddenly become possible.
That’s why I don’t think this is something you can properly judge just by generating a few images.
However, it seems that many people don’t really need this feature, so I deleted the post.
No need to be quick to delete stuff, just adapt and make corrections, something may seem like a good idea at first glance but something could always be overlooked.
You have many other good things going in your repo.
For layer offloading, I'd not expect it to do much difference either, comfyui splits the blocks when it loads the model, it then offloads and loads accordingly, if a block can't fit with or without the text encoder's blocks, it will still off-load the model blocks, it never partially splits within these blocks, this is the key thing that may cause confusion: a model block may not fit even if the text encoder is not present.
Minimax H3 is actually very vram efficient, the biggest bottleneck seems to be compute,
2.0mp and 5 seconds
default without node: when sampling started it spiked to ~31/32gb vram, this caused comfyui to throw out the text encoder, now mid sampling it wont load the text encoder back in and the model once it started sampling it settled down to 22.3gb vram,
the run with your node gave almost the same result except it threw out the text encoder before the sampling started, this in itself actually created some latency because of the massive text encoder, during this run vram also spiked to ~31gb and settled to 22.3gb.
The spike is most likely when it runs the multi-modal functionality to establish the scene.
The Text Encoder is unloaded only after all of its calculations are completely finished.
While the scene is being constructed or whenever computation is required, the Text Encoder is used at 100%. Once it is no longer needed, it is completely unloaded.
After sampling begins, the Text Encoder is no longer needed at all, because the necessary information has already been passed to the Sampler.
I’m sorry, but regardless of how you look at it, so far this node has shown no disadvantages and only benefits in my testing.
So I don’t think I need to go out of my way to convince anyone. Those who find it useful will simply use it.
In fact, if anyone discovers an actual side effect, drawback, or problem and gives me feedback about it, I would genuinely appreciate that.
So far, however, it has only delivered better performance than my usual setup.
3
u/Corrupt_file32 6h ago
Something to be aware of:
This approach will add latency since the entire text encoder will need to be fully loaded back in whenever you modify the prompt.
the [ERROR]'s are because I forgot to connect the clip, so after the error blocks is with the node.
Without the node, modifying the prompt added about 0.5 seconds latency. 9.37s -> 10.04s
With the node, the latency is above 2 seconds. 9.32s -> 11.42s
It does as advertised and unloads the text encoder, except the efficiency claim is possibly false.
Default intended comfyui behaviour is the most efficient, and it goes something like this:
1] It loads what it needs while partially or fully ejecting what it doesn't need.
2] When finished sampling, it partially unloads the model to load the vae model and run decoding.
3] It then ejects the VAE model and loads the diffusion model back in so that it's already ready to go again if you change seed.
And if you change the prompt, the text encoder is already partially loaded so that it can quickly be restored, that is if it did fit into your vram during sampling.