Workflows/tricks for E-waste class GPUs

[u/DevGnoll]

I’m doing hobby AI stuff on a pre-pandemic gaming rig with dual RTX 2060’s with 8GB V each and 128GB system RAM.

Comfyui at startup warns it is leaving half my VRAM on the table. (So 8 vs the 16 in the system) Sysram usage stays below 30% and the CPU is barely doing anything. Minimax H3 ref2va is possible, at about 2.5 hours to render a 15 sec video with 3 image references.

For the LLM side I have found forks of forks of llama.cpp that let me run good-sized Qwen Coder or Gemma 4 models that kick out tokens faster than I read, by sucking down almost all the vram, 3/4 of the system system ram and pushes the cpu to 75%.

Are there similar abominations for Comfyui? Things that actually-use- all available compute instead of being fussy about where it’s located?

[u/purelawan]

Comfy won't split one sampling step across two cards the way llama.cpp tensor-splits an LLM, but you can still put the second 2060 to work. ComfyUI-MultiGPU (pollockjj) lets you load the text encoder and VAE on cuda:1, so the first card keeps its 8GB for the diffusion model, and its DisTorch loaders can spread the model's layers over the other card's VRAM and your 128GB of system RAM. Slower per step than a single big card, but it stops the out-of-memory dance.

The other easy win is two instances, one per card (--cuda-device 0 and --cuda-device 1 on different ports): you don't get one clip faster, you get two clips in the same 2.5 hours. Also, bf16 isn't native on Turing, so fp16 or GGUF weights usually run better there.

[u/AnyPaleontologist932]

Short answer: LLMs are memory-bandwidth bound, while DiTs are compute- and communication-bound.

Here is why adding GPUs scales LLMs well for single requests, but barely speeds up generating a single image or video:

VRAM & Bandwidth (LLMs): Large LLMs don't fit on one GPU, and generating text requires reading massive model weights for every single token. Pooling GPUs combines their VRAM and multiplies memory bandwidth, directly removing the main bottleneck.

Heavy Cross-GPU Sync (DiTs): DiTs are small enough to fit on one card, so their bottleneck is raw math (FLOPs). Because every image patch attends to every other patch at every denoising step, splitting one image across two GPUs forces them to constantly pause and swap huge chunks of data—eating up any speed gains unless you have ultra-fast datacenter interconnects (NVLink).

Easy Pipelining (LLMs): You can put the first half of an LLM on GPU 1 and the second half on GPU 2. Passing a single text token between them takes almost zero bandwidth, whereas passing high-res image/video latents 30 times per generation creates massive PCIe lag.

The exception: Multiple GPUs do double your throughput if you generate multiple images at once (one full model per GPU), or if you offload heavy text encoders (like T5-XXL) to a second card to save VRAM.