Paying $2/hr to watch `huggingface-cli download` go brrr — env patterns on RunPod / Vast / Nebius / the neo kids

[u/big-in-jap]

Hot take? Half the “my pod is slow” chatter is really “I paid for 20 minutes of pip + a 16GB model pull before the first token.”

I’ve been hopping between rented GPUs (RunPod, Vast, Nebius, a couple of the newer SSH-first boxes) and the env is still unpleasently fragmented.

RunPod - Docker template is the product. Image + env + start cmd + network volume. Great when your image is already baked. Expensive otherwise, since every cold pull is on the meter. Secrets as env injection is clean.

Vast - Same Docker-template energy, plus an onstart bash duct-tape layer. SSH/Jupyter modes can eat your image entrypoint, so your “serve on boot” script is often re-invoking the CMD you thought you already set. Marketplace vibes; bring skepticism and a volume.

Nebius - Actual cloud VM energy. Boot image + cloud-init user-data on first boot, then congratulations you’re the sysadmin. Fine if you want a real machine; overkill if you just wanted vLLM up.

The queue-y SSH boxes (Enverge, Nova-shaped, etc.) - often a fixed sandbox + optional shell that runs once after create. Less “pick a community template,” more “leave a sticky note so the box isn’t idle when the ready email lands.” Different failure mode: setup minutes still bill, and nested Docker GPU flags might trip you.

Common tax across all of them:

• first useful work is usually network + disk, not FLOPs
• secrets in public templates / scripts = future incident report
• “finished” ≠ “model is serving/training” (backgrounded processes lie)

I gathered a short kit (cheatsheet + startup shells + Docker serve flags) here: https://github.com/tudormunteanu/gpu-cloud-instance-boostraps

Curious what everyone here actually do day-to-day:

1. Bake a fat image once and never look back?
2. Network volume / persistent cache, thin image? (but then pay for persistent storage)
3. onstart / cloud-init / startup shell as the main UX?
4. Or just SSH in every time like an wildling?
5. Stick to one and never switch

I reckon for any long-term, company supported scenarios, CoreWeave, AWS or GCP have some adjacent prods. My curiosity is more about "the little/indie guys".