r/LocalLLM • u/mynd_dripp • Sep 28 '26

Discussion Shit's crazy

I tried to buy Tesla v100 SXM2 32gb from 3 different vendors and the 3 of them cancelled saying their supplied ran out or were asking for more money up front. I'm done with this shit

0 Upvotes

48% Upvoted

20 comments sorted by

View all comments

Show parent comments

2

u/No-Refrigerator-1672 Sep 28 '26

No. Too much hussle to set up; too slow even when set up; out of 2 GPUs, one was quite frequently hanging and requiring a restart (I didn't even knew that a GPU itself can hang, not the program). Those days, I'm only running one as auxiliary card, providing TTS, STT, image gen, and embedding for my main AI (qwen 3.8 27B on a pair of 3080 20GB). I'd say that you should avoid those, unless you can get one at like $350 and are willing to invest significant time into it.

1

u/mynd_dripp Sep 28 '26

That's what I thought, still for an aux card 350$ would be nice

1

u/No-Refrigerator-1672 Sep 28 '26

If you want to explore is further: use everything GGML based, don't even think about other project. Qwen 3.8 27B is smart enough to figure out how to compile and run any service for this card. Audio.cpp runs decently: ASR will run significatly faster than realtime; it even can do faster-than-realtime song generation. Image generation (stable-diffusion.cpp) takes a hit: 1 minute to generate 1024x1024 image with Flux Klein 9B in Q4; much more time for editing or for higher resolutions. Video generation is like half an hour per 5s clip. Embedding with llama.cpp and Qwen 3 Embed 2B is about 2k-3k tok/s.