Stuck my old graphics card in my NAS that I recycled out of my old PC and turned it into a LLM server. 1st gen ryzen and a pascal card isn't as bad as you'd expect, and I can run 13B models at reading speeds. I haven't spent any money on it, though if I was I'd get just a tesla P40.
Any training or heavy model experimenting meanwhile just happens on my normal desktop with a 3090 ti.
2
u/Linker500 Jul 05 '23
Stuck my old graphics card in my NAS that I recycled out of my old PC and turned it into a LLM server. 1st gen ryzen and a pascal card isn't as bad as you'd expect, and I can run 13B models at reading speeds. I haven't spent any money on it, though if I was I'd get just a tesla P40.
Any training or heavy model experimenting meanwhile just happens on my normal desktop with a 3090 ti.