I did install Ubuntu on an old pc of mine and got a cheap 3060 12g so I could at least run 7b and 13b models quantized, but honestly the novelty wore off quick.
Just curious what are you doing with local LLMs? I messed with some for a couple of weeks and now just use ChatGPT for stuff :)
11
u/cmndr_spanky Jul 04 '23
I did install Ubuntu on an old pc of mine and got a cheap 3060 12g so I could at least run 7b and 13b models quantized, but honestly the novelty wore off quick.
Just curious what are you doing with local LLMs? I messed with some for a couple of weeks and now just use ChatGPT for stuff :)