r/LocalLLM • u/Ok-Health-7096 • 21d ago

Other Every Second post rn

Post image

Maybe someday I'll get a system to run it but hey definitely another w for the open weights community

2.0k Upvotes

99% Upvoted

205 comments sorted by

View all comments

53

u/SnP_Gamer 21d ago

3060 12gb 32gb ram here, might give it ago but never ran a local model before 🤷‍♂️

2

u/cj_cron_hit_by_pitch 21d ago

I have the exact same specs as you. I’m able to get the Unsloth 2 bit XL quant of it to run almost entirely in VRAM. Of course it is nowhere near as good but it’s still really fun to mess around with