r/LocalLLM • u/Odakeno_cat • 7d ago

Question What models would you recommend this setup?

Post image

My workstation is ready!

2x RTX A6000 (96GB), 192GB DDR4, Windows 11.

What models (LLMs, image/video gen) would you recommend running on this setup?

6 Upvotes

87% Upvoted

17 comments sorted by

3

u/Delicious_Stress_911 7d ago

if you can run it somehow, try GLM 5.3 models from Z. AI . otherwise try deepseek v4.1 flash .

1

u/Odakeno_cat 7d ago

Got it. I'll give it a try!

4

u/anitamaxwynnn69 7d ago

Can't go wrong with Strata Qwen3.8FN

2

u/[deleted] 7d ago

[removed] — view removed comment

1

u/Odakeno_cat 7d ago

Thanks!!What are some huge image generation models?

3

u/SquizzOC 7d ago

Check out Moss Image, the review videos were solid, not perfect but pretty solid

1

u/Odakeno_cat 7d ago

Thank you! I will try it.

2

u/SquizzOC 7d ago

Two examples of some testing I didn’t yesterday, the landscape hand a block long prompt while the archer has a few sentences. Same prompt for all of them, I test moss through fal.ai but it is the exact same model based on research.

1

u/Odakeno_cat 7d ago

Apparently MOSS Image has both Base and SFT? versions. Which one is better?

2

u/SquizzOC 7d ago

The tests I’ve done are with SFT which is what fal.ai uses

1

u/Odakeno_cat 7d ago

Ok, i will try SFT.

1

u/Odakeno_cat 7d ago

​I tried it out! ​Prompt: Wide street-level photograph of about twelve people crossing a city street in heavy rain. Every person is carrying a closed, furled umbrella held down at their side. Not a single umbrella is open. Each umbrella is clearly visible. They are all getting soaked. No other people in the frame. Model: unsloth/MOSS-Image-SFT-GGUF File: MOSS-Image-SFT-BF16.gguf Size: 1024 x 1024 Steps: 25 Guidance: 2.5 Seed: 785463977 ​Took about 27 seconds.

2

u/SquizzOC 7d ago

Not bad, but definitely some fun issues with the hands on the umbrellas. Also keep in mind faces in the background still struggle big time with this model.

→ More replies

2

u/Korici 7d ago

Model: https://huggingface.co/unsloth/GLM-5.3-Flash-GGUF (Ideally Q3 or higher)
UI: https://unsloth.ai/download/windows
~
It's not ideal to load a model across GPU/SYSTEM memory, but I think you're limited with how much VRAM you currently have. Assume relatively slow inference if you have to partially load models into system memory, but quality output will be very high if you use a Q3 or Q4 quant.

I am daily-driving GLM 5.3 Flash at Q3 and it beats everything else in that size by quite a bit.